System

Abstract

A system is provided.SOLUTION: A system, comprising: means for receiving administrative paper-based or electronic-based documents; means for extracting text from the received documents using a generation AI and converting the text to structured documents; means for sending the structured documents converted by the generation AI to a data cleansing module to correct incomplete or erroneous documents; means for converting the cleansed documents to a standard format; means for visually displaying the standard formatted documents; and means for storing and sharing the final documents with other systems and users.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to improve the efficiency and transparency of government services in a digital society, it is essential to digitize the data held by government agencies. However, because much of the data held by government agencies is currently in paper or PDF format, it is difficult to integrate and share the data. This is a significant barrier to building a data integration platform. Furthermore, manually converting non-digital data into a digital format requires a great deal of time and effort. Furthermore, maintaining the quality and consistency of the converted data is also a difficult challenge. There is a need for a method to resolve these issues and efficiently and accurately digitize data and convert it into a standard format. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. First, it provides a means for receiving data held by government agencies in paper or electronic file format. This data is uploaded to a server from a user or device and temporarily stored by the server. Next, it provides a means for extracting text from the received data using a generation AI and converting it into structured data. This generation AI incorporates an OCR engine and natural language processing technology to automatically extract text and map fields. Furthermore, it provides a means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data. This data cleansing maintains the integrity and accuracy of the data. Next, it provides a means for converting the cleansed data into a standard format. This standard format facilitates data integration with different systems. Finally, it provides a means for visually displaying the data converted into the standard format and a means for saving the final data and sharing it with other systems or users. This makes it easier to understand and use data, thereby improving the efficiency and transparency of government services.

[0006] "Data" refers to recorded information, including information held by government agencies in paper form or electronic file format.

[0007] "Generative AI" refers to artificial intelligence technologies for automating text extraction and data structuring, including OCR engines and natural language processing technologies.

[0008] An "OCR engine" is an optical character recognition technology, a software engine that extracts text information from scanned paper documents or image files.

[0009] "Natural language processing technology" is a computer science technology used to analyze text data and understand its meaning and grammatical structure.

[0010] "Data cleansing" is the process of identifying, correcting, removing, or completing incomplete or erroneous data to ensure accuracy and integrity.

[0011] A "standard format" is a common data format agreed upon to enable multiple systems and organizations to handle data consistently.

[0012] "Government agency" refers to an agency or department of a government that provides public services and operates under statutes and regulations.

[0013] "Structured data" refers to data that is organized into specific formats and fields, making it easy to parse and query. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The system of the present invention aims to improve the efficiency of data sharing infrastructure by converting data held by government agencies in paper or electronic file format into a format suitable for digital use and formatting it into a standard format.

[0036] Entering data

[0037] server

[0038] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[0039] Specific examples

[0040] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0041] Data analysis with generative AI

[0042] server

[0043] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[0044] Specific examples

[0045] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[0046] Data Cleansing

[0047] server

[0048] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[0049] Specific examples

[0050] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[0051] Converting data into a standard format

[0052] server

[0053] Consistently format the cleansed data into a predefined standard format.

[0054] Specific examples

[0055] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[0056] Data Visualization

[0057] server

[0058] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[0059] Specific examples

[0060] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[0061] Data storage and sharing

[0062] server

[0063] The final formatted data is securely stored and shared with other systems and users.

[0064] Specific examples

[0065] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[0066] Through this system, non-digital data stored on paper or in PDF format can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The server receives data in the form of paper or PDF files sent from the device or user. The server receives files uploaded via HTTP requests and saves them in a temporary directory. It also checks the format of the uploaded file and prepares it for passing to the appropriate analysis module.

[0070] Step 2:

[0071] The server invokes the generative AI engine to extract text from the received data and convert it into structured data. It also uses the OCR engine to extract text from paper documents and image files, analyzes the text using natural language processing technology, and maps it to specific fields. The analysis results are then saved in a temporary file or database.

[0072] Step 3:

[0073] The server sends the analyzed data output by the generative AI engine to the data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. If there is duplicate data, it is removed or merged, and if there are missing values, it is imputed.

[0074] Step 4:

[0075] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The server then stores the converted data.

[0076] Step 5:

[0077] The server sends the transformed data to the visualization module for easy-to-understand display, passes the structured data to the visualization engine, which displays the data in an appropriate format such as a graph or map, and saves the generated visualization for use in a web portal.

[0078] Step 6:

[0079] The final formatted data is securely stored and shared with other systems and users. The server persists the data stored in the database and shares it with other local governments and related organizations via API as needed. It also allows users to access their own data through a web portal.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Modern government agencies have a lot of data in paper or PDF format, and converting this data into a format suitable for digital use requires a large amount of manual work, which is inefficient. It is also difficult to convert data in different formats into a unified standard format, making data integration and centralized management difficult. This hinders efforts to improve the efficiency and transparency of government services.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for receiving and temporarily storing data in paper or electronic file format sent from a user or terminal, means for passing the data received by the server to a generation AI engine and performing text extraction and data structuring using an OCR engine and natural language processing technology, means for sending the structured data converted by the generation AI engine to a data cleansing module and correcting incomplete or erroneous data, means for converting the cleansed data into a predefined standard format, means for visually displaying the data converted into the standard format using a visualization engine, and means for securely storing the final formatted data and sharing it with other systems and users. This makes it possible to efficiently digitize data in different formats and unify them into a standard format.

[0085] A "user" is a person or organization that accesses the system and submits or retrieves data.

[0086] A "terminal" is a device operated by a user, such as a computer, smartphone, or tablet.

[0087] A "server" is a computer system for receiving, processing, storing, and sharing data.

[0088] "Paper media" refers to papers or documents containing information printed on paper.

[0089] An "electronic file format" is a document stored in a digital format, such as a PDF or Word file.

[0090] "Temporary storage" refers to the operation of temporarily storing data before final processing.

[0091] A "generative AI engine" is an artificial intelligence engine that automatically performs text extraction and data structuring.

[0092] An "OCR engine" is a system that uses optical character recognition technology to extract text from image data.

[0093] "Natural language processing technology" is an artificial intelligence technology for understanding and analyzing natural language text.

[0094] "Data structuring" is the process of mapping extracted text to specific fields to make it easier to process and use in a system.

[0095] "Data Cleansing Module" means a software component for correcting and completing incomplete or erroneous data.

[0096] "Standard format" refers to a standard or format for converting different data into a unified format.

[0097] A "visualization engine" is a system for displaying data in a visually easy-to-understand manner.

[0098] "Visual display" refers to the act of visualizing data using text, graphs, maps, etc.

[0099] "Storage" is the act of storing data in a physical or virtual storage device.

[0100] "Sharing" is the act of making data accessible to other systems or users.

[0101] The present invention is a system that converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, thereby streamlining the data sharing infrastructure. This system includes the following components.

[0102] Entering data

[0103] Users can send data in paper or PDF file format from their own devices by logging in to the city hall's web portal and using the file upload function. The server receives the data sent from users and devices and temporarily stores it.

[0104] Specific examples

[0105] A user accesses the city hall web portal and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0106] Data analysis with generative AI

[0107] The server then passes the temporarily stored data to a generative AI engine, which incorporates an OCR engine and natural language processing technology. The OCR engine extracts the text from the PDF and converts it into structured data using natural language processing technology.

[0108] Specific examples

[0109] The generation AI reads the text in the PDF and extracts it using OCR technology, then uses natural language processing technology to analyze information such as name, address, and issue date, and converts it into JSON format data.

[0110] Data Cleansing

[0111] The server sends the analysis data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data from the analysis data.

[0112] Specific examples

[0113] A data cleansing module is used to remove inappropriate strings from name fields and to impute missing values ​​in address data from a postal code database.

[0114] Converting data into a standard format

[0115] The server standardizes the cleansed data into a standard format.

[0116] Specific examples

[0117] The JSON format data will be converted into a nationwide standard resident information format, making it easily usable by other local governments and institutions.

[0118] Data Visualization

[0119] The server sends the data, formatted in a standard format, to a visualization engine for visual display.

[0120] Specific examples

[0121] Resident data is plotted on a map to visually display the coverage area of ​​government services and the distribution of residents. This data can be used as a dashboard by users and other government agencies.

[0122] Data storage and sharing

[0123] The server securely stores the final formatted data and shares it with other systems and users, allowing users to access their own data by setting access rights.

[0124] Specific examples

[0125] Store the data in a standardized format in a city database, share the information with other departments and related agencies, and allow users to access their resident data through a web portal.

[0126] Example prompts for generative AI models

[0127] An example of a prompt is, "Upload a PDF file, extract the text data from it, and structure it into each field (name, address, publication date, etc.). Convert this into JSON format, cleanse the data, and then unify it into a standard format."

[0128] Through this system, non-digital data such as paper documents and PDF files can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Step 1:

[0131] A user logs into the city hall web portal using a terminal and performs an operation to upload data in paper or PDF file format. Specifically, the user selects a file and clicks the upload button. This operation sends the data in paper or PDF file format to the server. The input is the paper or PDF file provided by the user, and the output is a file stored on the server.

[0132] Step 2:

[0133] The server saves the data sent by the user in a temporary folder. The server sends a receipt confirmation message to the user to confirm receipt of the data. The input is the file sent by the user, and the output is the file stored in the temporary folder.

[0134] Step 3:

[0135] The server passes the temporarily stored data to the generative AI engine. The generative AI engine uses an OCR engine to extract text from the PDF and uses natural language processing technology to structure the data. Specifically, it analyzes the extracted text and converts information such as name, address, and publication date into JSON format. The input is the file stored in the temporary storage folder, and the output is structured data in JSON format.

[0136] Step 4:

[0137] The server sends the JSON-formatted structured data output by the generation AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data and completes missing values. Specifically, it removes inappropriate strings from name fields and completes missing values ​​in address data from a postal code database. The input is JSON-formatted structured data, and the output is the corrected and completed structured data.

[0138] Step 5:

[0139] The server converts the cleansed structured data into a standard format. Specifically, it converts the cleansed JSON data into a nationwide standard resident information format. This makes it easy for other local governments and organizations to use the data. The input is corrected and supplemented structured data, and the output is data unified in a standard format.

[0140] Step 6:

[0141] The server sends the data converted into a standard format to the visualization engine. The visualization engine converts the data into a format that is easy to display visually. Specifically, it plots the resident data on a map and visually displays the coverage area of ​​administrative services and the distribution of residents. The input is data standardized in a standard format, and the output is visually displayed data.

[0142] Step 7:

[0143] The server securely stores the final formatted data and shares it with other systems and users. Specifically, the data is stored in the city's database and access permissions are set to make it available to other departments and related organizations. Users can also access their own resident data through a web portal. The input is data standardized in a standard format, and the output is securely stored data and shared data.

[0144] (Application example 1)

[0145] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0146] The challenge is to efficiently digitize paper and electronic file data in government agencies and to use that data to provide effective customer support in virtual stores. In particular, there is a need to improve the user experience by quickly and accurately extracting necessary information from uploaded documents and proposing products and services that are suitable for each customer.

[0147] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0148] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems or users; means for extracting necessary information from files uploaded by customers and proposing products and services; and means for visually displaying product information in a virtual store. This enables efficient digitization of government agency data and improved customer support in the virtual store.

[0149] An "administrative agency" is a government organization established for the public interest, including local governments and central government agencies.

[0150] "Paper media" means physical documents in which information is printed or written on paper.

[0151] "Electronic file format" means a digital file format that is stored and transmitted electronically, including formats such as PDFs and Word documents.

[0152] "Means for receiving" refers to a method or device for taking in data on paper or in electronic file format from outside and temporarily storing it.

[0153] "Generative AI" refers to systems or programs that use artificial intelligence technology to analyze input data and perform specific tasks.

[0154] "Text extraction methods" are techniques and devices used to detect and extract textual information from non-text data such as images and PDF files.

[0155] "Structured data" is data that is organized according to a specific format or schema, and is in a form that makes it easy to search and manipulate.

[0156] A "data cleansing module" is software or a system for detecting and correcting or removing inaccurate or incomplete data.

[0157] A "standard format" is a data format that is widely accepted and used by multiple systems and institutions, and is a format defined to ensure compatibility.

[0158] "Visual display means" refers to techniques and devices that visually present information in a form that is easy for humans to understand.

[0159] "Storage and sharing means" refers to methods and devices for securely storing data and sharing information with other systems and users as needed.

[0160] "Customer" means a user who visits the virtual store to use or purchase goods or services.

[0161] An "uploading means" is a method or device that allows a user to transfer files or data from their device to a server.

[0162] "Means for extracting information" refers to the technology or devices used to extract necessary information from uploaded files or data.

[0163] The "means for proposing products and services" refers to a technology or device for recommending related products and services to customers based on the extracted information.

[0164] A "virtual store" is an online store or service platform that exists on the Internet and is independent of a physical location.

[0165] The present invention is a system for digitizing paper or electronic file data held by government agencies and converting it into a standard format, and further applying this technology to customer support within a virtual store. Specific embodiments for implementing the present invention are described below.

[0166] 1. Hardware Configuration

[0167] The main hardware for implementing the present invention includes:

[0168] server

[0169] Smartphone (iOS / ANDROID (registered trademark))

[0170] Smart Glasses

[0171] Head-mounted displays (e.g., Oculus Rift)

[0172] 2. Software Configuration

[0173] The main software used includes:

[0174] OCR engine (e.g., Google® Cloud Vision API)

[0175] Natural language processing technology (e.g., GPT-3 (registered trademark), BERT)

[0176] Data cleansing module (either in-house or via "Trifacta" etc.)

[0177] Visualization engines (e.g. D3.js, Google Charts)

[0178] 3. Data Entry

[0179] Using a smartphone or head-mounted display, users can access the city hall's web portal or virtual store and upload scanned paper documents or PDF files, such as a scanned PDF of a resident registration card.

[0180] 4. Data analysis using generative AI

[0181] The server receives the uploaded file and temporarily stores it. It then passes the file to a generation AI engine, which uses OCR and natural language processing to extract text and structure the data. For example, it extracts text from a PDF and converts information such as name, address, and publication date into JSON format.

[0182] 5. Data Cleansing

[0183] The analyzed data output by the generative AI engine is processed by the data cleansing module, which corrects incomplete or incorrect data to ensure data accuracy. For example, if an inappropriate string is included in a name field, it is removed, and missing values ​​in address data are filled in using postal codes.

[0184] 6. Converting data into a standard format

[0185] The cleansed data is standardized into a predefined standard format. For example, JSON data can be converted into a nationwide standard format for resident information.

[0186] 7. Data Visualization

[0187] Once the data has been converted into a standard format, it can be displayed using a visualization engine. For example, resident data can be plotted on a map to visually display the coverage area of ​​government services and the distribution of residents.

[0188] 8. Data Storage and Sharing

[0189] The final formatted data can then be securely stored and shared with other systems and users. For example, data converted into a standard format can be stored in a city database and shared with other departments and related agencies.

[0190] 9. Customer Support Applications

[0191] The system receives PDF files of product catalogs and invoices uploaded by customers and analyzes them using a generative AI engine. Based on the analyzed data, it proposes products and services in a virtual store. This information is displayed visually on a smartphone or head-mounted display. For example, it prepares JSON data including product names, model numbers, and price information to make optimal proposals to customers.

[0192] Examples and prompts

[0193] As a concrete example, the following prompt sentence is input into the generative AI model:

[0194] "Please extract the product characteristics and prices listed in this PDF document and convert them into JSON format."

[0195] "Extract the product attributes and prices from this PDF document and convert them into JSON format."

[0196] Template prompt:

[0197] "Based on the contents of the PDF document, extract product characteristics, prices, reviews, etc. and output them in JSON format, which is the standard format for virtual stores."

[0198] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[0199] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0200] Step 1:

[0201] Users access the city hall's web portal or virtual store using a smartphone or head-mounted display and upload PDF or scanned document files, which are then transferred to the server and temporarily stored.

[0202] Input: PDF files or scanned documents uploaded by the user

[0203] Output: A file temporarily saved on the server

[0204] Step 2:

[0205] The server passes the received file to the generative AI engine, which uses OCR technology to extract the text from the file. The generative AI engine uses the Google Cloud Vision API, for example, to detect the text.

[0206] Input: PDF file stored on the server

[0207] Output: Raw text data extracted by OCR

[0208] Step 3:

[0209] The server then analyzes the extracted text data using natural language processing techniques (e.g., GPT-3, BERT) and structures the data, for example, generating data mapped to specific fields such as name, address, and publication date.

[0210] Input: Raw text data extracted by OCR

[0211] Output: Structured data (e.g., JSON format) mapped to fields

[0212] Step 4:

[0213] The server sends the generated structured data to a data cleansing module (e.g., Trifacta) to correct erroneous or incomplete data, such as removing inappropriate strings and imputing missing values.

[0214] Input: Structured text data

[0215] Output: Cleansed and accurate text data

[0216] Step 5:

[0217] The server converts the cleansed data into a standard format, for example, converting resident information into a nationally standard format that can be easily used by other systems and organizations.

[0218] Input: Cleansed text data

[0219] Output: Data converted to a standard format (e.g., JSON data in a common format)

[0220] Step 6:

[0221] The server passes the converted data to a visualization engine (e.g., D3.js, Google Charts) for visualization. For example, it plots the resident data on a map to help users understand it intuitively.

[0222] Input: Data in standard format

[0223] Output: Visualized display data (e.g. dashboards and map plots)

[0224] Step 7:

[0225] The final formatted data is stored securely on a server and shared with other systems and users, for example, by storing it in a resident information database and providing access rights to share the data with related organizations.

[0226] Input: Data converted to a standard format

[0227] Output: Data stored and shared

[0228] Step 8:

[0229] Customers visit the virtual store and upload a PDF file, which the server then passes to a generative AI engine for analysis, extracting and structuring information for, for example, a product catalog or invoice.

[0230] Input: PDF file uploaded by customer

[0231] Output: Parsed and structured product information data

[0232] Step 9:

[0233] The server then proposes appropriate products and services to customers based on the structured product information, which is visually displayed on a smartphone or head-mounted display.

[0234] Input: Structured product information data

[0235] Output: Visual display of suggested products and services

[0236] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[0237] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0238] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes user emotions and uses them to process the data.

[0239] Entering data

[0240] server

[0241] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[0242] Specific examples

[0243] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0244] Data analysis with generative AI

[0245] server

[0246] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[0247] Specific examples

[0248] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[0249] Data Cleansing

[0250] server

[0251] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[0252] Specific examples

[0253] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[0254] Converting data into a standard format

[0255] server

[0256] Consistently format the cleansed data into a predefined standard format.

[0257] Specific examples

[0258] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[0259] Data Visualization

[0260] server

[0261] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[0262] Specific examples

[0263] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[0264] Data storage and sharing

[0265] server

[0266] The final formatted data is securely stored and shared with other systems and users.

[0267] Specific examples

[0268] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[0269] User Emotion Recognition

[0270] server

[0271] The emotion engine analyzes emotions based on user input and operations, and uses the results for data processing and display.

[0272] Specific examples

[0273] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[0274] Emotion-Based User Interface Adjustment

[0275] server

[0276] The emotion engine dynamically changes the display method and interface based on the user's emotions, improving the user experience.

[0277] Specific examples

[0278] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, while if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[0279] This system efficiently converts paper and PDF data into digital format, and realizes a series of processes from data standardization to visualization, storage, and sharing. Furthermore, it can recognize user emotions and process and display data accordingly, improving the user experience.

[0280] The processing flow will be explained below.

[0281] Step 1:

[0282] The server receives data sent from the device or user in the form of paper or PDF files. The received data is saved in a temporary directory, the file format is checked, and it is prepared for passing to the appropriate analysis module.

[0283] Step 2:

[0284] The server uses an OCR engine to extract text from paper documents and image files. OCR technology analyzes the text in the image and extracts it as digital text data.

[0285] Step 3:

[0286] The server then calls the generative AI engine, which uses natural language processing to analyze the extracted text. The generative AI analyzes the text, maps it to specific fields such as name, address, and publication date, and converts it into structured data in JSON format.

[0287] Step 4:

[0288] The server sends the JSON data output by the generative AI to a data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. Duplicate data is removed or merged, and missing values ​​are imputed.

[0289] Step 5:

[0290] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The converted data is then stored.

[0291] Step 6:

[0292] The server passes the converted data to a visualization module for visual display. The visualization module displays the structured data in the form of graphs, maps, etc. The generated visualizations are saved and made available to a web portal.

[0293] Step 7:

[0294] The server securely stores the final formatted data and shares it with other systems and users. The data stored in the database is provided to local governments and related organizations via API, and users can access their own data through a web portal.

[0295] Step 8:

[0296] The server uses an emotion engine to analyze emotions from user input and operations. The emotion engine analyzes user input data in real time and identifies emotions such as stress and dissatisfaction.

[0297] Step 9:

[0298] The server dynamically adjusts the data display method and user interface based on the user's emotions analyzed by the emotion engine. If the user's emotions are positive, detailed data and graphs are displayed, but if the user expresses irritation, the interface switches to a simple and intuitive one.

[0299] Step 10:

[0300] The server provides the user with an adjusted user interface to improve the user experience. The emotion engine monitors the user's emotions in real time and changes the behavior of the interface accordingly, if necessary.

[0301] In this way, the present invention enables efficient digitization and data processing, and also provides an interface that adapts to the user's emotions, thereby achieving improved efficiency and transparency in government services.

[0302] Example 2

[0303] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0304] Government agencies are required to efficiently convert paper and electronic file data into digital format, and then standardize that data into a standard format so that it can be shared with other systems and users. Another challenge with the digitalization of data is recognizing user emotions and reflecting them in system operations to improve the user experience.

[0305] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0306] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for analyzing emotions based on user input and operations and using the results for data processing and display; and means for dynamically changing the user interface based on the emotions. This enables efficient digital conversion, standardization, visual display, and sharing of data, and further improves the operability by dynamically adjusting the user interface to reflect the user's emotions.

[0307] An "administrative agency" is an organization that carries out public duties, such as the national government or local governments.

[0308] "Paper media" refers to the physical format of documents, papers, etc. printed on paper.

[0309] "Electronic file format" refers to digital data formats such as PDF, Word, and Excel.

[0310] "Means for receiving data" refers to the function by which the server acquires data sent from a user or terminal.

[0311] "Generative AI" is an engine that uses machine learning and artificial intelligence techniques to analyze data.

[0312] "Means for extracting text" refers to a function that uses generative AI to extract text information from data.

[0313] "Structured data" is data that is organized according to a particular format or structure.

[0314] A "data cleansing module" is a program for correcting incomplete or erroneous data.

[0315] A "standard format" is a format for arranging data in a unified format.

[0316] "Visual display means" refers to techniques for visualizing data in an easy-to-understand manner.

[0317] "Means for storing final data" refers to a function for safely storing data for a long period of time.

[0318] "Means for sharing with other systems and users" refers to a function that provides the stored data so that it can be used by other systems and users.

[0319] "Means for analyzing emotions" refers to technology that reads emotions from user input and operations.

[0320] "Means for dynamically changing the user interface based on emotions" is a function that changes the screen display and operation method according to the analyzed emotions.

[0321] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes the user's emotions and uses them in data processing. Specific embodiments of the system are described below.

[0322] Entering data

[0323] The server receives data sent from users or devices in the form of paper or PDF files. The received data is temporarily stored. This process uses, for example, an Amazon S3 bucket.

[0324] Examples:

[0325] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0326] Data analysis with generative AI

[0327] The server passes the received data to a generative AI engine, which extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology, using, for example, Google Cloud Vision API or Microsoft® Azure® OCR services.

[0328] Examples:

[0329] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[0330] Data Cleansing

[0331] The server sends the analysis data output by the generative AI engine to a data cleansing module. The data cleansing module includes a program for correcting incomplete or incorrect data. For example, Apache Spark (registered trademark) can be used.

[0332] Examples:

[0333] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[0334] Converting data into a standard format

[0335] The server standardizes the cleansed data into a predefined standard format, ensuring data consistency and compatibility.

[0336] Examples:

[0337] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[0338] Data Visualization

[0339] The server uses a visualization engine, such as Tableau or D3.js, to display the data converted into a standard format in an easy-to-understand format.

[0340] Examples:

[0341] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[0342] Data storage and sharing

[0343] The server securely stores the final formatted data and shares it with other systems and users. For example, the data could be stored in a city database and other departments and agencies could access it through an API.

[0344] Examples:

[0345] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[0346] User Emotion Recognition

[0347] The emotion engine analyzes emotions based on user input and operations, and uses the results for data processing and display. For example, it uses Amazon Comprehend or IBM Watson (registered trademark).

[0348] Examples:

[0349] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[0350] Emotion-Based User Interface Adjustment

[0351] Based on the user's emotions recognized by the emotion engine, the server dynamically changes the display method and interface, thereby improving the user experience.

[0352] Examples:

[0353] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, whereas if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[0354] Prompt Sentence Examples

[0355] "Extract text from PDF using OCR technology."

[0356] "Analyze the extracted text using natural language processing technology and convert it into JSON format."

[0357] "Analyze emotions based on user actions and adjust your interface accordingly."

[0358] This system efficiently converts paper and PDF data into digital format, enabling a series of processes from data standardization to visualization, storage, and sharing. It also recognizes user emotions and processes and displays data accordingly, improving the user experience.

[0359] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0360] Step 1: User Data Entry

[0361] The user accesses the city hall web portal using their own device and uploads the scanned PDF file of the resident registration card. Specifically, they click the upload button on the web portal and select the corresponding PDF file in the file selection dialog. The input here is the PDF file of the resident registration card, and the output is the completion of uploading it to the server.

[0362] Step 2: Data reception and temporary storage by the server

[0363] The server receives the uploaded PDF file and temporarily stores it in a database or file storage (e.g., Amazon S3 bucket). Specifically, it stores the uploaded file in the specified storage and records the file path. The input here is the PDF file uploaded by the user, and the output is the file path saved in the storage.

[0364] Step 3: Data analysis by server-generated AI

[0365] The server passes the saved PDF file to a generation AI engine (e.g., Google Cloud Vision API or Microsoft Azure's OCR service) to extract text and structure the data. Specifically, it sends the file path to the OCR engine and receives the extracted text data. It then uses natural language processing technology to analyze the text and convert it to JSON format. The input is the temporarily saved PDF file, and the output is the analyzed data in JSON format.

[0366] Step 4: Server-based data cleansing

[0367] The server sends the JSON data obtained from the generative AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data. Specifically, if a field contains an inappropriate string, it deletes it and fills in missing data with appropriate values ​​from an external database. The input is the JSON data obtained from the generative AI engine, and the output is the cleansed data.

[0368] Step 5: Server converts data into standard format

[0369] The server standardizes the cleansed data into a predefined standard format. Specifically, it converts the JSON data into a nationwide standard format for resident information. The input is the cleansed data, and the output is data standardized in the standard format.

[0370] Step 6: Visualizing the data on the server

[0371] The server uses a visualization engine (e.g., Tableau or D3.js) to display the data converted into a standard format in a way that is easy to understand visually. Specifically, it plots resident data on a map and displays the coverage area of ​​administrative services and resident distribution as a dashboard. The input is data in a standard format, and the output is a visualized dashboard.

[0372] Step 7: Storing and Sharing Data on a Server

[0373] The server securely stores the final formatted data and shares it with other systems and users. Specifically, it stores the data in the city's database and provides access to other departments and related organizations through APIs. The input is the visualized data, and the output is the stored data and access rights settings for sharing.

[0374] Step 8: User emotion recognition by the server

[0375] The server's emotion engine analyzes emotions based on user input and operations. Specifically, it measures response times when users operate the system and analyzes stress and dissatisfaction. The input is the user's operation log, and the output is the emotion analysis results.

[0376] Step 9: Emotion-based user interface adjustment by the server

[0377] The server dynamically changes the display method and interface based on the user's emotions recognized by the emotion engine. Specifically, if a positive emotion is recognized, the server provides an interface displaying detailed data and graphs, and if a negative emotion is recognized, the server switches to a simple and intuitive interface. The input is the emotion analysis results, and the output is the adjusted user interface.

[0378] (Application example 2)

[0379] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] While conventional data processing systems can efficiently digitize paper and electronic file data held by government agencies and convert them into standard formats, they have the problem of being unable to recognize the user's emotions during operation and dynamically adjust data processing and display accordingly. Furthermore, when it comes to customer service, it has been difficult to instantly analyze paper or PDF data and provide the results in a visually easy-to-understand format.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving data in paper or electronic file format held by an administrative agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module and correcting incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for a robot to respond to customer inquiries; means for recognizing customer emotions and presenting information based on those emotions; and means for displaying the data converted into the standard format on a display in a physical store or on digital signage. This makes it possible to recognize customer emotions and respond appropriately when dealing with customers, and to instantly analyze and visually provide data.

[0382] "Administrative agencies" are public institutions such as the national government and local governments, and are organizations responsible for citizen services and public affairs.

[0383] "Paper media" refers to documents and materials that are printed or handwritten on paper.

[0384] An "electronic file format" is a digital data file format that can be read by a computer, and generally includes PDFs and text files.

[0385] "Generative AI" refers to software or systems that use artificial intelligence technology to process and analyze data.

[0386] "Text extraction" refers to the process of extracting textual information from digital data or images.

[0387] "Structured data" is data that is organized according to specific formats and conventions, and is the format used in databases.

[0388] "Data cleansing" is the process of correcting or removing incomplete or erroneous information from data.

[0389] A "standard format" is a format in which data is standardized based on certain rules, and is used to ensure compatibility between different systems.

[0390] A "visual display" is a representation of data in a visual format, such as a graph, chart, or map, that facilitates understanding of the information.

[0391] A "customer inquiry" is an in-store or online customer requesting information about a product or service.

[0392] A "robot" is a machine or device that uses artificial intelligence and machine learning to operate autonomously and perform specific tasks.

[0393] "Emotion recognition" is a technology that analyzes a user's emotional state from their facial expressions, voice, behavioral patterns, etc.

[0394] A "display" is an electronic device for displaying visual information, such as those used in computers and digital signage.

[0395] "Digital signage" refers to an advertising or information system that uses a display to display digital content.

[0396] This invention is a system for improving the efficiency of customer service in brick-and-mortar stores and providing services based on customer emotions. This system is implemented using the following hardware and software.

[0397] System Configuration

[0398] Hardware

[0399] 1. Robot:

[0400] It has audio input and output functions.

[0401] It is equipped with a high-resolution camera that recognizes customers' facial expressions.

[0402] It has a communication module for sending data from the terminal to the server.

[0403] 2. Server:

[0404] Store and analyze the data.

[0405] It is equipped with a high-performance GPU and is used to run generative AI models.

[0406] 3. Displays and Digital Signage:

[0407] Used to display information to customers in physical stores.

[0408] software

[0409] 1. Python:

[0410] Used for data analysis and running generative AI.

[0411] 2. Flask:

[0412] Used as a server-side web application framework.

[0413] 3. Tesseract OCR:

[0414] Used to extract text from paper and electronic file formats.

[0415] 4. Transformers (BERT):

[0416] Text analysis is performed using natural language processing technology.

[0417] 5. OpenCV:

[0418] Used to analyze customer facial expressions from camera footage.

[0419] 6. Google Speech-to-Text API:

[0420] Used to convert voice input into text.

[0421] 7. Pandas and Regular Expressions:

[0422] Used to cleanse and shape data.

[0423] 8. Matplotlib:

[0424] Used for data visualization.

[0425] 9. PostgreSQL:

[0426] Used to store data.

[0427] Example of operation

[0428] Entering and saving data

[0429] The user (customer) presents the product catalogue as a paper document or PDF file to the robot, which then sends it to the server and temporarily stores it.

[0430] Data analysis and normalization

[0431] The server uses generative AI to extract text from the received data and convert it into structured data. Specifically, it uses Tesseract OCR to extract text from paper documents and analyzes it using Transformer (BERT).

[0432] Data cleansing and conversion to a standard format

[0433] The structured data transformed by the generative AI is sent to a data cleansing module using Pandas and regular expression libraries to correct any incomplete or incorrect data, and then converted into a standard format.

[0434] Recognizing customer emotions and adjusting user interfaces

[0435] The robot uses a camera mounted on it to recognize the customer's facial expressions, analyzes their emotions using OpenCV, and uses the Google Speech-to-Text API to analyze voice input and provide information based on their emotions.

[0436] Data visualization and presentation

[0437] The data converted into a standard format is visualized using Matplotlib and displayed on the robot's display and on digital signage in physical stores.

[0438] Data storage and sharing

[0439] The final data is stored in a PostgreSQL database and can be shared with other systems and users.

[0440] Prompt Sentence Examples

[0441] Please upload the following PDF file: Customer Inquiry Documents.pdf

[0442] In this way, the system of the present invention can quickly and efficiently digitize paper or PDF format data, enabling the provision of customized services based on customer emotions.

[0443] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0444] Step 1:

[0445] The user presents the robot with documents in paper or PDF format, which it then scans and converts into digital format. Specifically, the robot takes a photo of the paper document with a camera and converts it into a digital format.

[0446] Step 2:

[0447] The robot sends the digital data to the server and stores it temporarily. The server uses Flask to store the received digital data in a temporary directory. This input data is still raw image data or PDF files.

[0448] Step 3:

[0449] The server extracts text from the digital data using Tesseract OCR, and the extracted text data is passed to the generative AI engine. In this step, text is obtained as character information from the image data.

[0450] Step 4:

[0451] The server uses the Transformers (BERT) natural language processing model to convert text data into structured data. For example, it maps text to fields such as product name, price, and specifications. The result of this process is structured JSON data.

[0452] Step 5:

[0453] The server uses Pandas and regular expression libraries to cleanse the structured data, correcting incomplete or incorrect information by completing missing values, removing incorrect strings, and adjusting formatting.

[0454] Step 6:

[0455] The server converts the cleansed data into a standard format, ensuring data compatibility across different systems. This output data is also saved in JSON format.

[0456] Step 7:

[0457] The robot uses OpenCV to analyze the customer's face and recognize emotions from their facial expressions. The input data is video footage from a camera mounted on the robot. The results of emotion recognition are output as emotion labels such as "happiness," "sadness," and "irritation."

[0458] Step 8:

[0459] The robot converts the customer's speech into text using the Google Speech-to-Text API, which then analyzes the text data with the generative AI engine to understand the customer's intent. This step uses speech data as input and outputs text data.

[0460] Step 9:

[0461] The server uses Matplotlib to visualize the data in a standard format, specifically converting it into a format that can be displayed as graphs, charts, maps, etc. This visualized data is then displayed on the robot's display and on digital signage in the physical store.

[0462] Step 10:

[0463] The final data is stored in a PostgreSQL database, accessible to other systems and users, for future data analysis and reference.

[0464] In this way, each processing step works in tandem, making it possible to efficiently digitize paper and PDF documents and provide appropriate services based on customer sentiment.

[0465] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0466] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0467] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0468] [Second embodiment]

[0469] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0470] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0471] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0472] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0473] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0474] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0475] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0476] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0477] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0478] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0479] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0480] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0481] The system of the present invention aims to improve the efficiency of data sharing infrastructure by converting data held by government agencies in paper or electronic file format into a format suitable for digital use and formatting it into a standard format.

[0482] Entering data

[0483] server

[0484] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[0485] Specific examples

[0486] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0487] Data analysis with generative AI

[0488] server

[0489] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[0490] Specific examples

[0491] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[0492] Data Cleansing

[0493] server

[0494] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[0495] Specific examples

[0496] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[0497] Converting data into a standard format

[0498] server

[0499] Consistently format the cleansed data into a predefined standard format.

[0500] Specific examples

[0501] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[0502] Data Visualization

[0503] server

[0504] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[0505] Specific examples

[0506] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[0507] Data storage and sharing

[0508] server

[0509] The final formatted data is securely stored and shared with other systems and users.

[0510] Specific examples

[0511] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[0512] Through this system, non-digital data stored on paper or in PDF format can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[0513] The processing flow will be explained below.

[0514] Step 1:

[0515] The server receives data in the form of paper or PDF files sent from the device or user. The server receives files uploaded via HTTP requests and saves them in a temporary directory. It also checks the format of the uploaded file and prepares it for passing to the appropriate analysis module.

[0516] Step 2:

[0517] The server invokes the generative AI engine to extract text from the received data and convert it into structured data. It also uses the OCR engine to extract text from paper documents and image files, analyzes the text using natural language processing technology, and maps it to specific fields. The analysis results are then saved in a temporary file or database.

[0518] Step 3:

[0519] The server sends the analyzed data output by the generative AI engine to the data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. If there is duplicate data, it is removed or merged, and if there are missing values, it is imputed.

[0520] Step 4:

[0521] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The server then stores the converted data.

[0522] Step 5:

[0523] The server sends the transformed data to the visualization module for easy-to-understand display, passes the structured data to the visualization engine, which displays the data in an appropriate format such as a graph or map, and saves the generated visualization for use in a web portal.

[0524] Step 6:

[0525] The final formatted data is securely stored and shared with other systems and users. The server persists the data stored in the database and shares it with other local governments and related organizations via API as needed. It also allows users to access their own data through a web portal.

[0526] Example 1

[0527] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0528] Modern government agencies have a lot of data in paper or PDF format, and converting this data into a format suitable for digital use requires a large amount of manual work, which is inefficient. It is also difficult to convert data in different formats into a unified standard format, making data integration and centralized management difficult. This hinders efforts to improve the efficiency and transparency of government services.

[0529] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0530] In this invention, the server includes means for receiving and temporarily storing data in paper or electronic file format sent from a user or terminal, means for passing the data received by the server to a generation AI engine and performing text extraction and data structuring using an OCR engine and natural language processing technology, means for sending the structured data converted by the generation AI engine to a data cleansing module and correcting incomplete or erroneous data, means for converting the cleansed data into a predefined standard format, means for visually displaying the data converted into the standard format using a visualization engine, and means for securely storing the final formatted data and sharing it with other systems and users. This makes it possible to efficiently digitize data in different formats and unify them into a standard format.

[0531] A "user" is a person or organization that accesses the system and submits or retrieves data.

[0532] A "terminal" is a device operated by a user, such as a computer, smartphone, or tablet.

[0533] A "server" is a computer system for receiving, processing, storing, and sharing data.

[0534] "Paper media" refers to papers or documents containing information printed on paper.

[0535] An "electronic file format" is a document stored in a digital format, such as a PDF or Word file.

[0536] "Temporary storage" refers to the operation of temporarily storing data before final processing.

[0537] A "generative AI engine" is an artificial intelligence engine that automatically performs text extraction and data structuring.

[0538] An "OCR engine" is a system that uses optical character recognition technology to extract text from image data.

[0539] "Natural language processing technology" is an artificial intelligence technology for understanding and analyzing natural language text.

[0540] "Data structuring" is the process of mapping extracted text to specific fields to make it easier to process and use in a system.

[0541] "Data Cleansing Module" means a software component for correcting and completing incomplete or erroneous data.

[0542] "Standard format" refers to a standard or format for converting different data into a unified format.

[0543] A "visualization engine" is a system for displaying data in a visually easy-to-understand manner.

[0544] "Visual display" refers to the act of visualizing data using text, graphs, maps, etc.

[0545] "Storage" is the act of storing data in a physical or virtual storage device.

[0546] "Sharing" is the act of making data accessible to other systems or users.

[0547] The present invention is a system that converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, thereby streamlining the data sharing infrastructure. This system includes the following components.

[0548] Entering data

[0549] Users can send data in paper or PDF file format from their own devices by logging in to the city hall's web portal and using the file upload function. The server receives the data sent from users and devices and temporarily stores it.

[0550] Specific examples

[0551] A user accesses the city hall web portal and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0552] Data analysis with generative AI

[0553] The server then passes the temporarily stored data to a generative AI engine, which incorporates an OCR engine and natural language processing technology. The OCR engine extracts the text from the PDF and converts it into structured data using natural language processing technology.

[0554] Specific examples

[0555] The generation AI reads the text in the PDF and extracts it using OCR technology, then uses natural language processing technology to analyze information such as name, address, and issue date, and converts it into JSON format data.

[0556] Data Cleansing

[0557] The server sends the analysis data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data from the analysis data.

[0558] Specific examples

[0559] A data cleansing module is used to remove inappropriate strings from name fields and to impute missing values ​​in address data from a postal code database.

[0560] Converting data into a standard format

[0561] The server standardizes the cleansed data into a standard format.

[0562] Specific examples

[0563] The JSON format data will be converted into a nationwide standard resident information format, making it easily usable by other local governments and institutions.

[0564] Data Visualization

[0565] The server sends the data, formatted in a standard format, to a visualization engine for visual display.

[0566] Specific examples

[0567] Resident data is plotted on a map to visually display the coverage area of ​​government services and the distribution of residents. This data can be used as a dashboard by users and other government agencies.

[0568] Data storage and sharing

[0569] The server securely stores the final formatted data and shares it with other systems and users, allowing users to access their own data by setting access rights.

[0570] Specific examples

[0571] Store the data in a standardized format in a city database, share the information with other departments and related agencies, and allow users to access their resident data through a web portal.

[0572] Example prompts for generative AI models

[0573] An example of a prompt is, "Upload a PDF file, extract the text data from it, and structure it into each field (name, address, publication date, etc.). Convert this into JSON format, cleanse the data, and then unify it into a standard format."

[0574] Through this system, non-digital data such as paper documents and PDF files can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[0575] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0576] Step 1:

[0577] A user logs into the city hall web portal using a terminal and performs an operation to upload data in paper or PDF file format. Specifically, the user selects a file and clicks the upload button. This operation sends the data in paper or PDF file format to the server. The input is the paper or PDF file provided by the user, and the output is a file stored on the server.

[0578] Step 2:

[0579] The server saves the data sent by the user in a temporary folder. The server sends a receipt confirmation message to the user to confirm receipt of the data. The input is the file sent by the user, and the output is the file stored in the temporary folder.

[0580] Step 3:

[0581] The server passes the temporarily stored data to the generative AI engine. The generative AI engine uses an OCR engine to extract text from the PDF and uses natural language processing technology to structure the data. Specifically, it analyzes the extracted text and converts information such as name, address, and publication date into JSON format. The input is the file stored in the temporary storage folder, and the output is structured data in JSON format.

[0582] Step 4:

[0583] The server sends the JSON-formatted structured data output by the generation AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data and completes missing values. Specifically, it removes inappropriate strings from name fields and completes missing values ​​in address data from a postal code database. The input is JSON-formatted structured data, and the output is the corrected and completed structured data.

[0584] Step 5:

[0585] The server converts the cleansed structured data into a standard format. Specifically, it converts the cleansed JSON data into a nationwide standard resident information format. This makes it easy for other local governments and organizations to use the data. The input is corrected and supplemented structured data, and the output is data unified in a standard format.

[0586] Step 6:

[0587] The server sends the data converted into a standard format to the visualization engine. The visualization engine converts the data into a format that is easy to display visually. Specifically, it plots the resident data on a map and visually displays the coverage area of ​​administrative services and the distribution of residents. The input is data standardized in a standard format, and the output is visually displayed data.

[0588] Step 7:

[0589] The server securely stores the final formatted data and shares it with other systems and users. Specifically, the data is stored in the city's database and access permissions are set to make it available to other departments and related organizations. Users can also access their own resident data through a web portal. The input is data standardized in a standard format, and the output is securely stored data and shared data.

[0590] (Application example 1)

[0591] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0592] The challenge is to efficiently digitize paper and electronic file data in government agencies and to use that data to provide effective customer support in virtual stores. In particular, there is a need to improve the user experience by quickly and accurately extracting necessary information from uploaded documents and proposing products and services that are suitable for each customer.

[0593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0594] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems or users; means for extracting necessary information from files uploaded by customers and proposing products and services; and means for visually displaying product information in a virtual store. This enables efficient digitization of government agency data and improved customer support in the virtual store.

[0595] An "administrative agency" is a government organization established for the public interest, including local governments and central government agencies.

[0596] "Paper media" means physical documents in which information is printed or written on paper.

[0597] "Electronic file format" means a digital file format that is stored and transmitted electronically, including formats such as PDFs and Word documents.

[0598] "Means for receiving" refers to a method or device for taking in data on paper or in electronic file format from outside and temporarily storing it.

[0599] "Generative AI" refers to systems or programs that use artificial intelligence technology to analyze input data and perform specific tasks.

[0600] "Text extraction methods" are techniques and devices used to detect and extract textual information from non-text data such as images and PDF files.

[0601] "Structured data" is data that is organized according to a specific format or schema, and is in a form that makes it easy to search and manipulate.

[0602] A "data cleansing module" is software or a system for detecting and correcting or removing inaccurate or incomplete data.

[0603] A "standard format" is a data format that is widely accepted and used by multiple systems and institutions, and is a format defined to ensure compatibility.

[0604] "Visual display means" refers to techniques and devices that visually present information in a form that is easy for humans to understand.

[0605] "Storage and sharing means" refers to methods and devices for securely storing data and sharing information with other systems and users as needed.

[0606] "Customer" means a user who visits the virtual store to use or purchase goods or services.

[0607] An "uploading means" is a method or device that allows a user to transfer files or data from their device to a server.

[0608] "Means for extracting information" refers to the technology or devices used to extract necessary information from uploaded files or data.

[0609] The "means for proposing products and services" refers to a technology or device for recommending related products and services to customers based on the extracted information.

[0610] A "virtual store" is an online store or service platform that exists on the Internet and is independent of a physical location.

[0611] The present invention is a system for digitizing paper or electronic file data held by government agencies and converting it into a standard format, and further applying this technology to customer support within a virtual store. Specific embodiments for implementing the present invention are described below.

[0612] 1. Hardware Configuration

[0613] The main hardware for implementing the present invention includes:

[0614] server

[0615] Smartphone (iOS / Android)

[0616] Smart Glasses

[0617] Head-mounted displays (e.g., Oculus Rift)

[0618] 2. Software Configuration

[0619] The main software used includes:

[0620] OCR engine (e.g. Google Cloud Vision API)

[0621] Natural language processing technology (e.g., GPT-3, BERT)

[0622] Data cleansing module (either in-house or via "Trifacta" etc.)

[0623] Visualization engines (e.g. D3.js, Google Charts)

[0624] 3. Data Entry

[0625] Using a smartphone or head-mounted display, users can access the city hall's web portal or virtual store and upload scanned paper documents or PDF files, such as a scanned PDF of a resident registration card.

[0626] 4. Data analysis using generative AI

[0627] The server receives the uploaded file and temporarily stores it. It then passes the file to a generation AI engine, which uses OCR and natural language processing to extract text and structure the data. For example, it extracts text from a PDF and converts information such as name, address, and publication date into JSON format.

[0628] 5. Data Cleansing

[0629] The analyzed data output by the generative AI engine is processed by the data cleansing module, which corrects incomplete or incorrect data to ensure data accuracy. For example, if an inappropriate string is included in a name field, it is removed, and missing values ​​in address data are filled in using postal codes.

[0630] 6. Converting data into a standard format

[0631] The cleansed data is standardized into a predefined standard format. For example, JSON data can be converted into a nationwide standard format for resident information.

[0632] 7. Data Visualization

[0633] Once the data has been converted into a standard format, it can be displayed using a visualization engine. For example, resident data can be plotted on a map to visually display the coverage area of ​​government services and the distribution of residents.

[0634] 8. Data Storage and Sharing

[0635] The final formatted data can then be securely stored and shared with other systems and users. For example, data converted into a standard format can be stored in a city database and shared with other departments and related agencies.

[0636] 9. Customer Support Applications

[0637] The system receives PDF files of product catalogs and invoices uploaded by customers and analyzes them using a generative AI engine. Based on the analyzed data, it proposes products and services in a virtual store. This information is displayed visually on a smartphone or head-mounted display. For example, it prepares JSON data including product names, model numbers, and price information to make optimal proposals to customers.

[0638] Examples and prompts

[0639] As a concrete example, the following prompt sentence is input into the generative AI model:

[0640] "Please extract the product characteristics and prices listed in this PDF document and convert them into JSON format."

[0641] "Extract the product attributes and prices from this PDF document and convert them into JSON format."

[0642] Template prompt:

[0643] "Based on the contents of the PDF document, extract product characteristics, prices, reviews, etc. and output them in JSON format, which is the standard format for virtual stores."

[0644] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[0645] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0646] Step 1:

[0647] Users access the city hall's web portal or virtual store using a smartphone or head-mounted display and upload PDF or scanned document files, which are then transferred to the server and temporarily stored.

[0648] Input: PDF files or scanned documents uploaded by the user

[0649] Output: A file temporarily saved on the server

[0650] Step 2:

[0651] The server passes the received file to the generative AI engine, which uses OCR technology to extract the text from the file. The generative AI engine uses the Google Cloud Vision API, for example, to detect the text.

[0652] Input: PDF file stored on the server

[0653] Output: Raw text data extracted by OCR

[0654] Step 3:

[0655] The server then analyzes the extracted text data using natural language processing techniques (e.g., GPT-3, BERT) and structures the data, for example, generating data mapped to specific fields such as name, address, and publication date.

[0656] Input: Raw text data extracted by OCR

[0657] Output: Structured data (e.g., JSON format) mapped to fields

[0658] Step 4:

[0659] The server sends the generated structured data to a data cleansing module (e.g., Trifacta) to correct erroneous or incomplete data, such as removing inappropriate strings and imputing missing values.

[0660] Input: Structured text data

[0661] Output: Cleansed and accurate text data

[0662] Step 5:

[0663] The server converts the cleansed data into a standard format, for example, converting resident information into a nationally standard format that can be easily used by other systems and organizations.

[0664] Input: Cleansed text data

[0665] Output: Data converted to a standard format (e.g., JSON data in a common format)

[0666] Step 6:

[0667] The server passes the converted data to a visualization engine (e.g., D3.js, Google Charts) for visualization. For example, it plots the resident data on a map to help users understand it intuitively.

[0668] Input: Data in standard format

[0669] Output: Visualized display data (e.g. dashboards and map plots)

[0670] Step 7:

[0671] The final formatted data is stored securely on a server and shared with other systems and users, for example, by storing it in a resident information database and providing access rights to share the data with related organizations.

[0672] Input: Data converted to a standard format

[0673] Output: Data stored and shared

[0674] Step 8:

[0675] Customers visit the virtual store and upload a PDF file, which the server then passes to a generative AI engine for analysis, extracting and structuring information for, for example, a product catalog or invoice.

[0676] Input: PDF file uploaded by customer

[0677] Output: Parsed and structured product information data

[0678] Step 9:

[0679] The server then proposes appropriate products and services to customers based on the structured product information, which is visually displayed on a smartphone or head-mounted display.

[0680] Input: Structured product information data

[0681] Output: Visual display of suggested products and services

[0682] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[0683] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0684] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes user emotions and uses them to process the data.

[0685] Entering data

[0686] server

[0687] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[0688] Specific examples

[0689] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0690] Data analysis with generative AI

[0691] server

[0692] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[0693] Specific examples

[0694] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[0695] Data Cleansing

[0696] server

[0697] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[0698] Specific examples

[0699] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[0700] Converting data into a standard format

[0701] server

[0702] Consistently format the cleansed data into a predefined standard format.

[0703] Specific examples

[0704] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[0705] Data Visualization

[0706] server

[0707] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[0708] Specific examples

[0709] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[0710] Data storage and sharing

[0711] server

[0712] The final formatted data is securely stored and shared with other systems and users.

[0713] Specific examples

[0714] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[0715] User Emotion Recognition

[0716] server

[0717] The emotion engine analyzes emotions based on user input and operations, and uses the results for data processing and display.

[0718] Specific examples

[0719] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[0720] Emotion-Based User Interface Adjustment

[0721] server

[0722] The emotion engine dynamically changes the display method and interface based on the user's emotions, improving the user experience.

[0723] Specific examples

[0724] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, while if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[0725] This system efficiently converts paper and PDF data into digital format, and realizes a series of processes from data standardization to visualization, storage, and sharing. Furthermore, it can recognize user emotions and process and display data accordingly, improving the user experience.

[0726] The processing flow will be explained below.

[0727] Step 1:

[0728] The server receives data sent from the device or user in the form of paper or PDF files. The received data is saved in a temporary directory, the file format is checked, and it is prepared for passing to the appropriate analysis module.

[0729] Step 2:

[0730] The server uses an OCR engine to extract text from paper documents and image files. OCR technology analyzes the text in the image and extracts it as digital text data.

[0731] Step 3:

[0732] The server then calls the generative AI engine, which uses natural language processing to analyze the extracted text. The generative AI analyzes the text, maps it to specific fields such as name, address, and publication date, and converts it into structured data in JSON format.

[0733] Step 4:

[0734] The server sends the JSON data output by the generative AI to a data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. Duplicate data is removed or merged, and missing values ​​are imputed.

[0735] Step 5:

[0736] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The converted data is then stored.

[0737] Step 6:

[0738] The server passes the converted data to a visualization module for visual display. The visualization module displays the structured data in the form of graphs, maps, etc. The generated visualizations are saved and made available to a web portal.

[0739] Step 7:

[0740] The server securely stores the final formatted data and shares it with other systems and users. The data stored in the database is provided to local governments and related organizations via API, and users can access their own data through a web portal.

[0741] Step 8:

[0742] The server uses an emotion engine to analyze emotions from user input and operations. The emotion engine analyzes user input data in real time and identifies emotions such as stress and dissatisfaction.

[0743] Step 9:

[0744] The server dynamically adjusts the data display method and user interface based on the user's emotions analyzed by the emotion engine. If the user's emotions are positive, detailed data and graphs are displayed, but if the user expresses irritation, the interface switches to a simple and intuitive one.

[0745] Step 10:

[0746] The server provides the user with an adjusted user interface to improve the user experience. The emotion engine monitors the user's emotions in real time and changes the behavior of the interface accordingly, if necessary.

[0747] In this way, the present invention enables efficient digitization and data processing, and also provides an interface that adapts to the user's emotions, thereby achieving improved efficiency and transparency in government services.

[0748] Example 2

[0749] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0750] Government agencies are required to efficiently convert paper and electronic file data into digital format, and then standardize that data into a standard format so that it can be shared with other systems and users. Another challenge with the digitalization of data is recognizing user emotions and reflecting them in system operations to improve the user experience.

[0751] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0752] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for analyzing emotions based on user input and operations and using the results for data processing and display; and means for dynamically changing the user interface based on the emotions. This enables efficient digital conversion, standardization, visual display, and sharing of data, and further improves the operability by dynamically adjusting the user interface to reflect the user's emotions.

[0753] An "administrative agency" is an organization that carries out public duties, such as the national government or local governments.

[0754] "Paper media" refers to the physical format of documents, papers, etc. printed on paper.

[0755] "Electronic file format" refers to digital data formats such as PDF, Word, and Excel.

[0756] "Means for receiving data" refers to the function by which the server acquires data sent from a user or terminal.

[0757] "Generative AI" is an engine that uses machine learning and artificial intelligence techniques to analyze data.

[0758] "Means for extracting text" refers to a function that uses generative AI to extract text information from data.

[0759] "Structured data" is data that is organized according to a particular format or structure.

[0760] A "data cleansing module" is a program for correcting incomplete or erroneous data.

[0761] A "standard format" is a format for arranging data in a unified format.

[0762] "Visual display means" refers to techniques for visualizing data in an easy-to-understand manner.

[0763] "Means for storing final data" refers to a function for safely storing data for a long period of time.

[0764] "Means for sharing with other systems and users" refers to a function that provides the stored data so that it can be used by other systems and users.

[0765] "Means for analyzing emotions" refers to technology that reads emotions from user input and operations.

[0766] "Means for dynamically changing the user interface based on emotions" is a function that changes the screen display and operation method according to the analyzed emotions.

[0767] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes the user's emotions and uses them in data processing. Specific embodiments of the system are described below.

[0768] Entering data

[0769] The server receives data sent from users or devices in the form of paper or PDF files. The received data is temporarily stored. This process uses, for example, an Amazon S3 bucket.

[0770] Examples:

[0771] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0772] Data analysis with generative AI

[0773] The server then passes the received data to a generative AI engine, which extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology, such as Google Cloud Vision API or Microsoft Azure's OCR service.

[0774] Examples:

[0775] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[0776] Data Cleansing

[0777] The server sends the analysis data output by the generative AI engine to a data cleansing module. The data cleansing module includes a program for correcting incomplete or incorrect data. For example, Apache Spark can be used.

[0778] Examples:

[0779] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[0780] Converting data into a standard format

[0781] The server standardizes the cleansed data into a predefined standard format, ensuring data consistency and compatibility.

[0782] Examples:

[0783] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[0784] Data Visualization

[0785] The server uses a visualization engine, such as Tableau or D3.js, to display the data converted into a standard format in an easy-to-understand format.

[0786] Examples:

[0787] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[0788] Data storage and sharing

[0789] The server securely stores the final formatted data and shares it with other systems and users. For example, the data could be stored in a city database and other departments and agencies could access it through an API.

[0790] Examples:

[0791] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[0792] User Emotion Recognition

[0793] The emotion engine analyzes emotions based on user input and actions, and uses the results for data processing and display, for example, using Amazon Comprehend or IBM Watson.

[0794] Examples:

[0795] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[0796] Emotion-Based User Interface Adjustment

[0797] Based on the user's emotions recognized by the emotion engine, the server dynamically changes the display method and interface, thereby improving the user experience.

[0798] Examples:

[0799] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, whereas if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[0800] Prompt Sentence Examples

[0801] "Extract text from PDF using OCR technology."

[0802] "Analyze the extracted text using natural language processing technology and convert it into JSON format."

[0803] "Analyze emotions based on user actions and adjust your interface accordingly."

[0804] This system efficiently converts paper and PDF data into digital format, enabling a series of processes from data standardization to visualization, storage, and sharing. It also recognizes user emotions and processes and displays data accordingly, improving the user experience.

[0805] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0806] Step 1: User Data Entry

[0807] The user accesses the city hall web portal using their own device and uploads the scanned PDF file of the resident registration card. Specifically, they click the upload button on the web portal and select the corresponding PDF file in the file selection dialog. The input here is the PDF file of the resident registration card, and the output is the completion of uploading it to the server.

[0808] Step 2: Data reception and temporary storage by the server

[0809] The server receives the uploaded PDF file and temporarily stores it in a database or file storage (e.g., Amazon S3 bucket). Specifically, it stores the uploaded file in the specified storage and records the file path. The input here is the PDF file uploaded by the user, and the output is the file path saved in the storage.

[0810] Step 3: Data analysis by server-generated AI

[0811] The server passes the saved PDF file to a generation AI engine (e.g., Google Cloud Vision API or Microsoft Azure's OCR service) to extract text and structure the data. Specifically, it sends the file path to the OCR engine and receives the extracted text data. It then uses natural language processing technology to analyze the text and convert it to JSON format. The input is the temporarily saved PDF file, and the output is the analyzed data in JSON format.

[0812] Step 4: Server-based data cleansing

[0813] The server sends the JSON data obtained from the generative AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data. Specifically, if a field contains an inappropriate string, it deletes it and fills in missing data with appropriate values ​​from an external database. The input is the JSON data obtained from the generative AI engine, and the output is the cleansed data.

[0814] Step 5: Server converts data into standard format

[0815] The server standardizes the cleansed data into a predefined standard format. Specifically, it converts the JSON data into a nationwide standard format for resident information. The input is the cleansed data, and the output is data standardized in the standard format.

[0816] Step 6: Visualizing the data on the server

[0817] The server uses a visualization engine (e.g., Tableau or D3.js) to display the data converted into a standard format in a way that is easy to understand visually. Specifically, it plots resident data on a map and displays the coverage area of ​​administrative services and resident distribution as a dashboard. The input is data in a standard format, and the output is a visualized dashboard.

[0818] Step 7: Storing and Sharing Data on a Server

[0819] The server securely stores the final formatted data and shares it with other systems and users. Specifically, it stores the data in the city's database and provides access to other departments and related organizations through APIs. The input is the visualized data, and the output is the stored data and access rights settings for sharing.

[0820] Step 8: User emotion recognition by the server

[0821] The server's emotion engine analyzes emotions based on user input and operations. Specifically, it measures response times when users operate the system and analyzes stress and dissatisfaction. The input is the user's operation log, and the output is the emotion analysis results.

[0822] Step 9: Emotion-based user interface adjustment by the server

[0823] The server dynamically changes the display method and interface based on the user's emotions recognized by the emotion engine. Specifically, if a positive emotion is recognized, the server provides an interface displaying detailed data and graphs, and if a negative emotion is recognized, the server switches to a simple and intuitive interface. The input is the emotion analysis results, and the output is the adjusted user interface.

[0824] (Application example 2)

[0825] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0826] While conventional data processing systems can efficiently digitize paper and electronic file data held by government agencies and convert them into standard formats, they have the problem of being unable to recognize the user's emotions during operation and dynamically adjust data processing and display accordingly. Furthermore, when it comes to customer service, it has been difficult to instantly analyze paper or PDF data and provide the results in a visually easy-to-understand format.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving data in paper or electronic file format held by an administrative agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module and correcting incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for a robot to respond to customer inquiries; means for recognizing customer emotions and presenting information based on those emotions; and means for displaying the data converted into the standard format on a display in a physical store or on digital signage. This makes it possible to recognize customer emotions and respond appropriately when dealing with customers, and to instantly analyze and visually provide data.

[0828] "Administrative agencies" are public institutions such as the national government and local governments, and are organizations responsible for citizen services and public affairs.

[0829] "Paper media" refers to documents and materials that are printed or handwritten on paper.

[0830] An "electronic file format" is a digital data file format that can be read by a computer, and generally includes PDFs and text files.

[0831] "Generative AI" refers to software or systems that use artificial intelligence technology to process and analyze data.

[0832] "Text extraction" refers to the process of extracting textual information from digital data or images.

[0833] "Structured data" is data that is organized according to specific formats and conventions, and is the format used in databases.

[0834] "Data cleansing" is the process of correcting or removing incomplete or erroneous information from data.

[0835] A "standard format" is a format in which data is standardized based on certain rules, and is used to ensure compatibility between different systems.

[0836] A "visual display" is a representation of data in a visual format, such as a graph, chart, or map, that facilitates understanding of the information.

[0837] A "customer inquiry" is an in-store or online customer requesting information about a product or service.

[0838] A "robot" is a machine or device that uses artificial intelligence and machine learning to operate autonomously and perform specific tasks.

[0839] "Emotion recognition" is a technology that analyzes a user's emotional state from their facial expressions, voice, behavioral patterns, etc.

[0840] A "display" is an electronic device for displaying visual information, such as those used in computers and digital signage.

[0841] "Digital signage" refers to an advertising or information system that uses a display to display digital content.

[0842] This invention is a system for improving the efficiency of customer service in brick-and-mortar stores and providing services based on customer emotions. This system is implemented using the following hardware and software.

[0843] System Configuration

[0844] Hardware

[0845] 1. Robot:

[0846] It has audio input and output functions.

[0847] It is equipped with a high-resolution camera that recognizes customers' facial expressions.

[0848] It has a communication module for sending data from the terminal to the server.

[0849] 2. Server:

[0850] Store and analyze the data.

[0851] It is equipped with a high-performance GPU and is used to run generative AI models.

[0852] 3. Displays and Digital Signage:

[0853] Used to display information to customers in physical stores.

[0854] software

[0855] 1. Python:

[0856] Used for data analysis and running generative AI.

[0857] 2. Flask:

[0858] Used as a server-side web application framework.

[0859] 3. Tesseract OCR:

[0860] Used to extract text from paper and electronic file formats.

[0861] 4. Transformers (BERT):

[0862] Text analysis is performed using natural language processing technology.

[0863] 5. OpenCV:

[0864] Used to analyze customer facial expressions from camera footage.

[0865] 6. Google Speech-to-Text API:

[0866] Used to convert voice input into text.

[0867] 7. Pandas and Regular Expressions:

[0868] Used to cleanse and shape data.

[0869] 8. Matplotlib:

[0870] Used for data visualization.

[0871] 9. PostgreSQL:

[0872] Used to store data.

[0873] Example of operation

[0874] Entering and saving data

[0875] The user (customer) presents the product catalogue as a paper document or PDF file to the robot, which then sends it to the server and temporarily stores it.

[0876] Data analysis and normalization

[0877] The server uses generative AI to extract text from the received data and convert it into structured data. Specifically, it uses Tesseract OCR to extract text from paper documents and analyzes it using Transformer (BERT).

[0878] Data cleansing and conversion to a standard format

[0879] The structured data transformed by the generative AI is sent to a data cleansing module using Pandas and regular expression libraries to correct any incomplete or incorrect data, and then converted into a standard format.

[0880] Recognizing customer emotions and adjusting user interfaces

[0881] The robot uses a camera mounted on it to recognize the customer's facial expressions, analyzes their emotions using OpenCV, and uses the Google Speech-to-Text API to analyze voice input and provide information based on their emotions.

[0882] Data visualization and presentation

[0883] The data converted into a standard format is visualized using Matplotlib and displayed on the robot's display and on digital signage in physical stores.

[0884] Data storage and sharing

[0885] The final data is stored in a PostgreSQL database and can be shared with other systems and users.

[0886] Prompt Sentence Examples

[0887] Please upload the following PDF file: Customer Inquiry Documents.pdf

[0888] In this way, the system of the present invention can quickly and efficiently digitize paper or PDF format data, enabling the provision of customized services based on customer emotions.

[0889] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0890] Step 1:

[0891] The user presents the robot with documents in paper or PDF format, which it then scans and converts into digital format. Specifically, the robot takes a photo of the paper document with a camera and converts it into a digital format.

[0892] Step 2:

[0893] The robot sends the digital data to the server and stores it temporarily. The server uses Flask to store the received digital data in a temporary directory. This input data is still raw image data or PDF files.

[0894] Step 3:

[0895] The server extracts text from the digital data using Tesseract OCR, and the extracted text data is passed to the generative AI engine. In this step, text is obtained as character information from the image data.

[0896] Step 4:

[0897] The server uses the Transformers (BERT) natural language processing model to convert text data into structured data. For example, it maps text to fields such as product name, price, and specifications. The result of this process is structured JSON data.

[0898] Step 5:

[0899] The server uses Pandas and regular expression libraries to cleanse the structured data, correcting incomplete or incorrect information by completing missing values, removing incorrect strings, and adjusting formatting.

[0900] Step 6:

[0901] The server converts the cleansed data into a standard format, ensuring data compatibility across different systems. This output data is also saved in JSON format.

[0902] Step 7:

[0903] The robot uses OpenCV to analyze the customer's face and recognize emotions from their facial expressions. The input data is video footage from a camera mounted on the robot. The results of emotion recognition are output as emotion labels such as "happiness," "sadness," and "irritation."

[0904] Step 8:

[0905] The robot converts the customer's speech into text using the Google Speech-to-Text API, which then analyzes the text data with the generative AI engine to understand the customer's intent. This step uses speech data as input and outputs text data.

[0906] Step 9:

[0907] The server uses Matplotlib to visualize the data in a standard format, specifically converting it into a format that can be displayed as graphs, charts, maps, etc. This visualized data is then displayed on the robot's display and on digital signage in the physical store.

[0908] Step 10:

[0909] The final data is stored in a PostgreSQL database, accessible to other systems and users, for future data analysis and reference.

[0910] In this way, each processing step works in tandem, making it possible to efficiently digitize paper and PDF documents and provide appropriate services based on customer sentiment.

[0911] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0912] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0913] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0914] [Third embodiment]

[0915] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0916] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0917] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0918] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0919] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0920] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0921] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0922] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0923] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0924] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0925] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0926] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0927] The system of the present invention aims to improve the efficiency of data sharing infrastructure by converting data held by government agencies in paper or electronic file format into a format suitable for digital use and formatting it into a standard format.

[0928] Entering data

[0929] server

[0930] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[0931] Specific examples

[0932] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0933] Data analysis with generative AI

[0934] server

[0935] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[0936] Specific examples

[0937] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[0938] Data Cleansing

[0939] server

[0940] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[0941] Specific examples

[0942] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[0943] Converting data into a standard format

[0944] server

[0945] Consistently format the cleansed data into a predefined standard format.

[0946] Specific examples

[0947] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[0948] Data Visualization

[0949] server

[0950] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[0951] Specific examples

[0952] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[0953] Data storage and sharing

[0954] server

[0955] The final formatted data is securely stored and shared with other systems and users.

[0956] Specific examples

[0957] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[0958] Through this system, non-digital data stored on paper or in PDF format can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[0959] The processing flow will be explained below.

[0960] Step 1:

[0961] The server receives data in the form of paper or PDF files sent from the device or user. The server receives files uploaded via HTTP requests and saves them in a temporary directory. It also checks the format of the uploaded file and prepares it for passing to the appropriate analysis module.

[0962] Step 2:

[0963] The server invokes the generative AI engine to extract text from the received data and convert it into structured data. It also uses the OCR engine to extract text from paper documents and image files, analyzes the text using natural language processing technology, and maps it to specific fields. The analysis results are then saved in a temporary file or database.

[0964] Step 3:

[0965] The server sends the analyzed data output by the generative AI engine to the data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. If there is duplicate data, it is removed or merged, and if there are missing values, it is imputed.

[0966] Step 4:

[0967] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The server then stores the converted data.

[0968] Step 5:

[0969] The server sends the transformed data to the visualization module for easy-to-understand display, passes the structured data to the visualization engine, which displays the data in an appropriate format such as a graph or map, and saves the generated visualization for use in a web portal.

[0970] Step 6:

[0971] The final formatted data is securely stored and shared with other systems and users. The server persists the data stored in the database and shares it with other local governments and related organizations via API as needed. It also allows users to access their own data through a web portal.

[0972] Example 1

[0973] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0974] Modern government agencies have a lot of data in paper or PDF format, and converting this data into a format suitable for digital use requires a large amount of manual work, which is inefficient. It is also difficult to convert data in different formats into a unified standard format, making data integration and centralized management difficult. This hinders efforts to improve the efficiency and transparency of government services.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0976] In this invention, the server includes means for receiving and temporarily storing data in paper or electronic file format sent from a user or terminal, means for passing the data received by the server to a generation AI engine and performing text extraction and data structuring using an OCR engine and natural language processing technology, means for sending the structured data converted by the generation AI engine to a data cleansing module and correcting incomplete or erroneous data, means for converting the cleansed data into a predefined standard format, means for visually displaying the data converted into the standard format using a visualization engine, and means for securely storing the final formatted data and sharing it with other systems and users. This makes it possible to efficiently digitize data in different formats and unify them into a standard format.

[0977] A "user" is a person or organization that accesses the system and submits or retrieves data.

[0978] A "terminal" is a device operated by a user, such as a computer, smartphone, or tablet.

[0979] A "server" is a computer system for receiving, processing, storing, and sharing data.

[0980] "Paper media" refers to papers or documents containing information printed on paper.

[0981] An "electronic file format" is a document stored in a digital format, such as a PDF or Word file.

[0982] "Temporary storage" refers to the operation of temporarily storing data before final processing.

[0983] A "generative AI engine" is an artificial intelligence engine that automatically performs text extraction and data structuring.

[0984] An "OCR engine" is a system that uses optical character recognition technology to extract text from image data.

[0985] "Natural language processing technology" is an artificial intelligence technology for understanding and analyzing natural language text.

[0986] "Data structuring" is the process of mapping extracted text to specific fields to make it easier to process and use in a system.

[0987] "Data Cleansing Module" means a software component for correcting and completing incomplete or erroneous data.

[0988] "Standard format" refers to a standard or format for converting different data into a unified format.

[0989] A "visualization engine" is a system for displaying data in a visually easy-to-understand manner.

[0990] "Visual display" refers to the act of visualizing data using text, graphs, maps, etc.

[0991] "Storage" is the act of storing data in a physical or virtual storage device.

[0992] "Sharing" is the act of making data accessible to other systems or users.

[0993] The present invention is a system that converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, thereby streamlining the data sharing infrastructure. This system includes the following components.

[0994] Entering data

[0995] Users can send data in paper or PDF file format from their own devices by logging in to the city hall's web portal and using the file upload function. The server receives the data sent from users and devices and temporarily stores it.

[0996] Specific examples

[0997] A user accesses the city hall web portal and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[0998] Data analysis with generative AI

[0999] The server then passes the temporarily stored data to a generative AI engine, which incorporates an OCR engine and natural language processing technology. The OCR engine extracts the text from the PDF and converts it into structured data using natural language processing technology.

[1000] Specific examples

[1001] The generation AI reads the text in the PDF and extracts it using OCR technology, then uses natural language processing technology to analyze information such as name, address, and issue date, and converts it into JSON format data.

[1002] Data Cleansing

[1003] The server sends the analysis data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data from the analysis data.

[1004] Specific examples

[1005] A data cleansing module is used to remove inappropriate strings from name fields and to impute missing values ​​in address data from a postal code database.

[1006] Converting data into a standard format

[1007] The server standardizes the cleansed data into a standard format.

[1008] Specific examples

[1009] The JSON format data will be converted into a nationwide standard resident information format, making it easily usable by other local governments and institutions.

[1010] Data Visualization

[1011] The server sends the data, formatted in a standard format, to a visualization engine for visual display.

[1012] Specific examples

[1013] Resident data is plotted on a map to visually display the coverage area of ​​government services and the distribution of residents. This data can be used as a dashboard by users and other government agencies.

[1014] Data storage and sharing

[1015] The server securely stores the final formatted data and shares it with other systems and users, allowing users to access their own data by setting access rights.

[1016] Specific examples

[1017] Store the data in a standardized format in a city database, share the information with other departments and related agencies, and allow users to access their resident data through a web portal.

[1018] Example prompts for generative AI models

[1019] An example of a prompt is, "Upload a PDF file, extract the text data from it, and structure it into each field (name, address, publication date, etc.). Convert this into JSON format, cleanse the data, and then unify it into a standard format."

[1020] Through this system, non-digital data such as paper documents and PDF files can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[1021] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1022] Step 1:

[1023] A user logs into the city hall web portal using a terminal and performs an operation to upload data in paper or PDF file format. Specifically, the user selects a file and clicks the upload button. This operation sends the data in paper or PDF file format to the server. The input is the paper or PDF file provided by the user, and the output is a file stored on the server.

[1024] Step 2:

[1025] The server saves the data sent by the user in a temporary folder. The server sends a receipt confirmation message to the user to confirm receipt of the data. The input is the file sent by the user, and the output is the file stored in the temporary folder.

[1026] Step 3:

[1027] The server passes the temporarily stored data to the generative AI engine. The generative AI engine uses an OCR engine to extract text from the PDF and uses natural language processing technology to structure the data. Specifically, it analyzes the extracted text and converts information such as name, address, and publication date into JSON format. The input is the file stored in the temporary storage folder, and the output is structured data in JSON format.

[1028] Step 4:

[1029] The server sends the JSON-formatted structured data output by the generation AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data and completes missing values. Specifically, it removes inappropriate strings from name fields and completes missing values ​​in address data from a postal code database. The input is JSON-formatted structured data, and the output is the corrected and completed structured data.

[1030] Step 5:

[1031] The server converts the cleansed structured data into a standard format. Specifically, it converts the cleansed JSON data into a nationwide standard resident information format. This makes it easy for other local governments and organizations to use the data. The input is corrected and supplemented structured data, and the output is data unified in a standard format.

[1032] Step 6:

[1033] The server sends the data converted into a standard format to the visualization engine. The visualization engine converts the data into a format that is easy to display visually. Specifically, it plots the resident data on a map and visually displays the coverage area of ​​administrative services and the distribution of residents. The input is data standardized in a standard format, and the output is visually displayed data.

[1034] Step 7:

[1035] The server securely stores the final formatted data and shares it with other systems and users. Specifically, the data is stored in the city's database and access permissions are set to make it available to other departments and related organizations. Users can also access their own resident data through a web portal. The input is data standardized in a standard format, and the output is securely stored data and shared data.

[1036] (Application example 1)

[1037] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1038] The challenge is to efficiently digitize paper and electronic file data in government agencies and to use that data to provide effective customer support in virtual stores. In particular, there is a need to improve the user experience by quickly and accurately extracting necessary information from uploaded documents and proposing products and services that are suitable for each customer.

[1039] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1040] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems or users; means for extracting necessary information from files uploaded by customers and proposing products and services; and means for visually displaying product information in a virtual store. This enables efficient digitization of government agency data and improved customer support in the virtual store.

[1041] An "administrative agency" is a government organization established for the public interest, including local governments and central government agencies.

[1042] "Paper media" means physical documents in which information is printed or written on paper.

[1043] "Electronic file format" means a digital file format that is stored and transmitted electronically, including formats such as PDFs and Word documents.

[1044] "Means for receiving" refers to a method or device for taking in data on paper or in electronic file format from outside and temporarily storing it.

[1045] "Generative AI" refers to systems or programs that use artificial intelligence technology to analyze input data and perform specific tasks.

[1046] "Text extraction methods" are techniques and devices used to detect and extract textual information from non-text data such as images and PDF files.

[1047] "Structured data" is data that is organized according to a specific format or schema, and is in a form that makes it easy to search and manipulate.

[1048] A "data cleansing module" is software or a system for detecting and correcting or removing inaccurate or incomplete data.

[1049] A "standard format" is a data format that is widely accepted and used by multiple systems and institutions, and is a format defined to ensure compatibility.

[1050] "Visual display means" refers to techniques and devices that visually present information in a form that is easy for humans to understand.

[1051] "Storage and sharing means" refers to methods and devices for securely storing data and sharing information with other systems and users as needed.

[1052] "Customer" means a user who visits the virtual store to use or purchase goods or services.

[1053] An "uploading means" is a method or device that allows a user to transfer files or data from their device to a server.

[1054] "Means for extracting information" refers to the technology or devices used to extract necessary information from uploaded files or data.

[1055] The "means for proposing products and services" refers to a technology or device for recommending related products and services to customers based on the extracted information.

[1056] A "virtual store" is an online store or service platform that exists on the Internet and is independent of a physical location.

[1057] The present invention is a system for digitizing paper or electronic file data held by government agencies and converting it into a standard format, and further applying this technology to customer support within a virtual store. Specific embodiments for implementing the present invention are described below.

[1058] 1. Hardware Configuration

[1059] The main hardware for implementing the present invention includes:

[1060] server

[1061] Smartphone (iOS / Android)

[1062] Smart Glasses

[1063] Head-mounted displays (e.g., Oculus Rift)

[1064] 2. Software Configuration

[1065] The main software used includes:

[1066] OCR engine (e.g. Google Cloud Vision API)

[1067] Natural language processing technology (e.g., GPT-3, BERT)

[1068] Data cleansing module (either in-house or via "Trifacta" etc.)

[1069] Visualization engines (e.g. D3.js, Google Charts)

[1070] 3. Data Entry

[1071] Using a smartphone or head-mounted display, users can access the city hall's web portal or virtual store and upload scanned paper documents or PDF files, such as a scanned PDF of a resident registration card.

[1072] 4. Data analysis using generative AI

[1073] The server receives the uploaded file and temporarily stores it. It then passes the file to a generation AI engine, which uses OCR and natural language processing to extract text and structure the data. For example, it extracts text from a PDF and converts information such as name, address, and publication date into JSON format.

[1074] 5. Data Cleansing

[1075] The analyzed data output by the generative AI engine is processed by the data cleansing module, which corrects incomplete or incorrect data to ensure data accuracy. For example, if an inappropriate string is included in a name field, it is removed, and missing values ​​in address data are filled in using postal codes.

[1076] 6. Converting data into a standard format

[1077] The cleansed data is standardized into a predefined standard format. For example, JSON data can be converted into a nationwide standard format for resident information.

[1078] 7. Data Visualization

[1079] Once the data has been converted into a standard format, it can be displayed using a visualization engine. For example, resident data can be plotted on a map to visually display the coverage area of ​​government services and the distribution of residents.

[1080] 8. Data Storage and Sharing

[1081] The final formatted data can then be securely stored and shared with other systems and users. For example, data converted into a standard format can be stored in a city database and shared with other departments and related agencies.

[1082] 9. Customer Support Applications

[1083] The system receives PDF files of product catalogs and invoices uploaded by customers and analyzes them using a generative AI engine. Based on the analyzed data, it proposes products and services in a virtual store. This information is displayed visually on a smartphone or head-mounted display. For example, it prepares JSON data including product names, model numbers, and price information to make optimal proposals to customers.

[1084] Examples and prompts

[1085] As a concrete example, the following prompt sentence is input into the generative AI model:

[1086] "Please extract the product characteristics and prices listed in this PDF document and convert them into JSON format."

[1087] "Extract the product attributes and prices from this PDF document and convert them into JSON format."

[1088] Template prompt:

[1089] "Based on the contents of the PDF document, extract product characteristics, prices, reviews, etc. and output them in JSON format, which is the standard format for virtual stores."

[1090] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[1091] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1092] Step 1:

[1093] Users access the city hall's web portal or virtual store using a smartphone or head-mounted display and upload PDF or scanned document files, which are then transferred to the server and temporarily stored.

[1094] Input: PDF files or scanned documents uploaded by the user

[1095] Output: A file temporarily saved on the server

[1096] Step 2:

[1097] The server passes the received file to the generative AI engine, which uses OCR technology to extract the text from the file. The generative AI engine uses the Google Cloud Vision API, for example, to detect the text.

[1098] Input: PDF file stored on the server

[1099] Output: Raw text data extracted by OCR

[1100] Step 3:

[1101] The server then analyzes the extracted text data using natural language processing techniques (e.g., GPT-3, BERT) and structures the data, for example, generating data mapped to specific fields such as name, address, and publication date.

[1102] Input: Raw text data extracted by OCR

[1103] Output: Structured data (e.g., JSON format) mapped to fields

[1104] Step 4:

[1105] The server sends the generated structured data to a data cleansing module (e.g., Trifacta) to correct erroneous or incomplete data, such as removing inappropriate strings and imputing missing values.

[1106] Input: Structured text data

[1107] Output: Cleansed and accurate text data

[1108] Step 5:

[1109] The server converts the cleansed data into a standard format, for example, converting resident information into a nationally standard format that can be easily used by other systems and organizations.

[1110] Input: Cleansed text data

[1111] Output: Data converted to a standard format (e.g., JSON data in a common format)

[1112] Step 6:

[1113] The server passes the converted data to a visualization engine (e.g., D3.js, Google Charts) for visualization. For example, it plots the resident data on a map to help users understand it intuitively.

[1114] Input: Data in standard format

[1115] Output: Visualized display data (e.g. dashboards and map plots)

[1116] Step 7:

[1117] The final formatted data is stored securely on a server and shared with other systems and users, for example, by storing it in a resident information database and providing access rights to share the data with related organizations.

[1118] Input: Data converted to a standard format

[1119] Output: Data stored and shared

[1120] Step 8:

[1121] Customers visit the virtual store and upload a PDF file, which the server then passes to a generative AI engine for analysis, extracting and structuring information for, for example, a product catalog or invoice.

[1122] Input: PDF file uploaded by customer

[1123] Output: Parsed and structured product information data

[1124] Step 9:

[1125] The server then proposes appropriate products and services to customers based on the structured product information, which is visually displayed on a smartphone or head-mounted display.

[1126] Input: Structured product information data

[1127] Output: Visual display of suggested products and services

[1128] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[1129] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1130] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes user emotions and uses them to process the data.

[1131] Entering data

[1132] server

[1133] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[1134] Specific examples

[1135] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[1136] Data analysis with generative AI

[1137] server

[1138] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[1139] Specific examples

[1140] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[1141] Data Cleansing

[1142] server

[1143] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[1144] Specific examples

[1145] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[1146] Converting data into a standard format

[1147] server

[1148] Consistently format the cleansed data into a predefined standard format.

[1149] Specific examples

[1150] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[1151] Data Visualization

[1152] server

[1153] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[1154] Specific examples

[1155] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[1156] Data storage and sharing

[1157] server

[1158] The final formatted data is securely stored and shared with other systems and users.

[1159] Specific examples

[1160] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[1161] User Emotion Recognition

[1162] server

[1163] The emotion engine analyzes emotions based on user input and operations, and uses the results for data processing and display.

[1164] Specific examples

[1165] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[1166] Emotion-Based User Interface Adjustment

[1167] server

[1168] The emotion engine dynamically changes the display method and interface based on the user's emotions, improving the user experience.

[1169] Specific examples

[1170] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, while if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[1171] This system efficiently converts paper and PDF data into digital format, and realizes a series of processes from data standardization to visualization, storage, and sharing. Furthermore, it can recognize user emotions and process and display data accordingly, improving the user experience.

[1172] The processing flow will be explained below.

[1173] Step 1:

[1174] The server receives data sent from the device or user in the form of paper or PDF files. The received data is saved in a temporary directory, the file format is checked, and it is prepared for passing to the appropriate analysis module.

[1175] Step 2:

[1176] The server uses an OCR engine to extract text from paper documents and image files. OCR technology analyzes the text in the image and extracts it as digital text data.

[1177] Step 3:

[1178] The server then calls the generative AI engine, which uses natural language processing to analyze the extracted text. The generative AI analyzes the text, maps it to specific fields such as name, address, and publication date, and converts it into structured data in JSON format.

[1179] Step 4:

[1180] The server sends the JSON data output by the generative AI to a data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. Duplicate data is removed or merged, and missing values ​​are imputed.

[1181] Step 5:

[1182] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The converted data is then stored.

[1183] Step 6:

[1184] The server passes the converted data to a visualization module for visual display. The visualization module displays the structured data in the form of graphs, maps, etc. The generated visualizations are saved and made available to a web portal.

[1185] Step 7:

[1186] The server securely stores the final formatted data and shares it with other systems and users. The data stored in the database is provided to local governments and related organizations via API, and users can access their own data through a web portal.

[1187] Step 8:

[1188] The server uses an emotion engine to analyze emotions from user input and operations. The emotion engine analyzes user input data in real time and identifies emotions such as stress and dissatisfaction.

[1189] Step 9:

[1190] The server dynamically adjusts the data display method and user interface based on the user's emotions analyzed by the emotion engine. If the user's emotions are positive, detailed data and graphs are displayed, but if the user expresses irritation, the interface switches to a simple and intuitive one.

[1191] Step 10:

[1192] The server provides the user with an adjusted user interface to improve the user experience. The emotion engine monitors the user's emotions in real time and changes the behavior of the interface accordingly, if necessary.

[1193] In this way, the present invention enables efficient digitization and data processing, and also provides an interface that adapts to the user's emotions, thereby achieving improved efficiency and transparency in government services.

[1194] Example 2

[1195] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1196] Government agencies are required to efficiently convert paper and electronic file data into digital format, and then standardize that data into a standard format so that it can be shared with other systems and users. Another challenge with the digitalization of data is recognizing user emotions and reflecting them in system operations to improve the user experience.

[1197] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1198] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for analyzing emotions based on user input and operations and using the results for data processing and display; and means for dynamically changing the user interface based on the emotions. This enables efficient digital conversion, standardization, visual display, and sharing of data, and further improves the operability by dynamically adjusting the user interface to reflect the user's emotions.

[1199] An "administrative agency" is an organization that carries out public duties, such as the national government or local governments.

[1200] "Paper media" refers to the physical format of documents, papers, etc. printed on paper.

[1201] "Electronic file format" refers to digital data formats such as PDF, Word, and Excel.

[1202] "Means for receiving data" refers to the function by which the server acquires data sent from a user or terminal.

[1203] "Generative AI" is an engine that uses machine learning and artificial intelligence techniques to analyze data.

[1204] "Means for extracting text" refers to a function that uses generative AI to extract text information from data.

[1205] "Structured data" is data that is organized according to a particular format or structure.

[1206] A "data cleansing module" is a program for correcting incomplete or erroneous data.

[1207] A "standard format" is a format for arranging data in a unified format.

[1208] "Visual display means" refers to techniques for visualizing data in an easy-to-understand manner.

[1209] "Means for storing final data" refers to a function for safely storing data for a long period of time.

[1210] "Means for sharing with other systems and users" refers to a function that provides the stored data so that it can be used by other systems and users.

[1211] "Means for analyzing emotions" refers to technology that reads emotions from user input and operations.

[1212] "Means for dynamically changing the user interface based on emotions" is a function that changes the screen display and operation method according to the analyzed emotions.

[1213] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes the user's emotions and uses them in data processing. Specific embodiments of the system are described below.

[1214] Entering data

[1215] The server receives data sent from users or devices in the form of paper or PDF files. The received data is temporarily stored. This process uses, for example, an Amazon S3 bucket.

[1216] Examples:

[1217] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[1218] Data analysis with generative AI

[1219] The server then passes the received data to a generative AI engine, which extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology, such as Google Cloud Vision API or Microsoft Azure's OCR service.

[1220] Examples:

[1221] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[1222] Data Cleansing

[1223] The server sends the analysis data output by the generative AI engine to a data cleansing module. The data cleansing module includes a program for correcting incomplete or incorrect data. For example, Apache Spark can be used.

[1224] Examples:

[1225] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[1226] Converting data into a standard format

[1227] The server standardizes the cleansed data into a predefined standard format, ensuring data consistency and compatibility.

[1228] Examples:

[1229] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[1230] Data Visualization

[1231] The server uses a visualization engine, such as Tableau or D3.js, to display the data converted into a standard format in an easy-to-understand format.

[1232] Examples:

[1233] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[1234] Data storage and sharing

[1235] The server securely stores the final formatted data and shares it with other systems and users. For example, the data could be stored in a city database and other departments and agencies could access it through an API.

[1236] Examples:

[1237] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[1238] User Emotion Recognition

[1239] The emotion engine analyzes emotions based on user input and actions, and uses the results for data processing and display, for example, using Amazon Comprehend or IBM Watson.

[1240] Examples:

[1241] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[1242] Emotion-Based User Interface Adjustment

[1243] Based on the user's emotions recognized by the emotion engine, the server dynamically changes the display method and interface, thereby improving the user experience.

[1244] Examples:

[1245] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, whereas if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[1246] Prompt Sentence Examples

[1247] "Extract text from PDF using OCR technology."

[1248] "Analyze the extracted text using natural language processing technology and convert it into JSON format."

[1249] "Analyze emotions based on user actions and adjust your interface accordingly."

[1250] This system efficiently converts paper and PDF data into digital format, enabling a series of processes from data standardization to visualization, storage, and sharing. It also recognizes user emotions and processes and displays data accordingly, improving the user experience.

[1251] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1252] Step 1: User Data Entry

[1253] The user accesses the city hall web portal using their own device and uploads the scanned PDF file of the resident registration card. Specifically, they click the upload button on the web portal and select the corresponding PDF file in the file selection dialog. The input here is the PDF file of the resident registration card, and the output is the completion of uploading it to the server.

[1254] Step 2: Data reception and temporary storage by the server

[1255] The server receives the uploaded PDF file and temporarily stores it in a database or file storage (e.g., Amazon S3 bucket). Specifically, it stores the uploaded file in the specified storage and records the file path. The input here is the PDF file uploaded by the user, and the output is the file path saved in the storage.

[1256] Step 3: Data analysis by server-generated AI

[1257] The server passes the saved PDF file to a generation AI engine (e.g., Google Cloud Vision API or Microsoft Azure's OCR service) to extract text and structure the data. Specifically, it sends the file path to the OCR engine and receives the extracted text data. It then uses natural language processing technology to analyze the text and convert it to JSON format. The input is the temporarily saved PDF file, and the output is the analyzed data in JSON format.

[1258] Step 4: Server-based data cleansing

[1259] The server sends the JSON data obtained from the generative AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data. Specifically, if a field contains an inappropriate string, it deletes it and fills in missing data with appropriate values ​​from an external database. The input is the JSON data obtained from the generative AI engine, and the output is the cleansed data.

[1260] Step 5: Server converts data into standard format

[1261] The server standardizes the cleansed data into a predefined standard format. Specifically, it converts the JSON data into a nationwide standard format for resident information. The input is the cleansed data, and the output is data standardized in the standard format.

[1262] Step 6: Visualizing the data on the server

[1263] The server uses a visualization engine (e.g., Tableau or D3.js) to display the data converted into a standard format in a way that is easy to understand visually. Specifically, it plots resident data on a map and displays the coverage area of ​​administrative services and resident distribution as a dashboard. The input is data in a standard format, and the output is a visualized dashboard.

[1264] Step 7: Storing and Sharing Data on a Server

[1265] The server securely stores the final formatted data and shares it with other systems and users. Specifically, it stores the data in the city's database and provides access to other departments and related organizations through APIs. The input is the visualized data, and the output is the stored data and access rights settings for sharing.

[1266] Step 8: User emotion recognition by the server

[1267] The server's emotion engine analyzes emotions based on user input and operations. Specifically, it measures response times when users operate the system and analyzes stress and dissatisfaction. The input is the user's operation log, and the output is the emotion analysis results.

[1268] Step 9: Emotion-based user interface adjustment by the server

[1269] The server dynamically changes the display method and interface based on the user's emotions recognized by the emotion engine. Specifically, if a positive emotion is recognized, the server provides an interface displaying detailed data and graphs, and if a negative emotion is recognized, the server switches to a simple and intuitive interface. The input is the emotion analysis results, and the output is the adjusted user interface.

[1270] (Application example 2)

[1271] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1272] While conventional data processing systems can efficiently digitize paper and electronic file data held by government agencies and convert them into standard formats, they have the problem of being unable to recognize the user's emotions during operation and dynamically adjust data processing and display accordingly. Furthermore, when it comes to customer service, it has been difficult to instantly analyze paper or PDF data and provide the results in a visually easy-to-understand format.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving data in paper or electronic file format held by an administrative agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module and correcting incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for a robot to respond to customer inquiries; means for recognizing customer emotions and presenting information based on those emotions; and means for displaying the data converted into the standard format on a display in a physical store or on digital signage. This makes it possible to recognize customer emotions and respond appropriately when dealing with customers, and to instantly analyze and visually provide data.

[1274] "Administrative agencies" are public institutions such as the national government and local governments, and are organizations responsible for citizen services and public affairs.

[1275] "Paper media" refers to documents and materials that are printed or handwritten on paper.

[1276] An "electronic file format" is a digital data file format that can be read by a computer, and generally includes PDFs and text files.

[1277] "Generative AI" refers to software or systems that use artificial intelligence technology to process and analyze data.

[1278] "Text extraction" refers to the process of extracting textual information from digital data or images.

[1279] "Structured data" is data that is organized according to specific formats and conventions, and is the format used in databases.

[1280] "Data cleansing" is the process of correcting or removing incomplete or erroneous information from data.

[1281] A "standard format" is a format in which data is standardized based on certain rules, and is used to ensure compatibility between different systems.

[1282] A "visual display" is a representation of data in a visual format, such as a graph, chart, or map, that facilitates understanding of the information.

[1283] A "customer inquiry" is an in-store or online customer requesting information about a product or service.

[1284] A "robot" is a machine or device that uses artificial intelligence and machine learning to operate autonomously and perform specific tasks.

[1285] "Emotion recognition" is a technology that analyzes a user's emotional state from their facial expressions, voice, behavioral patterns, etc.

[1286] A "display" is an electronic device for displaying visual information, such as those used in computers and digital signage.

[1287] "Digital signage" refers to an advertising or information system that uses a display to display digital content.

[1288] This invention is a system for improving the efficiency of customer service in brick-and-mortar stores and providing services based on customer emotions. This system is implemented using the following hardware and software.

[1289] System Configuration

[1290] Hardware

[1291] 1. Robot:

[1292] It has audio input and output functions.

[1293] It is equipped with a high-resolution camera that recognizes customers' facial expressions.

[1294] It has a communication module for sending data from the terminal to the server.

[1295] 2. Server:

[1296] Store and analyze the data.

[1297] It is equipped with a high-performance GPU and is used to run generative AI models.

[1298] 3. Displays and Digital Signage:

[1299] Used to display information to customers in physical stores.

[1300] software

[1301] 1. Python:

[1302] Used for data analysis and running generative AI.

[1303] 2. Flask:

[1304] Used as a server-side web application framework.

[1305] 3. Tesseract OCR:

[1306] Used to extract text from paper and electronic file formats.

[1307] 4. Transformers (BERT):

[1308] Text analysis is performed using natural language processing technology.

[1309] 5. OpenCV:

[1310] Used to analyze customer facial expressions from camera footage.

[1311] 6. Google Speech-to-Text API:

[1312] Used to convert voice input into text.

[1313] 7. Pandas and Regular Expressions:

[1314] Used to cleanse and shape data.

[1315] 8. Matplotlib:

[1316] Used for data visualization.

[1317] 9. PostgreSQL:

[1318] Used to store data.

[1319] Example of operation

[1320] Entering and saving data

[1321] The user (customer) presents the product catalogue as a paper document or PDF file to the robot, which then sends it to the server and temporarily stores it.

[1322] Data analysis and normalization

[1323] The server uses generative AI to extract text from the received data and convert it into structured data. Specifically, it uses Tesseract OCR to extract text from paper documents and analyzes it using Transformer (BERT).

[1324] Data cleansing and conversion to a standard format

[1325] The structured data transformed by the generative AI is sent to a data cleansing module using Pandas and regular expression libraries to correct any incomplete or incorrect data, and then converted into a standard format.

[1326] Recognizing customer emotions and adjusting user interfaces

[1327] The robot uses a camera mounted on it to recognize the customer's facial expressions, analyzes their emotions using OpenCV, and uses the Google Speech-to-Text API to analyze voice input and provide information based on their emotions.

[1328] Data visualization and presentation

[1329] The data converted into a standard format is visualized using Matplotlib and displayed on the robot's display and on digital signage in physical stores.

[1330] Data storage and sharing

[1331] The final data is stored in a PostgreSQL database and can be shared with other systems and users.

[1332] Prompt Sentence Examples

[1333] Please upload the following PDF file: Customer Inquiry Documents.pdf

[1334] In this way, the system of the present invention can quickly and efficiently digitize paper or PDF format data, enabling the provision of customized services based on customer emotions.

[1335] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1336] Step 1:

[1337] The user presents the robot with documents in paper or PDF format, which it then scans and converts into digital format. Specifically, the robot takes a photo of the paper document with a camera and converts it into a digital format.

[1338] Step 2:

[1339] The robot sends the digital data to the server and stores it temporarily. The server uses Flask to store the received digital data in a temporary directory. This input data is still raw image data or PDF files.

[1340] Step 3:

[1341] The server extracts text from the digital data using Tesseract OCR, and the extracted text data is passed to the generative AI engine. In this step, text is obtained as character information from the image data.

[1342] Step 4:

[1343] The server uses the Transformers (BERT) natural language processing model to convert text data into structured data. For example, it maps text to fields such as product name, price, and specifications. The result of this process is structured JSON data.

[1344] Step 5:

[1345] The server uses Pandas and regular expression libraries to cleanse the structured data, correcting incomplete or incorrect information by completing missing values, removing incorrect strings, and adjusting formatting.

[1346] Step 6:

[1347] The server converts the cleansed data into a standard format, ensuring data compatibility across different systems. This output data is also saved in JSON format.

[1348] Step 7:

[1349] The robot uses OpenCV to analyze the customer's face and recognize emotions from their facial expressions. The input data is video footage from a camera mounted on the robot. The results of emotion recognition are output as emotion labels such as "happiness," "sadness," and "irritation."

[1350] Step 8:

[1351] The robot converts the customer's speech into text using the Google Speech-to-Text API, which then analyzes the text data with the generative AI engine to understand the customer's intent. This step uses speech data as input and outputs text data.

[1352] Step 9:

[1353] The server uses Matplotlib to visualize the data in a standard format, specifically converting it into a format that can be displayed as graphs, charts, maps, etc. This visualized data is then displayed on the robot's display and on digital signage in the physical store.

[1354] Step 10:

[1355] The final data is stored in a PostgreSQL database, accessible to other systems and users, for future data analysis and reference.

[1356] In this way, each processing step works in tandem, making it possible to efficiently digitize paper and PDF documents and provide appropriate services based on customer sentiment.

[1357] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1358] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1359] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1360] [Fourth embodiment]

[1361] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1362] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1363] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1364] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1365] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1366] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1367] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1368] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1369] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1370] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1371] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1372] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1373] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1374] The system of the present invention aims to improve the efficiency of data sharing infrastructure by converting data held by government agencies in paper or electronic file format into a format suitable for digital use and formatting it into a standard format.

[1375] Entering data

[1376] server

[1377] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[1378] Specific examples

[1379] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[1380] Data analysis with generative AI

[1381] server

[1382] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[1383] Specific examples

[1384] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[1385] Data Cleansing

[1386] server

[1387] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[1388] Specific examples

[1389] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[1390] Converting data into a standard format

[1391] server

[1392] Consistently format the cleansed data into a predefined standard format.

[1393] Specific examples

[1394] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[1395] Data Visualization

[1396] server

[1397] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[1398] Specific examples

[1399] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[1400] Data storage and sharing

[1401] server

[1402] The final formatted data is securely stored and shared with other systems and users.

[1403] Specific examples

[1404] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[1405] Through this system, non-digital data stored on paper or in PDF format can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[1406] The processing flow will be explained below.

[1407] Step 1:

[1408] The server receives data in the form of paper or PDF files sent from the device or user. The server receives files uploaded via HTTP requests and saves them in a temporary directory. It also checks the format of the uploaded file and prepares it for passing to the appropriate analysis module.

[1409] Step 2:

[1410] The server invokes the generative AI engine to extract text from the received data and convert it into structured data. It also uses the OCR engine to extract text from paper documents and image files, analyzes the text using natural language processing technology, and maps it to specific fields. The analysis results are then saved in a temporary file or database.

[1411] Step 3:

[1412] The server sends the analyzed data output by the generative AI engine to the data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. If there is duplicate data, it is removed or merged, and if there are missing values, it is imputed.

[1413] Step 4:

[1414] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The server then stores the converted data.

[1415] Step 5:

[1416] The server sends the transformed data to the visualization module for easy-to-understand display, passes the structured data to the visualization engine, which displays the data in an appropriate format such as a graph or map, and saves the generated visualization for use in a web portal.

[1417] Step 6:

[1418] The final formatted data is securely stored and shared with other systems and users. The server persists the data stored in the database and shares it with other local governments and related organizations via API as needed. It also allows users to access their own data through a web portal.

[1419] Example 1

[1420] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1421] Modern government agencies have a lot of data in paper or PDF format, and converting this data into a format suitable for digital use requires a large amount of manual work, which is inefficient. It is also difficult to convert data in different formats into a unified standard format, making data integration and centralized management difficult. This hinders efforts to improve the efficiency and transparency of government services.

[1422] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1423] In this invention, the server includes means for receiving and temporarily storing data in paper or electronic file format sent from a user or terminal, means for passing the data received by the server to a generation AI engine and performing text extraction and data structuring using an OCR engine and natural language processing technology, means for sending the structured data converted by the generation AI engine to a data cleansing module and correcting incomplete or erroneous data, means for converting the cleansed data into a predefined standard format, means for visually displaying the data converted into the standard format using a visualization engine, and means for securely storing the final formatted data and sharing it with other systems and users. This makes it possible to efficiently digitize data in different formats and unify them into a standard format.

[1424] A "user" is a person or organization that accesses the system and submits or retrieves data.

[1425] A "terminal" is a device operated by a user, such as a computer, smartphone, or tablet.

[1426] A "server" is a computer system for receiving, processing, storing, and sharing data.

[1427] "Paper media" refers to papers or documents containing information printed on paper.

[1428] An "electronic file format" is a document stored in a digital format, such as a PDF or Word file.

[1429] "Temporary storage" refers to the operation of temporarily storing data before final processing.

[1430] A "generative AI engine" is an artificial intelligence engine that automatically performs text extraction and data structuring.

[1431] An "OCR engine" is a system that uses optical character recognition technology to extract text from image data.

[1432] "Natural language processing technology" is an artificial intelligence technology for understanding and analyzing natural language text.

[1433] "Data structuring" is the process of mapping extracted text to specific fields to make it easier to process and use in a system.

[1434] "Data Cleansing Module" means a software component for correcting and completing incomplete or erroneous data.

[1435] "Standard format" refers to a standard or format for converting different data into a unified format.

[1436] A "visualization engine" is a system for displaying data in a visually easy-to-understand manner.

[1437] "Visual display" refers to the act of visualizing data using text, graphs, maps, etc.

[1438] "Storage" is the act of storing data in a physical or virtual storage device.

[1439] "Sharing" is the act of making data accessible to other systems or users.

[1440] The present invention is a system that converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, thereby streamlining the data sharing infrastructure. This system includes the following components.

[1441] Entering data

[1442] Users can send data in paper or PDF file format from their own devices by logging in to the city hall's web portal and using the file upload function. The server receives the data sent from users and devices and temporarily stores it.

[1443] Specific examples

[1444] A user accesses the city hall web portal and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[1445] Data analysis with generative AI

[1446] The server then passes the temporarily stored data to a generative AI engine, which incorporates an OCR engine and natural language processing technology. The OCR engine extracts the text from the PDF and converts it into structured data using natural language processing technology.

[1447] Specific examples

[1448] The generation AI reads the text in the PDF and extracts it using OCR technology, then uses natural language processing technology to analyze information such as name, address, and issue date, and converts it into JSON format data.

[1449] Data Cleansing

[1450] The server sends the analysis data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data from the analysis data.

[1451] Specific examples

[1452] A data cleansing module is used to remove inappropriate strings from name fields and to impute missing values ​​in address data from a postal code database.

[1453] Converting data into a standard format

[1454] The server standardizes the cleansed data into a standard format.

[1455] Specific examples

[1456] The JSON format data will be converted into a nationwide standard resident information format, making it easily usable by other local governments and institutions.

[1457] Data Visualization

[1458] The server sends the data, formatted in a standard format, to a visualization engine for visual display.

[1459] Specific examples

[1460] Resident data is plotted on a map to visually display the coverage area of ​​government services and the distribution of residents. This data can be used as a dashboard by users and other government agencies.

[1461] Data storage and sharing

[1462] The server securely stores the final formatted data and shares it with other systems and users, allowing users to access their own data by setting access rights.

[1463] Specific examples

[1464] Store the data in a standardized format in a city database, share the information with other departments and related agencies, and allow users to access their resident data through a web portal.

[1465] Example prompts for generative AI models

[1466] An example of a prompt is, "Upload a PDF file, extract the text data from it, and structure it into each field (name, address, publication date, etc.). Convert this into JSON format, cleanse the data, and then unify it into a standard format."

[1467] Through this system, non-digital data such as paper documents and PDF files can be efficiently converted into a format suitable for digital use and converted into a standard format, thereby improving the efficiency and transparency of government services.

[1468] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1469] Step 1:

[1470] A user logs into the city hall web portal using a terminal and performs an operation to upload data in paper or PDF file format. Specifically, the user selects a file and clicks the upload button. This operation sends the data in paper or PDF file format to the server. The input is the paper or PDF file provided by the user, and the output is a file stored on the server.

[1471] Step 2:

[1472] The server saves the data sent by the user in a temporary folder. The server sends a receipt confirmation message to the user to confirm receipt of the data. The input is the file sent by the user, and the output is the file stored in the temporary folder.

[1473] Step 3:

[1474] The server passes the temporarily stored data to the generative AI engine. The generative AI engine uses an OCR engine to extract text from the PDF and uses natural language processing technology to structure the data. Specifically, it analyzes the extracted text and converts information such as name, address, and publication date into JSON format. The input is the file stored in the temporary storage folder, and the output is structured data in JSON format.

[1475] Step 4:

[1476] The server sends the JSON-formatted structured data output by the generation AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data and completes missing values. Specifically, it removes inappropriate strings from name fields and completes missing values ​​in address data from a postal code database. The input is JSON-formatted structured data, and the output is the corrected and completed structured data.

[1477] Step 5:

[1478] The server converts the cleansed structured data into a standard format. Specifically, it converts the cleansed JSON data into a nationwide standard resident information format. This makes it easy for other local governments and organizations to use the data. The input is corrected and supplemented structured data, and the output is data unified in a standard format.

[1479] Step 6:

[1480] The server sends the data converted into a standard format to the visualization engine. The visualization engine converts the data into a format that is easy to display visually. Specifically, it plots the resident data on a map and visually displays the coverage area of ​​administrative services and the distribution of residents. The input is data standardized in a standard format, and the output is visually displayed data.

[1481] Step 7:

[1482] The server securely stores the final formatted data and shares it with other systems and users. Specifically, the data is stored in the city's database and access permissions are set to make it available to other departments and related organizations. Users can also access their own resident data through a web portal. The input is data standardized in a standard format, and the output is securely stored data and shared data.

[1483] (Application example 1)

[1484] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1485] The challenge is to efficiently digitize paper and electronic file data in government agencies and to use that data to provide effective customer support in virtual stores. In particular, there is a need to improve the user experience by quickly and accurately extracting necessary information from uploaded documents and proposing products and services that are suitable for each customer.

[1486] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1487] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems or users; means for extracting necessary information from files uploaded by customers and proposing products and services; and means for visually displaying product information in a virtual store. This enables efficient digitization of government agency data and improved customer support in the virtual store.

[1488] An "administrative agency" is a government organization established for the public interest, including local governments and central government agencies.

[1489] "Paper media" means physical documents in which information is printed or written on paper.

[1490] "Electronic file format" means a digital file format that is stored and transmitted electronically, including formats such as PDFs and Word documents.

[1491] "Means for receiving" refers to a method or device for taking in data on paper or in electronic file format from outside and temporarily storing it.

[1492] "Generative AI" refers to systems or programs that use artificial intelligence technology to analyze input data and perform specific tasks.

[1493] "Text extraction methods" are techniques and devices used to detect and extract textual information from non-text data such as images and PDF files.

[1494] "Structured data" is data that is organized according to a specific format or schema, and is in a form that makes it easy to search and manipulate.

[1495] A "data cleansing module" is software or a system for detecting and correcting or removing inaccurate or incomplete data.

[1496] A "standard format" is a data format that is widely accepted and used by multiple systems and institutions, and is a format defined to ensure compatibility.

[1497] "Visual display means" refers to techniques and devices that visually present information in a form that is easy for humans to understand.

[1498] "Storage and sharing means" refers to methods and devices for securely storing data and sharing information with other systems and users as needed.

[1499] "Customer" means a user who visits the virtual store to use or purchase goods or services.

[1500] An "uploading means" is a method or device that allows a user to transfer files or data from their device to a server.

[1501] "Means for extracting information" refers to the technology or devices used to extract necessary information from uploaded files or data.

[1502] The "means for proposing products and services" refers to a technology or device for recommending related products and services to customers based on the extracted information.

[1503] A "virtual store" is an online store or service platform that exists on the Internet and is independent of a physical location.

[1504] The present invention is a system for digitizing paper or electronic file data held by government agencies and converting it into a standard format, and further applying this technology to customer support within a virtual store. Specific embodiments for implementing the present invention are described below.

[1505] 1. Hardware Configuration

[1506] The main hardware for implementing the present invention includes:

[1507] server

[1508] Smartphone (iOS / Android)

[1509] Smart Glasses

[1510] Head-mounted displays (e.g., Oculus Rift)

[1511] 2. Software Configuration

[1512] The main software used includes:

[1513] OCR engine (e.g. Google Cloud Vision API)

[1514] Natural language processing technology (e.g., GPT-3, BERT)

[1515] Data cleansing module (either in-house or via "Trifacta" etc.)

[1516] Visualization engines (e.g. D3.js, Google Charts)

[1517] 3. Data Entry

[1518] Using a smartphone or head-mounted display, users can access the city hall's web portal or virtual store and upload scanned paper documents or PDF files, such as a scanned PDF of a resident registration card.

[1519] 4. Data analysis using generative AI

[1520] The server receives the uploaded file and temporarily stores it. It then passes the file to a generation AI engine, which uses OCR and natural language processing to extract text and structure the data. For example, it extracts text from a PDF and converts information such as name, address, and publication date into JSON format.

[1521] 5. Data Cleansing

[1522] The analyzed data output by the generative AI engine is processed by the data cleansing module, which corrects incomplete or incorrect data to ensure data accuracy. For example, if an inappropriate string is included in a name field, it is removed, and missing values ​​in address data are filled in using postal codes.

[1523] 6. Converting data into a standard format

[1524] The cleansed data is standardized into a predefined standard format. For example, JSON data can be converted into a nationwide standard format for resident information.

[1525] 7. Data Visualization

[1526] Once the data has been converted into a standard format, it can be displayed using a visualization engine. For example, resident data can be plotted on a map to visually display the coverage area of ​​government services and the distribution of residents.

[1527] 8. Data Storage and Sharing

[1528] The final formatted data can then be securely stored and shared with other systems and users. For example, data converted into a standard format can be stored in a city database and shared with other departments and related agencies.

[1529] 9. Customer Support Applications

[1530] The system receives PDF files of product catalogs and invoices uploaded by customers and analyzes them using a generative AI engine. Based on the analyzed data, it proposes products and services in a virtual store. This information is displayed visually on a smartphone or head-mounted display. For example, it prepares JSON data including product names, model numbers, and price information to make optimal proposals to customers.

[1531] Examples and prompts

[1532] As a concrete example, the following prompt sentence is input into the generative AI model:

[1533] "Please extract the product characteristics and prices listed in this PDF document and convert them into JSON format."

[1534] "Extract the product attributes and prices from this PDF document and convert them into JSON format."

[1535] Template prompt:

[1536] "Based on the contents of the PDF document, extract product characteristics, prices, reviews, etc. and output them in JSON format, which is the standard format for virtual stores."

[1537] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[1538] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1539] Step 1:

[1540] Users access the city hall's web portal or virtual store using a smartphone or head-mounted display and upload PDF or scanned document files, which are then transferred to the server and temporarily stored.

[1541] Input: PDF files or scanned documents uploaded by the user

[1542] Output: A file temporarily saved on the server

[1543] Step 2:

[1544] The server passes the received file to the generative AI engine, which uses OCR technology to extract the text from the file. The generative AI engine uses the Google Cloud Vision API, for example, to detect the text.

[1545] Input: PDF file stored on the server

[1546] Output: Raw text data extracted by OCR

[1547] Step 3:

[1548] The server then analyzes the extracted text data using natural language processing techniques (e.g., GPT-3, BERT) and structures the data, for example, generating data mapped to specific fields such as name, address, and publication date.

[1549] Input: Raw text data extracted by OCR

[1550] Output: Structured data (e.g., JSON format) mapped to fields

[1551] Step 4:

[1552] The server sends the generated structured data to a data cleansing module (e.g., Trifacta) to correct erroneous or incomplete data, such as removing inappropriate strings and imputing missing values.

[1553] Input: Structured text data

[1554] Output: Cleansed and accurate text data

[1555] Step 5:

[1556] The server converts the cleansed data into a standard format, for example, converting resident information into a nationally standard format that can be easily used by other systems and organizations.

[1557] Input: Cleansed text data

[1558] Output: Data converted to a standard format (e.g., JSON data in a common format)

[1559] Step 6:

[1560] The server passes the converted data to a visualization engine (e.g., D3.js, Google Charts) for visualization. For example, it plots the resident data on a map to help users understand it intuitively.

[1561] Input: Data in standard format

[1562] Output: Visualized display data (e.g. dashboards and map plots)

[1563] Step 7:

[1564] The final formatted data is stored securely on a server and shared with other systems and users, for example, by storing it in a resident information database and providing access rights to share the data with related organizations.

[1565] Input: Data converted to a standard format

[1566] Output: Data stored and shared

[1567] Step 8:

[1568] Customers visit the virtual store and upload a PDF file, which the server then passes to a generative AI engine for analysis, extracting and structuring information for, for example, a product catalog or invoice.

[1569] Input: PDF file uploaded by customer

[1570] Output: Parsed and structured product information data

[1571] Step 9:

[1572] The server then proposes appropriate products and services to customers based on the structured product information, which is visually displayed on a smartphone or head-mounted display.

[1573] Input: Structured product information data

[1574] Output: Visual display of suggested products and services

[1575] In this way, the present invention allows for efficient digitization of government data and improved customer support within the virtual store.

[1576] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1577] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes user emotions and uses them to process the data.

[1578] Entering data

[1579] server

[1580] The server receives data sent from users or devices in paper or PDF file format, and temporarily stores the received data.

[1581] Specific examples

[1582] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[1583] Data analysis with generative AI

[1584] server

[1585] The server passes the received data to a generative AI engine that extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology.

[1586] Specific examples

[1587] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[1588] Data Cleansing

[1589] server

[1590] The server sends the analyzed data output by the generative AI engine to a data cleansing module, which corrects incomplete or incorrect data.

[1591] Specific examples

[1592] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[1593] Converting data into a standard format

[1594] server

[1595] Consistently format the cleansed data into a predefined standard format.

[1596] Specific examples

[1597] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[1598] Data Visualization

[1599] server

[1600] The data converted into a standard format is displayed using a visualization engine to make it visually easy to understand.

[1601] Specific examples

[1602] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[1603] Data storage and sharing

[1604] server

[1605] The final formatted data is securely stored and shared with other systems and users.

[1606] Specific examples

[1607] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[1608] User Emotion Recognition

[1609] server

[1610] The emotion engine analyzes emotions based on user input and operations, and uses the results for data processing and display.

[1611] Specific examples

[1612] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[1613] Emotion-Based User Interface Adjustment

[1614] server

[1615] The emotion engine dynamically changes the display method and interface based on the user's emotions, improving the user experience.

[1616] Specific examples

[1617] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, while if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[1618] This system efficiently converts paper and PDF data into digital format, and realizes a series of processes from data standardization to visualization, storage, and sharing. Furthermore, it can recognize user emotions and process and display data accordingly, improving the user experience.

[1619] The processing flow will be explained below.

[1620] Step 1:

[1621] The server receives data sent from the device or user in the form of paper or PDF files. The received data is saved in a temporary directory, the file format is checked, and it is prepared for passing to the appropriate analysis module.

[1622] Step 2:

[1623] The server uses an OCR engine to extract text from paper documents and image files. OCR technology analyzes the text in the image and extracts it as digital text data.

[1624] Step 3:

[1625] The server then calls the generative AI engine, which uses natural language processing to analyze the extracted text. The generative AI analyzes the text, maps it to specific fields such as name, address, and publication date, and converts it into structured data in JSON format.

[1626] Step 4:

[1627] The server sends the JSON data output by the generative AI to a data cleansing module, which identifies incomplete or incorrect data and corrects it based on normalization rules. Duplicate data is removed or merged, and missing values ​​are imputed.

[1628] Step 5:

[1629] The server converts the cleansed data into a standard format, applies a set of rules to map data fields to the standard format, and formats the data into the target data structure. The converted data is then stored.

[1630] Step 6:

[1631] The server passes the converted data to a visualization module for visual display. The visualization module displays the structured data in the form of graphs, maps, etc. The generated visualizations are saved and made available to a web portal.

[1632] Step 7:

[1633] The server securely stores the final formatted data and shares it with other systems and users. The data stored in the database is provided to local governments and related organizations via API, and users can access their own data through a web portal.

[1634] Step 8:

[1635] The server uses an emotion engine to analyze emotions from user input and operations. The emotion engine analyzes user input data in real time and identifies emotions such as stress and dissatisfaction.

[1636] Step 9:

[1637] The server dynamically adjusts the data display method and user interface based on the user's emotions analyzed by the emotion engine. If the user's emotions are positive, detailed data and graphs are displayed, but if the user expresses irritation, the interface switches to a simple and intuitive one.

[1638] Step 10:

[1639] The server provides the user with an adjusted user interface to improve the user experience. The emotion engine monitors the user's emotions in real time and changes the behavior of the interface accordingly, if necessary.

[1640] In this way, the present invention enables efficient digitization and data processing, and also provides an interface that adapts to the user's emotions, thereby achieving improved efficiency and transparency in government services.

[1641] Example 2

[1642] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1643] Government agencies are required to efficiently convert paper and electronic file data into digital format, and then standardize that data into a standard format so that it can be shared with other systems and users. Another challenge with the digitalization of data is recognizing user emotions and reflecting them in system operations to improve the user experience.

[1644] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1645] In this invention, the server includes: means for receiving data in paper or electronic file format held by a government agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module to correct incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for analyzing emotions based on user input and operations and using the results for data processing and display; and means for dynamically changing the user interface based on the emotions. This enables efficient digital conversion, standardization, visual display, and sharing of data, and further improves the operability by dynamically adjusting the user interface to reflect the user's emotions.

[1646] An "administrative agency" is an organization that carries out public duties, such as the national government or local governments.

[1647] "Paper media" refers to the physical format of documents, papers, etc. printed on paper.

[1648] "Electronic file format" refers to digital data formats such as PDF, Word, and Excel.

[1649] "Means for receiving data" refers to the function by which the server acquires data sent from a user or terminal.

[1650] "Generative AI" is an engine that uses machine learning and artificial intelligence techniques to analyze data.

[1651] "Means for extracting text" refers to a function that uses generative AI to extract text information from data.

[1652] "Structured data" is data that is organized according to a particular format or structure.

[1653] A "data cleansing module" is a program for correcting incomplete or erroneous data.

[1654] A "standard format" is a format for arranging data in a unified format.

[1655] "Visual display means" refers to techniques for visualizing data in an easy-to-understand manner.

[1656] "Means for storing final data" refers to a function for safely storing data for a long period of time.

[1657] "Means for sharing with other systems and users" refers to a function that provides the stored data so that it can be used by other systems and users.

[1658] "Means for analyzing emotions" refers to technology that reads emotions from user input and operations.

[1659] "Means for dynamically changing the user interface based on emotions" is a function that changes the screen display and operation method according to the analyzed emotions.

[1660] The system of the present invention not only converts paper or electronic file data held by government agencies into a format suitable for digital use and formats it into a standard format, but also recognizes the user's emotions and uses them in data processing. Specific embodiments of the system are described below.

[1661] Entering data

[1662] The server receives data sent from users or devices in the form of paper or PDF files. The received data is temporarily stored. This process uses, for example, an Amazon S3 bucket.

[1663] Examples:

[1664] The user logs in to the city hall web portal from their device and uploads a scanned PDF file of their resident registration card. The server receives the file and temporarily stores it.

[1665] Data analysis with generative AI

[1666] The server then passes the received data to a generative AI engine, which extracts text and structures the data. This generative AI incorporates an OCR engine and natural language processing technology, such as Google Cloud Vision API or Microsoft Azure's OCR service.

[1667] Examples:

[1668] The generative AI extracts text from the PDF using OCR technology, then analyzes the extracted text using natural language processing technology. The resulting information, such as name, address, and issue date, is converted into JSON format.

[1669] Data Cleansing

[1670] The server sends the analysis data output by the generative AI engine to a data cleansing module. The data cleansing module includes a program for correcting incomplete or incorrect data. For example, Apache Spark can be used.

[1671] Examples:

[1672] If a name field contains inappropriate characters, the data cleansing module removes them. If an address field contains missing values, the postal code is filled in from the database.

[1673] Converting data into a standard format

[1674] The server standardizes the cleansed data into a predefined standard format, ensuring data consistency and compatibility.

[1675] Examples:

[1676] The JSON data will be converted into a nationwide standard resident information format and formatted so that it can be easily used by other local governments and institutions.

[1677] Data Visualization

[1678] The server uses a visualization engine, such as Tableau or D3.js, to display the data converted into a standard format in an easy-to-understand format.

[1679] Examples:

[1680] By plotting resident data on a map, the coverage area of ​​government services and the distribution of residents are visually displayed, which can be used as a dashboard by users and other government agencies.

[1681] Data storage and sharing

[1682] The server securely stores the final formatted data and shares it with other systems and users. For example, the data could be stored in a city database and other departments and agencies could access it through an API.

[1683] Examples:

[1684] The data will be converted into a standard format and stored in the city's database, providing a mechanism for sharing it with other departments and related agencies. Users will be able to access their resident data through a web portal.

[1685] User Emotion Recognition

[1686] The emotion engine analyzes emotions based on user input and actions, and uses the results for data processing and display, for example, using Amazon Comprehend or IBM Watson.

[1687] Examples:

[1688] If data is not displayed promptly while a user is operating the system, the emotion engine analyzes the user's stress or dissatisfaction and adjusts the user interface accordingly. If the user expresses negative emotions, the server responds quickly by changing the priority of data processing.

[1689] Emotion-Based User Interface Adjustment

[1690] Based on the user's emotions recognized by the emotion engine, the server dynamically changes the display method and interface, thereby improving the user experience.

[1691] Examples:

[1692] If the user's sentiment is determined to be positive, the server presents an interface displaying detailed data and graphs, whereas if the user expresses frustration, the server switches to a simpler, more intuitive interface.

[1693] Prompt Sentence Examples

[1694] "Extract text from PDF using OCR technology."

[1695] "Analyze the extracted text using natural language processing technology and convert it into JSON format."

[1696] "Analyze emotions based on user actions and adjust your interface accordingly."

[1697] This system efficiently converts paper and PDF data into digital format, enabling a series of processes from data standardization to visualization, storage, and sharing. It also recognizes user emotions and processes and displays data accordingly, improving the user experience.

[1698] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1699] Step 1: User Data Entry

[1700] The user accesses the city hall web portal using their own device and uploads the scanned PDF file of the resident registration card. Specifically, they click the upload button on the web portal and select the corresponding PDF file in the file selection dialog. The input here is the PDF file of the resident registration card, and the output is the completion of uploading it to the server.

[1701] Step 2: Data reception and temporary storage by the server

[1702] The server receives the uploaded PDF file and temporarily stores it in a database or file storage (e.g., Amazon S3 bucket). Specifically, it stores the uploaded file in the specified storage and records the file path. The input here is the PDF file uploaded by the user, and the output is the file path saved in the storage.

[1703] Step 3: Data analysis by server-generated AI

[1704] The server passes the saved PDF file to a generation AI engine (e.g., Google Cloud Vision API or Microsoft Azure's OCR service) to extract text and structure the data. Specifically, it sends the file path to the OCR engine and receives the extracted text data. It then uses natural language processing technology to analyze the text and convert it to JSON format. The input is the temporarily saved PDF file, and the output is the analyzed data in JSON format.

[1705] Step 4: Server-based data cleansing

[1706] The server sends the JSON data obtained from the generative AI engine to the data cleansing module. The data cleansing module corrects incomplete or incorrect data. Specifically, if a field contains an inappropriate string, it deletes it and fills in missing data with appropriate values ​​from an external database. The input is the JSON data obtained from the generative AI engine, and the output is the cleansed data.

[1707] Step 5: Server converts data into standard format

[1708] The server standardizes the cleansed data into a predefined standard format. Specifically, it converts the JSON data into a nationwide standard format for resident information. The input is the cleansed data, and the output is data standardized in the standard format.

[1709] Step 6: Visualizing the data on the server

[1710] The server uses a visualization engine (e.g., Tableau or D3.js) to display the data converted into a standard format in a way that is easy to understand visually. Specifically, it plots resident data on a map and displays the coverage area of ​​administrative services and resident distribution as a dashboard. The input is data in a standard format, and the output is a visualized dashboard.

[1711] Step 7: Storing and Sharing Data on a Server

[1712] The server securely stores the final formatted data and shares it with other systems and users. Specifically, it stores the data in the city's database and provides access to other departments and related organizations through APIs. The input is the visualized data, and the output is the stored data and access rights settings for sharing.

[1713] Step 8: User emotion recognition by the server

[1714] The server's emotion engine analyzes emotions based on user input and operations. Specifically, it measures response times when users operate the system and analyzes stress and dissatisfaction. The input is the user's operation log, and the output is the emotion analysis results.

[1715] Step 9: Emotion-based user interface adjustment by the server

[1716] The server dynamically changes the display method and interface based on the user's emotions recognized by the emotion engine. Specifically, if a positive emotion is recognized, the server provides an interface displaying detailed data and graphs, and if a negative emotion is recognized, the server switches to a simple and intuitive interface. The input is the emotion analysis results, and the output is the adjusted user interface.

[1717] (Application example 2)

[1718] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1719] While conventional data processing systems can efficiently digitize paper and electronic file data held by government agencies and convert them into standard formats, they have the problem of being unable to recognize the user's emotions during operation and dynamically adjust data processing and display accordingly. Furthermore, when it comes to customer service, it has been difficult to instantly analyze paper or PDF data and provide the results in a visually easy-to-understand format.

[1720] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving data in paper or electronic file format held by an administrative agency; means for extracting text from the received data using a generation AI and converting it into structured data; means for sending the structured data converted by the generation AI to a data cleansing module and correcting incomplete or erroneous data; means for converting the cleansed data into a standard format; means for visually displaying the data converted into the standard format; means for saving the final data and sharing it with other systems and users; means for a robot to respond to customer inquiries; means for recognizing customer emotions and presenting information based on those emotions; and means for displaying the data converted into the standard format on a display in a physical store or on digital signage. This makes it possible to recognize customer emotions and respond appropriately when dealing with customers, and to instantly analyze and visually provide data.

[1721] "Administrative agencies" are public institutions such as the national government and local governments, and are organizations responsible for citizen services and public affairs.

[1722] "Paper media" refers to documents and materials that are printed or handwritten on paper.

[1723] An "electronic file format" is a digital data file format that can be read by a computer, and generally includes PDFs and text files.

[1724] "Generative AI" refers to software or systems that use artificial intelligence technology to process and analyze data.

[1725] "Text extraction" refers to the process of extracting textual information from digital data or images.

[1726] "Structured data" is data that is organized according to specific formats and conventions, and is the format used in databases.

[1727] "Data cleansing" is the process of correcting or removing incomplete or erroneous information from data.

[1728] A "standard format" is a format in which data is standardized based on certain rules, and is used to ensure compatibility between different systems.

[1729] A "visual display" is a representation of data in a visual format, such as a graph, chart, or map, that facilitates understanding of the information.

[1730] A "customer inquiry" is an in-store or online customer requesting information about a product or service.

[1731] A "robot" is a machine or device that uses artificial intelligence and machine learning to operate autonomously and perform specific tasks.

[1732] "Emotion recognition" is a technology that analyzes a user's emotional state from their facial expressions, voice, behavioral patterns, etc.

[1733] A "display" is an electronic device for displaying visual information, such as those used in computers and digital signage.

[1734] "Digital signage" refers to an advertising or information system that uses a display to display digital content.

[1735] This invention is a system for improving the efficiency of customer service in brick-and-mortar stores and providing services based on customer emotions. This system is implemented using the following hardware and software.

[1736] System Configuration

[1737] Hardware

[1738] 1. Robot:

[1739] It has audio input and output functions.

[1740] It is equipped with a high-resolution camera that recognizes customers' facial expressions.

[1741] It has a communication module for sending data from the terminal to the server.

[1742] 2. Server:

[1743] Store and analyze the data.

[1744] It is equipped with a high-performance GPU and is used to run generative AI models.

[1745] 3. Displays and Digital Signage:

[1746] Used to display information to customers in physical stores.

[1747] software

[1748] 1. Python:

[1749] Used for data analysis and running generative AI.

[1750] 2. Flask:

[1751] Used as a server-side web application framework.

[1752] 3. Tesseract OCR:

[1753] Used to extract text from paper and electronic file formats.

[1754] 4. Transformers (BERT):

[1755] Text analysis is performed using natural language processing technology.

[1756] 5. OpenCV:

[1757] Used to analyze customer facial expressions from camera footage.

[1758] 6. Google Speech-to-Text API:

[1759] Used to convert voice input into text.

[1760] 7. Pandas and Regular Expressions:

[1761] Used to cleanse and shape data.

[1762] 8. Matplotlib:

[1763] Used for data visualization.

[1764] 9. PostgreSQL:

[1765] Used to store data.

[1766] Example of operation

[1767] Entering and saving data

[1768] The user (customer) presents the product catalogue as a paper document or PDF file to the robot, which then sends it to the server and temporarily stores it.

[1769] Data analysis and normalization

[1770] The server uses generative AI to extract text from the received data and convert it into structured data. Specifically, it uses Tesseract OCR to extract text from paper documents and analyzes it using Transformer (BERT).

[1771] Data cleansing and conversion to a standard format

[1772] The structured data transformed by the generative AI is sent to a data cleansing module using Pandas and regular expression libraries to correct any incomplete or incorrect data, and then converted into a standard format.

[1773] Recognizing customer emotions and adjusting user interfaces

[1774] The robot uses a camera mounted on it to recognize the customer's facial expressions, analyzes their emotions using OpenCV, and uses the Google Speech-to-Text API to analyze voice input and provide information based on their emotions.

[1775] Data visualization and presentation

[1776] The data converted into a standard format is visualized using Matplotlib and displayed on the robot's display and on digital signage in physical stores.

[1777] Data storage and sharing

[1778] The final data is stored in a PostgreSQL database and can be shared with other systems and users.

[1779] Prompt Sentence Examples

[1780] Please upload the following PDF file: Customer Inquiry Documents.pdf

[1781] In this way, the system of the present invention can quickly and efficiently digitize paper or PDF format data, enabling the provision of customized services based on customer emotions.

[1782] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1783] Step 1:

[1784] The user presents the robot with documents in paper or PDF format, which it then scans and converts into digital format. Specifically, the robot takes a photo of the paper document with a camera and converts it into a digital format.

[1785] Step 2:

[1786] The robot sends the digital data to the server and stores it temporarily. The server uses Flask to store the received digital data in a temporary directory. This input data is still raw image data or PDF files.

[1787] Step 3:

[1788] The server extracts text from the digital data using Tesseract OCR, and the extracted text data is passed to the generative AI engine. In this step, text is obtained as character information from the image data.

[1789] Step 4:

[1790] The server uses the Transformers (BERT) natural language processing model to convert text data into structured data. For example, it maps text to fields such as product name, price, and specifications. The result of this process is structured JSON data.

[1791] Step 5:

[1792] The server uses Pandas and regular expression libraries to cleanse the structured data, correcting incomplete or incorrect information by completing missing values, removing incorrect strings, and adjusting formatting.

[1793] Step 6:

[1794] The server converts the cleansed data into a standard format, ensuring data compatibility across different systems. This output data is also saved in JSON format.

[1795] Step 7:

[1796] The robot uses OpenCV to analyze the customer's face and recognize emotions from their facial expressions. The input data is video footage from a camera mounted on the robot. The results of emotion recognition are output as emotion labels such as "happiness," "sadness," and "irritation."

[1797] Step 8:

[1798] The robot converts the customer's speech into text using the Google Speech-to-Text API, which then analyzes the text data with the generative AI engine to understand the customer's intent. This step uses speech data as input and outputs text data.

[1799] Step 9:

[1800] The server uses Matplotlib to visualize the data in a standard format, specifically converting it into a format that can be displayed as graphs, charts, maps, etc. This visualized data is then displayed on the robot's display and on digital signage in the physical store.

[1801] Step 10:

[1802] The final data is stored in a PostgreSQL database, accessible to other systems and users, for future data analysis and reference.

[1803] In this way, each processing step works in tandem, making it possible to efficiently digitize paper and PDF documents and provide appropriate services based on customer sentiment.

[1804] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1805] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1806] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1807] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1808] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1809] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1810] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1811] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1812] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1813] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1814] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1815] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1816] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1817] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1818] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1819] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1820] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1821] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1822] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1823] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1824] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1825] The following is further disclosed regarding the above embodiment.

[1826] (Claim 1)

[1827] A means for receiving data held by government agencies in paper or electronic file format;

[1828] A means for extracting text from received data using generative AI and converting it into structured data;

[1829] A means for sending the structured data transformed by the generative AI to a data cleansing module to correct incomplete or erroneous data;

[1830] a means for converting the cleansed data into a standard format;

[1831] a means for visually displaying the data converted into a standard format;

[1832] A means to store and share the final data with other systems and users;

[1833] A system including:

[1834] (Claim 2)

[1835] 10. The system of claim 1, further comprising means for extracting text using an OCR engine if the received data is on paper.

[1836] (Claim 3)

[1837] 2. The system of claim 1, wherein the generation AI has means for analyzing the extracted text using natural language processing techniques and mapping it to specific fields.

[1838] "Example 1"

[1839] (Claim 1)

[1840] A means for the server to receive and temporarily store data in paper or electronic file format transmitted from a user or a terminal;

[1841] The server passes the received data to the AI ​​engine, and uses the OCR engine and natural language processing technology to extract text and structure the data.

[1842] a means for sending the structured data transformed by the generative AI engine to a data cleansing module to correct incomplete or erroneous data;

[1843] a means for converting the cleansed data into a predefined standard format;

[1844] a means for visually displaying the data converted into the standard format using a visualization engine;

[1845] A means to securely store the final formatted data and share it with other systems and users, and

[1846] A system including:

[1847] (Claim 2)

[1848] 10. The system of claim 1, further comprising means for the server to extract text using an OCR engine if the received data is on paper.

[1849] (Claim 3)

[1850] 2. The system of claim 1, wherein the generation AI has means for analyzing the extracted text using natural language processing techniques and mapping it to specific fields.

[1851] "Application Example 1"

[1852] (Claim 1)

[1853] A means for receiving data held by government agencies in paper or electronic file format;

[1854] A means for extracting text from received data using generative AI and converting it into structured data;

[1855] A means for sending the structured data transformed by the generative AI to a data cleansing module to correct incomplete or erroneous data;

[1856] a means for converting the cleansed data into a standard format;

[1857] a means for visually displaying the data converted into a standard format;

[1858] A means to store and share the final data with other systems and users;

[1859] A means to extract necessary information from files uploaded by customers and propose products and services;

[1860] a means for visually displaying product information within the virtual store;

[1861] A system including:

[1862] (Claim 2)

[1863] 10. The system of claim 1, further comprising means for extracting text using an OCR engine if the received data is on paper.

[1864] (Claim 3)

[1865] 2. The system of claim 1, wherein the generation AI has means for analyzing the extracted text using natural language processing techniques and mapping it to specific fields.

[1866] "Example 2: Combining Emotion Engines"

[1867] (Claim 1)

[1868] A means for receiving data held by government agencies in paper or electronic file format;

[1869] A means for extracting text from received data using generative AI and converting it into structured data;

[1870] A means for sending the structured data transformed by the generative AI to a data cleansing module to correct incomplete or erroneous data;

[1871] a means for converting the cleansed data into a standard format;

[1872] a means for visually displaying the data converted into a standard format;

[1873] A means to store and share the final data with other systems and users;

[1874] A means of analyzing emotions based on user input and operations and using the results for data processing and display;

[1875] means for dynamically modifying a user interface based on emotions;

[1876] A system including:

[1877] (Claim 2)

[1878] 10. The system of claim 1, further comprising means for extracting text using an OCR engine if the received data is on paper.

[1879] (Claim 3)

[1880] 2. The system of claim 1, wherein the generation AI has means for analyzing the extracted text using natural language processing techniques and mapping it to specific fields.

[1881] "Application example 2 when combining emotion engines"

[1882] (Claim 1)

[1883] A means for receiving data held by government agencies in paper or electronic file format;

[1884] A means for extracting text from received data using generative AI and converting it into structured data;

[1885] A means for sending the structured data transformed by the generative AI to a data cleansing module to correct incomplete or erroneous data;

[1886] a means for converting the cleansed data into a standard format;

[1887] a means for visually displaying the data converted into a standard format;

[1888] A means to store and share the final data with other systems and users;

[1889] A means for the robot to respond to customer inquiries;

[1890] A means for recognizing customer emotions and presenting information based on those emotions;

[1891] A means to display the data converted into a standard format on displays and digital signage within physical stores,

[1892] A system including:

[1893] (Claim 2)

[1894] 10. The system of claim 1, further comprising means for extracting text using an OCR engine if the received data is on paper.

[1895] (Claim 3)

[1896] 2. The system of claim 1, wherein the generation AI has means for analyzing the extracted text using natural language processing techniques and mapping it to specific fields. [Explanation of symbols]

[1897] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving data held by government agencies in paper or electronic file format; A means for extracting text from received data using generative AI and converting it into structured data; A means for sending the structured data transformed by the generative AI to a data cleansing module to correct incomplete or erroneous data; a means for converting the cleansed data into a standard format; a means for visually displaying the data converted into a standard format; A means to store and share the final data with other systems and users; A system including:

2. 10. The system of claim 1, further comprising means for extracting text using an OCR engine if the received data is on paper.

3. 2. The system of claim 1, wherein the generation AI has means for analyzing the extracted text using natural language processing techniques and mapping it to specific fields.

Images