System

Abstract

A system is provided.SOLUTION: The system includes a means for reading a contract document, a means for extracting text data from the read contract document, a means for analyzing the extracted text data and extracting important information, a means for visually displaying the extracted important information, and a means for detecting an inappropriate clause in the contract document and presenting a correction proposal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, it is not easy for individuals to accurately understand contracts, increasing the risk of entering into unfavorable terms. While there are more opportunities to enter into contracts in a variety of situations, such as purchasing real estate, buying a car, attending English conversation classes, and engaging in leisure activities, the content of contracts is often difficult to understand and requires specialized knowledge, making it difficult for individuals to understand them voluntarily. As a result, contract disputes are more likely to occur, and methods to prevent them from occurring are needed. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides a system as set forth in the following claims. The system includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data to extract important information, means for visually displaying the extracted important information, and means for detecting inappropriate clauses in the contract document and suggesting amendments. This makes it easier for the user to intuitively understand the contents of the contract and provides amendments to inappropriate clauses, thereby reducing the risk of contract disputes.

[0006] A "contract document" is a document that describes the contents of a contract and specifies the terms by which the contracting parties are legally bound.

[0007] "Reading means" refers to the technology or device used to recognize documents or images and capture them as digital data.

[0008] "Text data" is a collection of textual information in a data format that can be electronically stored, processed, and analyzed.

[0009] "Analysis" is the process of examining data in detail and extracting meaning and patterns.

[0010] "Material information" refers to the key elements and conditions of the contract in the contract documents, which are particularly important for the performance of the contract.

[0011] "Visually displaying" means presenting data or information in a visual format such as a graph, chart, or flowchart.

[0012] An "inappropriate clause" refers to a clause that is unfavorable to the parties to the contract or that may conflict with the law.

[0013] "Amendments" refer to proposals for alternatives or improvements to the inappropriate clauses that have been detected. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[0036] User operations

[0037] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[0038] Sending and Reading Data

[0039] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[0040] Text data analysis

[0041] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[0042] Visual display of information

[0043] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[0044] Detecting inappropriate clauses and suggesting corrections

[0045] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[0046] User Feedback

[0047] The terminal presents the visual information and suggested amendments received from the server to the user, allowing the user to understand the contents of the contract and take action to avoid unfavorable terms.

[0048] Specific examples

[0049] Specific usage examples are shown below.

[0050] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[0051] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a modification: "Generally, this is reasonable, but three months' notice may be required." With this information, the user can review the contract and take appropriate action to avoid unfavorable conditions.

[0052] As described above, the present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and is a very useful tool for users.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[0056] The device will activate the camera function and temporarily save the image taken by the user, and will also save the uploaded PDF file within the app.

[0057] Step 2:

[0058] The device sends an image or PDF file of the contract to the server.

[0059] The device converts the saved image or PDF file into the appropriate format and sends it to the server via the API.

[0060] Step 3:

[0061] The server extracts the text data of the contract using OCR technology.

[0062] The server inputs the image or PDF file into the OCR tool, performs character recognition, extracts the recognized text data, and saves it.

[0063] Step 4:

[0064] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[0065] The server inputs the text data into an NLP engine and extracts important elements such as contract period, rent, security deposit, and cancellation conditions.

[0066] Step 5:

[0067] The server converts the extracted important information into a visually understandable format.

[0068] The server uses tools (e.g., D3.js) to convert the extracted information into flowcharts and graphical diagrams.

[0069] Step 6:

[0070] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[0071] The server refers to predefined rules and databases to analyze the detected problems.

[0072] Step 7:

[0073] The server generates corrections and recommendations for inappropriate clauses.

[0074] The server creates a revised proposal and presents favorable terms to the user.

[0075] Step 8:

[0076] The terminal displays the visualized contract details and amendment proposals to the user.

[0077] The device displays flowcharts, diagrams, and suggested revisions on the app's UI for the user to review.

[0078] Step 9:

[0079] The user reviews the presented information and deepens their understanding.

[0080] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[0081] Example 1

[0082] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0083] Conventional contract reading systems lack the support required to easily understand the content and quickly identify inappropriate clauses. As a result, users spend a lot of time and effort trying to accurately understand the content of the contract, and especially those without legal expertise run the risk of signing unfavorable clauses without realizing it. Another issue is that the process of generating proposed amendments to contracts is time-consuming and inefficient.

[0084] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0085] In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, and means including a generative AI model for generating suggested amendments based on prompts entered by a user. This makes it possible to quickly and accurately understand the contents of the contract document, identify inappropriate clauses, and present suggested amendments.

[0086] "Contract documents" refers to documents that detail business or legal arrangements, including contracts, agreements, and memoranda of understanding.

[0087] "Means for reading" refers to devices or systems for obtaining the contents of contract documents as digital data, such as scanners and cameras.

[0088] "Extraction methods" refers to techniques or processes used to extract specific information from digital data, including optical character recognition (OCR).

[0089] "Text data" refers to data that represents the text information in a contract document in digital form.

[0090] "Means of analysis" refers to technologies and systems for analyzing text data and extracting meanings and patterns according to the purpose. Natural language processing (NLP) technology is a typical example.

[0091] "Material Information" refers to elements of the contract that are considered particularly important, such as the contract duration, fees, and cancellation conditions.

[0092] "Visual display means" refers to techniques or devices for displaying extracted information in a form that is easily understandable to the user, including, for example, flow charts or diagrams.

[0093] An "inappropriate clause" refers to a clause in a contract document that is disadvantageous to the user or that is considered unfair.

[0094] "Means for suggesting amendments" refers to a technology or system for automatically generating and providing to a user suggestions for amending inappropriate clauses.

[0095] A "generative AI model" refers to an artificial intelligence model that automatically generates document content and suggestions based on user input.

[0096] A "prompt" is a text instruction you provide to a generative AI model, which then generates a specific output.

[0097] MODE FOR CARRYING OUT THE INVENTION

[0098] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[0099] Hardware and software used

[0100] Users can start using the system by taking a photo of the contract using their smartphone or uploading a PDF file. The device (smartphone) acquires the image data or PDF file and sends it to the server. The server receives this data and extracts the contents of the contract as text data using optical character recognition (OCR) technology. Tesseract OCR can be used for OCR.

[0101] The server analyzes the extracted text data using natural language processing (NLP) techniques, such as frameworks like spaCy, to identify and classify key elements of the contract (such as contract duration, fees, and cancellation conditions).

[0102] The server then converts the extracted key information into a visually understandable format, which can be visualized using Matplotlib or other chart generators, such as flowcharts and diagrams.

[0103] Furthermore, the server analyzes the contract clauses and uses a rule-based engine or machine learning model (e.g., Scikit-learn) to detect whether they contain inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, a generative AI model (e.g., GPT-3) is used to generate suggested amendments. Based on the prompt text entered by the user, the AI ​​model provides appropriate amendments.

[0104] Specific examples

[0105] Consider a scenario where a user takes a photo of a residential rental contract and uploads it to the system. The user launches a smartphone application and either takes a photo of the contract or uploads a PDF file. The device then sends this image data or PDF file to a server. The server processes the received data with an OCR engine (Tesseract OCR) and extracts text information such as "Contract period: January 1, 2023 to December 31, 2023."

[0106] The server then uses an NLP engine (spaCy) to analyze this text data and identify key elements such as "contract term," "rent," "security deposit," "cancellation conditions," etc. The server then visualizes these elements as flowcharts and diagrams and displays them to the user.

[0107] Next, the server detects that the cancellation condition is "one month's notice" and uses a rule-based engine to determine that this is generally reasonable, but uses a generative AI model (GPT-3) to generate a revised suggestion that "three months' notice may be required."

[0108] As an example of a prompt sentence, a user can input the following into the generative AI model: "Regarding the termination conditions in a residential rental agreement, there is a clause requiring one month's notice. Please provide a general amendment to this clause.", and appropriate amendments will be generated.

[0109] The terminal presents the user with visual information and suggested amendments received from the server, and by reviewing this, the user is able to understand the contents of the contract and take specific action to avoid unfavorable terms.

[0110] As described above, this system provides comprehensive support for individuals to quickly and accurately understand contracts, and is a very useful tool for users.

[0111] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0112] Program processing flow

[0113] Step 1:

[0114] The user launches the smartphone application and takes a picture of the contract or uploads a PDF file.

[0115] Input: Image or PDF file of the contract

[0116] Specific behavior: After launching the application, a user interface will be displayed, offering the option to take a photo of the contract or select a PDF file. The user selects one of the options to complete the operation.

[0117] Step 2:

[0118] The device acquires the captured image data or uploaded PDF file and sends it to the server.

[0119] Input: Image or PDF file of the contract

[0120] Output: Image data or PDF file sent to the server

[0121] Specific behavior: The device sends the selected image data or PDF file to the server using a secure protocol (e.g., HTTPS).

[0122] Step 3:

[0123] The server extracts text data from the data it receives using an OCR (optical character recognition) engine.

[0124] Input: Image data or PDF file

[0125] Output: Extracted text data

[0126] Specific operation: The server inputs the received data into the Tesseract OCR engine, extracts the contents of the contract document as text data, and saves the extracted text as a string.

[0127] Step 4:

[0128] The server analyzes the extracted text data using natural language processing (NLP) technology.

[0129] Input: Extracted text data

[0130] Output: Key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.)

[0131] What it does: The server uses an NLP engine such as spaCy to parse the text data, identify key elements of the contract, and classify them, allowing for efficient extraction of relevant information.

[0132] Step 5:

[0133] The server converts the analyzed important information into a visually understandable format.

[0134] Input: Key elements of the contract

[0135] Output: Visualized information (e.g., flowcharts, diagrams)

[0136] Specific operation: The server uses a chart generator such as Matplotlib to generate flowcharts and diagrams to visually display the extracted key information.

[0137] Step 6:

[0138] The server analyzes the clauses in the contract document and checks for inappropriate clauses.

[0139] Input: Key elements of the contract

[0140] Output: Proposed amendments to inappropriate clauses

[0141] How it works: The server uses a rule-based engine and machine learning models to inspect the clauses in the contract document and detect inappropriate terms or clauses that are disadvantageous to the user.

[0142] Step 7:

[0143] The server uses a generative AI model to generate suggested amendments to inappropriate clauses.

[0144] Input: Incorrect clause

[0145] Output: Proposed amendments to inappropriate clauses

[0146] What it does: The server uses a generative AI model (e.g., GPT-3) to generate appropriate correction suggestions based on the prompt entered by the user.

[0147] Example: An example of a prompt might be, "My residential lease agreement has a one-month notice clause regarding termination terms. Please provide a general amendment to this."

[0148] Step 8:

[0149] The terminal presents the visual information and correction suggestions received from the server to the user.

[0150] Input: Visualized information, suggested corrections

[0151] Output: Information displayed in the user interface

[0152] Specific operation: The device displays the data received from the server, allowing the user to easily check the contents of the contract and proposed amendments.

[0153] Step 9:

[0154] The user reviews the information presented and takes the necessary action.

[0155] Input: Visualized information, suggested corrections

[0156] Output: User action (e.g., contract renegotiation, amendment adoption)

[0157] Specific Action: The user understands the contract based on the information presented and takes the necessary action to remove any unfavorable terms. In addition, this may include consulting a lawyer or renegotiating the contract.

[0158] Through these steps, the system is able to quickly and accurately understand the contents of a contract, identify inappropriate clauses, and suggest amendments.

[0159] (Application example 1)

[0160] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0161] In modern society, understanding contracts is extremely important, but many people find it difficult to understand them because they are technically complex. Furthermore, if an inappropriate clause is included, it is even more difficult to identify and correct it. This problem is particularly pronounced for users of electronic payment services, who face the challenge of finding appropriate action to avoid entering into contracts with unfavorable terms.

[0162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0163] In this invention, the server includes a means for reading the contract document, a means for extracting text data, a means for analyzing the text data using natural language processing technology and extracting important information, a means for generating amendments and improvement proposals for inappropriate clauses, and a means for visually presenting the analysis results and amendment proposals to the user in a dashboard format, thereby enabling the content of the contract to be quickly and accurately understood, inappropriate conditions to be detected, and amendment proposals to be presented.

[0164] A "contract document" is a written legal agreement that specifies the rights and obligations of the parties.

[0165] "Reading means" refers to a means for mechanically or electronically reading the contents of a contract document.

[0166] "Text data" is character string data extracted from a contract document, and is used for content analysis and display.

[0167] "Extraction means" refers to methods or techniques for extracting specific information or data from the original document or data.

[0168] "Important information" refers to the contents and key elements of a contract document that require particular attention.

[0169] The "visual display means" is a means for displaying extracted information in a form that can be directly seen and understood by the user.

[0170] "Inappropriate clauses" refer to clauses in a contract document that violate the law or contain terms that are disadvantageous to the user.

[0171] An "amendment" is a new, appropriate clause proposed to correct an inappropriate clause.

[0172] "Natural language processing technology" is a technology that enables machines to understand, interpret, and generate human language.

[0173] A "dashboard format" is a visual interface format in which multiple pieces of information are arranged so that they can be viewed at a glance.

[0174] "User" refers to a person who uses this system to review and analyze contract documents.

[0175] A "server" is a centralized computer system for processing and storing data.

[0176] The present invention relates to a system that helps users accurately understand contracts and detect and correct inappropriate clauses. The system reads contract documents, extracts text data, visually displays important information, detects inappropriate clauses, and suggests corrections.

[0177] System configuration

[0178] Hardware

[0179] Smartphone: Equipped with a camera function for taking photos of contract documents and internet connection function.

[0180] Server: A centralized computer system for processing and storing data.

[0181] software

[0182] OCR software: Use Tesseract OCR to extract text data from images.

[0183] NLP software: Uses Google Cloud Natural Language API to analyze extracted text data and identify key elements.

[0184] Front-end software: Flutter is used to provide an interface that visually displays the extracted information to the user.

[0185] Data processing and calculation

[0186] 1. Data Collection:

[0187] Users can either take a photo of the contract document using their smartphone's camera or upload an existing PDF file, and the data is captured on the smartphone and sent to the server.

[0188] 2. Text extraction:

[0189] The image data or PDF file of the contract document sent by the terminal to the server is converted into character string data using Tesseract OCR on the server.

[0190] 3. Data Analysis:

[0191] The converted text data is then analyzed using the Google Cloud Natural Language API to identify key elements of the contract document (e.g., contract term, fee, cancellation terms, etc.).

[0192] 4. Visual Indication:

[0193] The extracted important information is visually displayed in a dashboard format using Flutter, allowing users to intuitively understand the contents of the contract.

[0194] 5. Detecting inappropriate clauses and suggesting amendments:

[0195] The server uses NLP technology to analyze the clauses in the contract document and check for inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, the server generates suggestions for amendments or improvements to the clause.

[0196] 6. User Feedback:

[0197] The visual information and suggested amendments sent from the server are displayed on the user's smartphone, allowing the user to confirm the information and review the contract.

[0198] Specific examples

[0199] Example prompt sentence:

[0200] python

[0201] OCR processing

[0202] import pytesseract

[0203] from PIL import Image

[0204] def extract_text_from_image(image_path):

[0205] image = Image.open(image_path)

[0206] text = pytesseract.image_to_string(image, lang='jpn')

[0207] return text

[0208] Text analytics

[0209] from google.cloud import language_v1

[0210] def analyze_text(text):

[0211] client = language_v1.LanguageServiceClient()

[0212] document = language_v1.Document(content=text, type_=language_v1.Document.Type.PLAIN_TEXT)

[0213] response = client.analyze_entities(document=document)

[0214] return response

[0215] Extracting and visualizing key elements

[0216] import json

[0217] def extract_and_visualize(response):

[0218] elements = {}

[0219] for entity in response.entities:

[0220] elements[entity.name] = language_v1.Entity.Type(entity.type_).name

[0221] return json.dumps(elements, indent=2)

[0222] Examples of prompts:

[0223] image_path = 'contract_image.jpg'

[0224] extracted_text = extract_text_from_image(image_path)

[0225] response = analyze_text(extracted_text)

[0226] visualization_data = extract_and_visualize(response)

[0227] print(visualization_data)

[0228] This system allows users to quickly understand the contents of contract documents, easily detect inappropriate clauses, and receive suggested amendments. This system is particularly useful for users of electronic payment services, and provides support for accurately understanding the contents of contracts.

[0229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0230] Step 1:

[0231] Users can either take a photo of the contract document using their smartphone camera or upload an existing PDF file. The image or PDF file is then saved on the smartphone.

[0232] Step 2:

[0233] The terminal sends the image data or PDF file of the acquired contract document to the server, which then receives the contract document as input data.

[0234] Step 3:

[0235] The server converts the received image data or PDF file of the contract document into text data using OCR software. Specifically, it uses Tesseract OCR to extract text data from the image. This outputs the contents of the contract document as character string data.

[0236] Step 4:

[0237] The server then analyzes the converted text data using NLP software (Google Cloud Natural Language API). By analyzing this input text, important elements of the contract (such as contract period, fees, and cancellation conditions) are identified and classified.

[0238] Step 5:

[0239] The server uses front-end software (Flutter) to generate a visual interface in the form of a dashboard to visually display the analyzed important information, allowing the analysis results to be displayed in a user-friendly format.

[0240] Step 6:

[0241] The server also uses NLP technology to detect inappropriate clauses in the contract documents and generate amendments or improvement proposals for those clauses. Specifically, it checks each clause in the contract to see if it violates the law or contains terms that are disadvantageous to the user.

[0242] Step 7:

[0243] The visual information and suggested amendments sent from the server are displayed on the smartphone, allowing the user to review the information, examine the contract details, and take appropriate action.

[0244] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0245] The present invention relates to a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. In particular, the present invention not only visually displays the contents of a contract, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[0246] User operations

[0247] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[0248] Sending and Reading Data

[0249] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[0250] Text data analysis

[0251] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[0252] Visual display of information

[0253] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[0254] Detecting inappropriate clauses and suggesting corrections

[0255] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[0256] Emotion Engine Functions

[0257] A feature of the present invention is the incorporation of an emotion engine. The terminal recognizes emotions by analyzing the user's facial expressions and tone of voice while the user is reviewing the contract. For example, if the user shows signs of discomfort or confusion regarding the contents of the contract, the emotion engine detects this and notifies the server.

[0258] Adjusting how information is presented

[0259] The server dynamically adjusts the presentation of the contract based on the emotional data obtained from the emotion engine. For example, if the user is confused, it may provide simpler illustrations or additional annotations to aid understanding. If the user expresses surprise or displeasure, it may highlight suggested amendments to inappropriate clauses.

[0260] User Feedback

[0261] The device then presents the user with visual information and suggested revisions received from the server, as well as a presentation tailored to the user's emotions. By reviewing this information, the user can understand the contents of the contract and take action to avoid unfavorable terms.

[0262] Specific examples

[0263] Specific usage examples are shown below.

[0264] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[0265] Furthermore, the server detects that the cancellation condition is "one month's notice" and presents this along with a suggested amendment: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the details of the illustrations to make the clause easier to understand.

[0266] Based on this information, users can review their contracts and take appropriate measures to avoid unfavorable terms.

[0267] The present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and by dynamically responding to the user's emotions, it is a system that can provide information that is more suited to each individual user.

[0268] The processing flow will be explained below.

[0269] Step 1:

[0270] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[0271] The device will activate the camera function and save the image taken by the user, and will also save the uploaded PDF file within the app.

[0272] Step 2:

[0273] The terminal sends the image data or PDF file of the contract to the server.

[0274] The device converts the stored image or PDF file into the appropriate format and sends it to the server using a secure protocol.

[0275] Step 3:

[0276] The server extracts the text data of the contract using OCR technology.

[0277] The server inputs the transmitted image or PDF file into the OCR tool, performs character recognition, and saves the extracted text data.

[0278] Step 4:

[0279] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[0280] The server inputs the text data of the contract into an NLP engine and extracts important elements such as the contract period, rent, security deposit, and cancellation conditions. This extracted information is then classified and saved.

[0281] Step 5:

[0282] The server converts the extracted important information into a visually understandable format.

[0283] The server visualizes the extracted data as flowcharts and diagrams using graphical tools (e.g., D3.js).

[0284] Step 6:

[0285] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[0286] The server analyzes the content based on pre-set rules and databases to detect inappropriate clauses.

[0287] Step 7:

[0288] The server generates corrections and recommendations for inappropriate clauses.

[0289] The server creates a correction proposal for the detected inappropriate clause and presents advantageous terms to the user.

[0290] Step 8:

[0291] The device analyzes the user's facial expressions and tone of voice to recognize emotions.

[0292] The device uses a built-in camera and microphone to analyze the user's facial expressions and voice in real time, and the emotion engine uses this data to recognize the user's emotions (e.g., confusion, anger, interest).

[0293] Step 9:

[0294] The server dynamically adjusts the presentation of the contract content based on the recognized emotion data.

[0295] The server adjusts the visualization depending on the user's emotional state, such as increasing the level of detail or simplifying the presentation.

[0296] Step 10:

[0297] The terminal presents the adjusted information to the user.

[0298] Based on the adjustment information received from the server, the terminal visually displays the contract contents and suggests amendments to inappropriate clauses.

[0299] Step 11:

[0300] The user reviews the presented information and deepens their understanding.

[0301] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[0302] Example 2

[0303] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0304] Conventional contract management systems have the problem of making it difficult to accurately understand the contents of a contract, and in particular, making it easy to overlook inappropriate clauses. Furthermore, there are no dynamic systems that present information based on the user's emotions, which often leaves users confused or takes a long time to understand the contract. For this reason, there is a need for a system that allows users to quickly and accurately understand the contents of a contract and take appropriate action to avoid unfavorable terms.

[0305] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring input contract document data, a means for transmitting the acquired contract document data to the server, a means for using character recognition technology to extract text data from the transmitted contract document data, a means for analyzing the extracted text data using natural language processing technology, a means for extracting important information from the analyzed text data and visually displaying it, a means for detecting inappropriate clauses contained in the contract document and generating proposed amendments, a means for recognizing the user's emotions and dynamically adjusting the information presentation method based on the emotion data, and a means for providing the adjusted information and proposed amendments as feedback to the user. This allows the user to quickly and accurately understand the contents of the contract, easily detect inappropriate clauses, and obtain optimal proposed amendments. Furthermore, dynamically adjusting the information presentation method aids the user's understanding and allows them to more efficiently grasp the contents of the contract.

[0306] "Input contract document data" refers to the image data or electronic document file of the contract provided to the system by the user.

[0307] "Means of acquisition" refers to the function for collecting and storing contract document data photographed or uploaded by the user.

[0308] "Means for sending" refers to the function for sending contract document data from the terminal to the server.

[0309] "Character recognition technology" refers to optical character recognition (OCR) technology for extracting text data from images and PDFs.

[0310] "Natural language processing technology" refers to technology that analyzes extracted text data and identifies important elements of a contract, such as analyzing nouns and verbs and understanding the context.

[0311] "Means for analysis" refers to the ability to process text data using natural language processing and extract important information.

[0312] "Visual display means" refers to a function for displaying important analyzed information in a user-friendly format (such as a flow chart or diagram).

[0313] "Inappropriate clauses" refers to terms and conditions contained in the contract documents that are unfavorable to the user or inappropriate content.

[0314] "Proposed Amendments" refers to the proposed amendments to improve an inappropriate provision.

[0315] "Emotional Data" refers to data such as a user's facial expressions and voice tone collected using emotion recognition technology.

[0316] "Dynamic adjustment means" refers to a function that changes the way information is presented based on the user's emotional data, making it easier for the user to understand the contract.

[0317] "Means for providing feedback" refers to the ability to provide tailored information or suggested revisions to the user to assist them in understanding the contents of the contract.

[0318] The present invention is a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. This system not only visually displays the contents of contracts, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[0319] System configuration and program processing

[0320] 1. Enter contract data

[0321] The user first launches the smartphone application and then takes a photo of the contract or uploads a PDF file, which saves the contract data on the device.

[0322] 2. Data transmission and OCR processing

[0323] The device will send the captured image data or uploaded PDF file to the server, which will then use optical character recognition (OCR) technology to extract the text data. Tools such as Tesseract will be used for this OCR technology.

[0324] 3. Data Analysis with NLP

[0325] The server then analyzes the extracted text data using natural language processing (NLP) techniques, such as using NLP libraries like spaCy or BERT. The analysis results identify key elements of the contract (such as contract duration, fees, and cancellation conditions) and categorize each element.

[0326] 4. Visual Indications

[0327] The server converts the identified important information into a visually easy-to-understand format and sends it to the device. This visualization is done using Diagramly (Draw.io) or D3.js, for example. The device displays this visual information to the user, allowing them to intuitively understand the contents of the contract.

[0328] 5. Detecting inappropriate clauses and suggesting amendments

[0329] The server analyzes the contract clauses using AI models (e.g., GPT-3 or BERT) to check for inappropriate terms or clauses that are disadvantageous to users. If any are detected, the server generates amendments or improvement suggestions for the clauses.

[0330] 6. Emotion analysis and dynamic adjustment of information presentation

[0331] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, capturing emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. The server dynamically adjusts the way information is presented based on the emotional data and presents the information in a way that is easy for the user to understand.

[0332] 7. User Feedback

[0333] The terminal presents the adjusted visual information and suggested modifications received from the server to the user, allowing the user to better understand the contents of the contract and take appropriate action.

[0334] Specific examples

[0335] Consider a scenario in which a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology (such as Tesseract) to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology (such as spaCy or BERT) to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions."

[0336] The server then visualizes these elements as a flowchart and displays it to the user. At this time, it detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required."

[0337] If the device's emotion engine detects a confused expression on the user's face, the server will increase the details of the illustrations and provide a clearer explanation of the clauses, allowing the user to review the contract and take appropriate action to avoid unfavorable terms.

[0338] Prompt Sentence Examples

[0339] "There is a clause in the residential rental agreement that allows for termination with one month's notice. Could you please suggest a general amendment to this?"

[0340] In this way, the system of the present invention provides comprehensive support functions for quickly and accurately understanding the contents of contracts and detecting and correcting inappropriate clauses. Furthermore, by dynamically changing the way information is presented according to the user's emotions, the system can provide support that is more suited to each individual user.

[0341] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0342] Step 1:

[0343] The user launches the smartphone application and either takes a photo of the contract or uploads a PDF file. Through the user's operation, the device acquires the captured image data or uploaded PDF file and saves it in local storage. The input is the contract data (image or PDF file) provided by the user, and the output is the contract data saved on the device.

[0344] Step 2:

[0345] The terminal transmits the stored contract data to the server. The contract data transmitted from the terminal to the server is the input, and the data received by the server is the output. In this step, data is transferred using a communication means. Specific operational examples include an operation in which the terminal transmits data to the server via the Internet.

[0346] Step 3:

[0347] The server performs OCR processing on the received contract data. Tesseract is used as the OCR technology, and the input data is image data or a PDF file. Through OCR processing, text data is extracted from the contract, and this is the output. Specific examples of operation include the process of recognizing characters in an image and extracting them as digital text.

[0348] Step 4:

[0349] The server analyzes the extracted text data using natural language processing (NLP) technology. For this analysis, NLP libraries such as spaCy and BERT are used. The input data is the text data extracted using OCR, and after analysis, the data is classified into key elements of the contract (contract period, fee, cancellation conditions, etc.). The output is this classified data. Specifically, the process involves extracting important elements from the text and classifying them into a database.

[0350] Step 5:

[0351] The server converts the identified important information for visual display. For example, Diagramly (Draw.io) or D3.js is used for this visualization. The input is the classified important information, and the output is the visually converted data (flowcharts and diagrams). Specifically, data is generated to present the contract period and cancellation conditions to the user as graphs and diagrams.

[0352] Step 6:

[0353] The server analyzes the clauses in the contract using an AI model (e.g., GPT-3 or BERT) to detect inappropriate clauses. The input is the analyzed text data, and the output is the detected inappropriate clauses and their corresponding correction suggestions. Specific examples of operation include a process to highlight inappropriate clauses and generate correction suggestions.

[0354] Step 7:

[0355] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, obtaining emotional data. The input is the user's facial expressions and voice tones, and the output is the analyzed emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. Specific examples of operation include the process of capturing and evaluating the user's facial expressions with a camera.

[0356] Step 8:

[0357] The server dynamically adjusts the information presentation method based on the emotion data. The input is the user's emotion data, and the output is the adjusted information presentation method. If the user is confused, the server includes a process to provide simpler illustrations or additional annotations. As a specific example of operation, the server increases the details of the illustrations according to the user's emotion, making the explanation of the clauses easier to understand.

[0358] Step 9:

[0359] The terminal presents the adjusted visual information and correction proposals received from the server to the user. The input is the adjusted information and correction proposals, and the output is the information presented to the user. Specific operations include a process of displaying the adjusted visual information and correction proposals on the terminal screen so that the user can confirm them.

[0360] Through the above processing steps, the system efficiently understands the contents of the contract, detects and corrects inappropriate clauses, and provides optimal information to the user.

[0361] (Application example 2)

[0362] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0363] It is difficult for many people to accurately understand the contents of contract documents, and there is a risk of overlooking inappropriate clauses. Therefore, there is a need for a system that can accurately understand contract documents and effectively detect terms that are unfavorable to the user. In particular, a system that can present information according to the emotions of each user is necessary to deepen the user's understanding and support them in taking appropriate action.

[0364] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, means for recognizing the user's emotions, and means for adjusting the information presentation method based on the recognized emotion data. This makes it possible to easily understand the contents of the contract document, detect inappropriate clauses, and present information according to the user's emotions.

[0365] A "contract document" is a document that describes the details of a transaction or agreement and is legally binding.

[0366] "Reading means" refers to any device or technology used to read contract documents in digital form.

[0367] "Text data" refers to data that represents character information in a digital format.

[0368] "Means of extraction" refers to techniques or methods for extracting specific information from contract documents.

[0369] "Means of analysis" refers to techniques and methods for examining the extracted text data and finding important information from it.

[0370] "Important Information" is information in a contract document that is particularly relevant and must be understood.

[0371] "Visual display means" refers to techniques and methods for visually expressing extracted important information in the form of diagrams, flow charts, etc.

[0372] "Inappropriate clauses" refer to conditions contained in contract documents that are inherently undesirable or disadvantageous to the user.

[0373] The "means for suggesting corrections" refers to techniques or methods for providing appropriate corrections or improvement suggestions for detected inappropriate clauses.

[0374] "Means for recognizing emotions" refers to technologies and methods for detecting emotions from a user's facial expressions and voice.

[0375] "Means for adjusting information presentation" refers to techniques and methods that dynamically change the way information is displayed based on recognized emotions.

[0376] The present invention is a system that presents the contents of a contract document to a user in an easy-to-understand manner, detects inappropriate clauses, and provides suggestions for correcting them.

[0377] This system uses the following hardware and software. Hardware includes devices such as smartphones, tablets, and computers. Software includes Tesseract OCR for optical character recognition (OCR), the Natural Language Toolkit (NLTK) for natural language processing (NLP), and the transformers library for generative AI models. To recognize user emotions, the system uses emotion analysis APIs such as Microsoft Azure Face API and Google Cloud Speech-to-Text.

[0378] The system analyzes the contract document and presents the information to the user by following the steps below: First, the user launches the smartphone application and takes a picture of the contract document or uploads a PDF file, which prepares the device to send the contract document data to the server.

[0379] The server receives the image data or PDF file of the contract document from the terminal and extracts the text data from the image using optical character recognition (OCR) technology (using Tesseract OCR), thereby obtaining the content of the contract document as character string data.

[0380] The server then uses natural language processing (NLP) techniques (using NLTK) to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fee, cancellation conditions, etc.). At this stage, the server efficiently extracts relevant information from the text data.

[0381] The extracted important information is converted into a visually easy-to-understand format, such as a flowchart or diagram, to help users intuitively understand the contents of the contract document.

[0382] The server then analyzes the contract clauses to check for inappropriate conditions or clauses that are disadvantageous to the user. If an inappropriate clause is detected, it generates a proposal to amend or improve the clause and provides it to the user.

[0383] The emotion engine, a feature of the present invention, recognizes emotions by analyzing facial expressions and tone of voice while the user is reviewing the contract document. For example, if the user shows discomfort or confusion regarding the contract contents, the emotion engine detects this and notifies the server.

[0384] The server then dynamically adjusts the presentation of the contract based on the emotion data, providing simpler illustrations or additional annotations to aid understanding if the user is confused, and highlighting suggested amendments to inappropriate clauses if the user expresses surprise or discomfort.

[0385] Finally, the terminal will present the visual information and suggested modifications received from the server, as well as the adjusted presentation method, to the user, allowing the user to understand the contents of the contract document and take appropriate action to avoid unfavorable terms.

[0386] As a concrete example, consider a case where a user uploads a residential rental contract to the system. The device sends this image data to the server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[0387] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the illustrated details to make the clause easier to understand.

[0388] An example of a prompt is as follows:

[0389] "Extract the duration, fees, and cancellation conditions from a residential rental agreement and visualize them as a flowchart. Also, perform a sentiment analysis of the user and provide detailed explanations for any areas that cause confusion or anxiety."

[0390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0391] Step 1:

[0392] The user launches the smartphone application and either takes a photo of the contract document or uploads a PDF file. The input is the contract document image or PDF file, and the output is that this data is saved on the device. Specifically, the user either takes a photo of the contract document using the app's camera function, or selects a PDF file using the file upload function.

[0393] Step 2:

[0394] The terminal prepares to send the contract document data to the server. The input is an image or PDF file of the contract document, and the output is the data converted into a format to be sent to the server. Specifically, the terminal encodes the image or PDF file and sends the data to the server over the network.

[0395] Step 3:

[0396] The server receives the image data or PDF file of the contract document and extracts the text data using optical character recognition (OCR) technology. The input is the contract document data, and the output is the extracted text data. Specifically, the server uses Tesseract OCR to detect characters in the image and convert them into text format.

[0397] Step 4:

[0398] The server uses natural language processing (NLP) techniques to analyze the extracted text data. The input is the extracted text data, and the output is the analyzed text data and key information extracted from it. Specifically, the server uses NLTK to tokenize and classify the text and identify key elements of the contract (e.g., contract term, fee, cancellation conditions, etc.).

[0399] Step 5:

[0400] The server converts the extracted key information into a visually understandable format. The input is the key information extracted from the parsed text data, and the output is a visually displayable format (e.g., a flowchart or diagram). Specifically, the server schematizes the data and renders it for the user interface.

[0401] Step 6:

[0402] The server analyzes the clauses in the contract document and detects inappropriate conditions or clauses that are disadvantageous to the user. The input is the full text of the contract document and extracted important information, and the output is the detected inappropriate clauses and suggested corrections. Specifically, the server analyzes the text using specific rules and generative AI models (e.g., transformers) and points out problems.

[0403] Step 7:

[0404] The device analyzes the user's facial expressions and voice using an emotion recognition API (for example, Microsoft Azure Face API or Google Cloud Speech-to-Text) to obtain emotion data. The input is the user's real-time facial and voice data, and the output is analyzed emotion data. Specifically, the device collects data using the camera and microphone, sends it to the emotion recognition API, and obtains the analysis results.

[0405] Step 8:

[0406] The server dynamically adjusts the information presentation method based on the emotional data. The input is the emotional data and extracted important information, and the output is the adjusted information presentation method. Specifically, the server evaluates the emotional data and adds explanations and annotations that match the user's emotions.

[0407] Step 9:

[0408] The device receives visual information and correction suggestions from the server and presents the presentation method adjusted based on the emotion to the user. The input is feedback data from the server, and the output is visual data displayed to the user. Specifically, the device renders the visual data on a user interface and displays it to the user.

[0409] Step 10:

[0410] The user reviews the contract document based on the information provided and takes appropriate action to avoid unfavorable terms. The input is visual information and suggested revisions, and the output is the user's actions and decisions. Specifically, the user understands the information provided and makes an appropriate decision regarding the contract.

[0411] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0412] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0413] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0414] [Second embodiment]

[0415] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0416] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0417] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0418] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0419] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0420] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0421] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0422] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0423] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0424] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0425] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0426] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0427] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[0428] User operations

[0429] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[0430] Sending and Reading Data

[0431] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[0432] Text data analysis

[0433] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[0434] Visual display of information

[0435] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[0436] Detecting inappropriate clauses and suggesting corrections

[0437] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[0438] User Feedback

[0439] The terminal presents the visual information and suggested amendments received from the server to the user, allowing the user to understand the contents of the contract and take action to avoid unfavorable terms.

[0440] Specific examples

[0441] Specific usage examples are shown below.

[0442] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[0443] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a modification: "Generally, this is reasonable, but three months' notice may be required." With this information, the user can review the contract and take appropriate action to avoid unfavorable conditions.

[0444] As described above, the present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and is a very useful tool for users.

[0445] The processing flow will be explained below.

[0446] Step 1:

[0447] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[0448] The device will activate the camera function and temporarily save the image taken by the user, and will also save the uploaded PDF file within the app.

[0449] Step 2:

[0450] The device sends an image or PDF file of the contract to the server.

[0451] The device converts the saved image or PDF file into the appropriate format and sends it to the server via the API.

[0452] Step 3:

[0453] The server extracts the text data of the contract using OCR technology.

[0454] The server inputs the image or PDF file into the OCR tool, performs character recognition, extracts the recognized text data, and saves it.

[0455] Step 4:

[0456] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[0457] The server inputs the text data into an NLP engine and extracts important elements such as contract period, rent, security deposit, and cancellation conditions.

[0458] Step 5:

[0459] The server converts the extracted important information into a visually understandable format.

[0460] The server uses tools (e.g., D3.js) to convert the extracted information into flowcharts and graphical diagrams.

[0461] Step 6:

[0462] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[0463] The server refers to predefined rules and databases to analyze the detected problems.

[0464] Step 7:

[0465] The server generates corrections and recommendations for inappropriate clauses.

[0466] The server creates a revised proposal and presents favorable terms to the user.

[0467] Step 8:

[0468] The terminal displays the visualized contract details and amendment proposals to the user.

[0469] The device displays flowcharts, diagrams, and suggested revisions on the app's UI for the user to review.

[0470] Step 9:

[0471] The user reviews the presented information and deepens their understanding.

[0472] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[0473] Example 1

[0474] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0475] Conventional contract reading systems lack the support required to easily understand the content and quickly identify inappropriate clauses. As a result, users spend a lot of time and effort trying to accurately understand the content of the contract, and especially those without legal expertise run the risk of signing unfavorable clauses without realizing it. Another issue is that the process of generating proposed amendments to contracts is time-consuming and inefficient.

[0476] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0477] In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, and means including a generative AI model for generating suggested amendments based on prompts entered by a user. This makes it possible to quickly and accurately understand the contents of the contract document, identify inappropriate clauses, and present suggested amendments.

[0478] "Contract documents" refers to documents that detail business or legal arrangements, including contracts, agreements, and memoranda of understanding.

[0479] "Means for reading" refers to devices or systems for obtaining the contents of contract documents as digital data, such as scanners and cameras.

[0480] "Extraction methods" refers to techniques or processes used to extract specific information from digital data, including optical character recognition (OCR).

[0481] "Text data" refers to data that represents the text information in a contract document in digital form.

[0482] "Means of analysis" refers to technologies and systems for analyzing text data and extracting meanings and patterns according to the purpose. Natural language processing (NLP) technology is a typical example.

[0483] "Material Information" refers to elements of the contract that are considered particularly important, such as the contract duration, fees, and cancellation conditions.

[0484] "Visual display means" refers to techniques or devices for displaying extracted information in a form that is easily understandable to the user, including, for example, flow charts or diagrams.

[0485] An "inappropriate clause" refers to a clause in a contract document that is disadvantageous to the user or that is considered unfair.

[0486] "Means for suggesting amendments" refers to a technology or system for automatically generating and providing to a user suggestions for amending inappropriate clauses.

[0487] A "generative AI model" refers to an artificial intelligence model that automatically generates document content and suggestions based on user input.

[0488] A "prompt" is a text instruction you provide to a generative AI model, which then generates a specific output.

[0489] MODE FOR CARRYING OUT THE INVENTION

[0490] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[0491] Hardware and software used

[0492] Users can start using the system by taking a photo of the contract using their smartphone or uploading a PDF file. The device (smartphone) acquires the image data or PDF file and sends it to the server. The server receives this data and extracts the contents of the contract as text data using optical character recognition (OCR) technology. Tesseract OCR can be used for OCR.

[0493] The server analyzes the extracted text data using natural language processing (NLP) techniques, such as frameworks like spaCy, to identify and classify key elements of the contract (such as contract duration, fees, and cancellation conditions).

[0494] The server then converts the extracted key information into a visually understandable format, which can be visualized using Matplotlib or other chart generators, such as flowcharts and diagrams.

[0495] Furthermore, the server analyzes the contract clauses and uses a rule-based engine or machine learning model (e.g., Scikit-learn) to detect whether they contain inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, a generative AI model (e.g., GPT-3) is used to generate suggested amendments. Based on the prompt text entered by the user, the AI ​​model provides appropriate amendments.

[0496] Specific examples

[0497] Consider a scenario where a user takes a photo of a residential rental contract and uploads it to the system. The user launches a smartphone application and either takes a photo of the contract or uploads a PDF file. The device then sends this image data or PDF file to a server. The server processes the received data with an OCR engine (Tesseract OCR) and extracts text information such as "Contract period: January 1, 2023 to December 31, 2023."

[0498] The server then uses an NLP engine (spaCy) to analyze this text data and identify key elements such as "contract term," "rent," "security deposit," "cancellation conditions," etc. The server then visualizes these elements as flowcharts and diagrams and displays them to the user.

[0499] Next, the server detects that the cancellation condition is "one month's notice" and uses a rule-based engine to determine that this is generally reasonable, but uses a generative AI model (GPT-3) to generate a revised suggestion that "three months' notice may be required."

[0500] As an example of a prompt sentence, a user can input the following into the generative AI model: "Regarding the termination conditions in a residential rental agreement, there is a clause requiring one month's notice. Please provide a general amendment to this clause.", and appropriate amendments will be generated.

[0501] The terminal presents the user with visual information and suggested amendments received from the server, and by reviewing this, the user is able to understand the contents of the contract and take specific action to avoid unfavorable terms.

[0502] As described above, this system provides comprehensive support for individuals to quickly and accurately understand contracts, and is a very useful tool for users.

[0503] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0504] Program processing flow

[0505] Step 1:

[0506] The user launches the smartphone application and takes a picture of the contract or uploads a PDF file.

[0507] Input: Image or PDF file of the contract

[0508] Specific behavior: After launching the application, a user interface will be displayed, offering the option to take a photo of the contract or select a PDF file. The user selects one of the options to complete the operation.

[0509] Step 2:

[0510] The device acquires the captured image data or uploaded PDF file and sends it to the server.

[0511] Input: Image or PDF file of the contract

[0512] Output: Image data or PDF file sent to the server

[0513] Specific behavior: The device sends the selected image data or PDF file to the server using a secure protocol (e.g., HTTPS).

[0514] Step 3:

[0515] The server extracts text data from the data it receives using an OCR (optical character recognition) engine.

[0516] Input: Image data or PDF file

[0517] Output: Extracted text data

[0518] Specific operation: The server inputs the received data into the Tesseract OCR engine, extracts the contents of the contract document as text data, and saves the extracted text as a string.

[0519] Step 4:

[0520] The server analyzes the extracted text data using natural language processing (NLP) technology.

[0521] Input: Extracted text data

[0522] Output: Key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.)

[0523] What it does: The server uses an NLP engine such as spaCy to parse the text data, identify key elements of the contract, and classify them, allowing for efficient extraction of relevant information.

[0524] Step 5:

[0525] The server converts the analyzed important information into a visually understandable format.

[0526] Input: Key elements of the contract

[0527] Output: Visualized information (e.g., flowcharts, diagrams)

[0528] Specific operation: The server uses a chart generator such as Matplotlib to generate flowcharts and diagrams to visually display the extracted key information.

[0529] Step 6:

[0530] The server analyzes the clauses in the contract document and checks for inappropriate clauses.

[0531] Input: Key elements of the contract

[0532] Output: Proposed amendments to inappropriate clauses

[0533] How it works: The server uses a rule-based engine and machine learning models to inspect the clauses in the contract document and detect inappropriate terms or clauses that are disadvantageous to the user.

[0534] Step 7:

[0535] The server uses a generative AI model to generate suggested amendments to inappropriate clauses.

[0536] Input: Incorrect clause

[0537] Output: Proposed amendments to inappropriate clauses

[0538] What it does: The server uses a generative AI model (e.g., GPT-3) to generate appropriate correction suggestions based on the prompt entered by the user.

[0539] Example: An example of a prompt might be, "My residential lease agreement has a one-month notice clause regarding termination terms. Please provide a general amendment to this."

[0540] Step 8:

[0541] The terminal presents the visual information and correction suggestions received from the server to the user.

[0542] Input: Visualized information, suggested corrections

[0543] Output: Information displayed in the user interface

[0544] Specific operation: The device displays the data received from the server, allowing the user to easily check the contents of the contract and proposed amendments.

[0545] Step 9:

[0546] The user reviews the information presented and takes the necessary action.

[0547] Input: Visualized information, suggested corrections

[0548] Output: User action (e.g., contract renegotiation, amendment adoption)

[0549] Specific Action: The user understands the contract based on the information presented and takes the necessary action to remove any unfavorable terms. In addition, this may include consulting a lawyer or renegotiating the contract.

[0550] Through these steps, the system is able to quickly and accurately understand the contents of a contract, identify inappropriate clauses, and suggest amendments.

[0551] (Application example 1)

[0552] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0553] In modern society, understanding contracts is extremely important, but many people find it difficult to understand them because they are technically complex. Furthermore, if an inappropriate clause is included, it is even more difficult to identify and correct it. This problem is particularly pronounced for users of electronic payment services, who face the challenge of finding appropriate action to avoid entering into contracts with unfavorable terms.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0555] In this invention, the server includes a means for reading the contract document, a means for extracting text data, a means for analyzing the text data using natural language processing technology and extracting important information, a means for generating amendments and improvement proposals for inappropriate clauses, and a means for visually presenting the analysis results and amendment proposals to the user in a dashboard format, thereby enabling the content of the contract to be quickly and accurately understood, inappropriate conditions to be detected, and amendment proposals to be presented.

[0556] A "contract document" is a written legal agreement that specifies the rights and obligations of the parties.

[0557] "Reading means" refers to a means for mechanically or electronically reading the contents of a contract document.

[0558] "Text data" is character string data extracted from a contract document, and is used for content analysis and display.

[0559] "Extraction means" refers to methods or techniques for extracting specific information or data from the original document or data.

[0560] "Important information" refers to the contents and key elements of a contract document that require particular attention.

[0561] The "visual display means" is a means for displaying extracted information in a form that can be directly seen and understood by the user.

[0562] "Inappropriate clauses" refer to clauses in a contract document that violate the law or contain terms that are disadvantageous to the user.

[0563] An "amendment" is a new, appropriate clause proposed to correct an inappropriate clause.

[0564] "Natural language processing technology" is a technology that enables machines to understand, interpret, and generate human language.

[0565] A "dashboard format" is a visual interface format in which multiple pieces of information are arranged so that they can be viewed at a glance.

[0566] "User" refers to a person who uses this system to review and analyze contract documents.

[0567] A "server" is a centralized computer system for processing and storing data.

[0568] The present invention relates to a system that helps users accurately understand contracts and detect and correct inappropriate clauses. The system reads contract documents, extracts text data, visually displays important information, detects inappropriate clauses, and suggests corrections.

[0569] System configuration

[0570] Hardware

[0571] Smartphone: Equipped with a camera function for taking photos of contract documents and internet connection function.

[0572] Server: A centralized computer system for processing and storing data.

[0573] software

[0574] OCR software: Use Tesseract OCR to extract text data from images.

[0575] NLP software: Uses Google Cloud Natural Language API to analyze extracted text data and identify key elements.

[0576] Front-end software: Flutter is used to provide an interface that visually displays the extracted information to the user.

[0577] Data processing and calculation

[0578] 1. Data Collection:

[0579] Users can either take a photo of the contract document using their smartphone's camera or upload an existing PDF file, and the data is captured on the smartphone and sent to the server.

[0580] 2. Text extraction:

[0581] The image data or PDF file of the contract document sent by the terminal to the server is converted into character string data using Tesseract OCR on the server.

[0582] 3. Data Analysis:

[0583] The converted text data is then analyzed using the Google Cloud Natural Language API to identify key elements of the contract document (e.g., contract term, fee, cancellation terms, etc.).

[0584] 4. Visual Indication:

[0585] The extracted important information is visually displayed in a dashboard format using Flutter, allowing users to intuitively understand the contents of the contract.

[0586] 5. Detecting inappropriate clauses and suggesting amendments:

[0587] The server uses NLP technology to analyze the clauses in the contract document and check for inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, the server generates suggestions for amendments or improvements to the clause.

[0588] 6. User Feedback:

[0589] The visual information and suggested amendments sent from the server are displayed on the user's smartphone, allowing the user to confirm the information and review the contract.

[0590] Specific examples

[0591] Example prompt sentence:

[0592] python

[0593] OCR processing

[0594] import pytesseract

[0595] from PIL import Image

[0596] def extract_text_from_image(image_path):

[0597] image = Image.open(image_path)

[0598] text = pytesseract.image_to_string(image, lang='jpn')

[0599] return text

[0600] Text analytics

[0601] from google.cloud import language_v1

[0602] def analyze_text(text):

[0603] client = language_v1.LanguageServiceClient()

[0604] document = language_v1.Document(content=text, type_=language_v1.Document.Type.PLAIN_TEXT)

[0605] response = client.analyze_entities(document=document)

[0606] return response

[0607] Extracting and visualizing key elements

[0608] import json

[0609] def extract_and_visualize(response):

[0610] elements = {}

[0611] for entity in response.entities:

[0612] elements[entity.name] = language_v1.Entity.Type(entity.type_).name

[0613] return json.dumps(elements, indent=2)

[0614] Specific example of the prompt sentence:

[0615] image_path = 'contract_image.jpg'

[0616] extracted_text = extract_text_from_image(image_path)

[0617] response = analyze_text(extracted_text)

[0618] visualization_data = extract_and_visualize(response)

[0619] print(visualization_data)

[0620] This system allows users to quickly understand the contents of contract documents, easily detect inappropriate clauses, and receive suggested amendments. This system is particularly useful for users of electronic payment services, and provides support for accurately understanding the contents of contracts.

[0621] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0622] Step 1:

[0623] Users can either take a photo of the contract document using their smartphone camera or upload an existing PDF file. The image or PDF file is then saved on the smartphone.

[0624] Step 2:

[0625] The terminal sends the image data or PDF file of the acquired contract document to the server, which then receives the contract document as input data.

[0626] Step 3:

[0627] The server converts the received image data or PDF file of the contract document into text data using OCR software. Specifically, it uses Tesseract OCR to extract text data from the image. This outputs the contents of the contract document as character string data.

[0628] Step 4:

[0629] The server then analyzes the converted text data using NLP software (Google Cloud Natural Language API). By analyzing this input text, important elements of the contract (such as contract period, fees, and cancellation conditions) are identified and classified.

[0630] Step 5:

[0631] The server uses front-end software (Flutter) to generate a visual interface in the form of a dashboard to visually display the analyzed important information, allowing the analysis results to be displayed in a user-friendly format.

[0632] Step 6:

[0633] The server also uses NLP technology to detect inappropriate clauses in the contract documents and generate amendments or improvement proposals for those clauses. Specifically, it checks each clause in the contract to see if it violates the law or contains terms that are disadvantageous to the user.

[0634] Step 7:

[0635] The visual information and suggested amendments sent from the server are displayed on the smartphone, allowing the user to review the information, examine the contract details, and take appropriate action.

[0636] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0637] The present invention relates to a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. In particular, the present invention not only visually displays the contents of a contract, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[0638] User operations

[0639] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[0640] Sending and Reading Data

[0641] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[0642] Text data analysis

[0643] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[0644] Visual display of information

[0645] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[0646] Detecting inappropriate clauses and suggesting corrections

[0647] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[0648] Emotion Engine Functions

[0649] A feature of the present invention is the incorporation of an emotion engine. The terminal recognizes emotions by analyzing the user's facial expressions and tone of voice while the user is reviewing the contract. For example, if the user shows signs of discomfort or confusion regarding the contents of the contract, the emotion engine detects this and notifies the server.

[0650] Adjusting how information is presented

[0651] The server dynamically adjusts the presentation of the contract based on the emotional data obtained from the emotion engine. For example, if the user is confused, it may provide simpler illustrations or additional annotations to aid understanding. If the user expresses surprise or displeasure, it may highlight suggested amendments to inappropriate clauses.

[0652] User Feedback

[0653] The device then presents the user with visual information and suggested revisions received from the server, as well as a presentation tailored to the user's emotions. By reviewing this information, the user can understand the contents of the contract and take action to avoid unfavorable terms.

[0654] Specific examples

[0655] Specific usage examples are shown below.

[0656] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[0657] Furthermore, the server detects that the cancellation condition is "one month's notice" and presents this along with a suggested amendment: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the details of the illustrations to make the clause easier to understand.

[0658] Based on this information, users can review their contracts and take appropriate measures to avoid unfavorable terms.

[0659] The present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and by dynamically responding to the user's emotions, it is a system that can provide information that is more suited to each individual user.

[0660] The processing flow will be explained below.

[0661] Step 1:

[0662] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[0663] The device will activate the camera function and save the image taken by the user, and will also save the uploaded PDF file within the app.

[0664] Step 2:

[0665] The terminal sends the image data or PDF file of the contract to the server.

[0666] The device converts the stored image or PDF file into the appropriate format and sends it to the server using a secure protocol.

[0667] Step 3:

[0668] The server extracts the text data of the contract using OCR technology.

[0669] The server inputs the transmitted image or PDF file into the OCR tool, performs character recognition, and saves the extracted text data.

[0670] Step 4:

[0671] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[0672] The server inputs the text data of the contract into an NLP engine and extracts important elements such as the contract period, rent, security deposit, and cancellation conditions. This extracted information is then classified and saved.

[0673] Step 5:

[0674] The server converts the extracted important information into a visually understandable format.

[0675] The server visualizes the extracted data as flowcharts and diagrams using graphical tools (e.g., D3.js).

[0676] Step 6:

[0677] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[0678] The server analyzes the content based on pre-set rules and databases to detect inappropriate clauses.

[0679] Step 7:

[0680] The server generates corrections and recommendations for inappropriate clauses.

[0681] The server creates a correction proposal for the detected inappropriate clause and presents advantageous terms to the user.

[0682] Step 8:

[0683] The device analyzes the user's facial expressions and tone of voice to recognize emotions.

[0684] The device uses a built-in camera and microphone to analyze the user's facial expressions and voice in real time, and the emotion engine uses this data to recognize the user's emotions (e.g., confusion, anger, interest).

[0685] Step 9:

[0686] The server dynamically adjusts the presentation of the contract content based on the recognized emotion data.

[0687] The server adjusts the visualization depending on the user's emotional state, such as increasing the level of detail or simplifying the presentation.

[0688] Step 10:

[0689] The terminal presents the adjusted information to the user.

[0690] Based on the adjustment information received from the server, the terminal visually displays the contract contents and suggests amendments to inappropriate clauses.

[0691] Step 11:

[0692] The user reviews the presented information and deepens their understanding.

[0693] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[0694] Example 2

[0695] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0696] Conventional contract management systems have the problem of making it difficult to accurately understand the contents of a contract, and in particular, making it easy to overlook inappropriate clauses. Furthermore, there are no dynamic systems that present information based on the user's emotions, which often leaves users confused or takes a long time to understand the contract. For this reason, there is a need for a system that allows users to quickly and accurately understand the contents of a contract and take appropriate action to avoid unfavorable terms.

[0697] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring input contract document data, a means for transmitting the acquired contract document data to the server, a means for using character recognition technology to extract text data from the transmitted contract document data, a means for analyzing the extracted text data using natural language processing technology, a means for extracting important information from the analyzed text data and visually displaying it, a means for detecting inappropriate clauses contained in the contract document and generating proposed amendments, a means for recognizing the user's emotions and dynamically adjusting the information presentation method based on the emotion data, and a means for providing the adjusted information and proposed amendments as feedback to the user. This allows the user to quickly and accurately understand the contents of the contract, easily detect inappropriate clauses, and obtain optimal proposed amendments. Furthermore, dynamically adjusting the information presentation method aids the user's understanding and allows them to more efficiently grasp the contents of the contract.

[0698] "Input contract document data" refers to the image data or electronic document file of the contract provided to the system by the user.

[0699] "Means of acquisition" refers to the function for collecting and storing contract document data photographed or uploaded by the user.

[0700] "Means for sending" refers to the function for sending contract document data from the terminal to the server.

[0701] "Character recognition technology" refers to optical character recognition (OCR) technology for extracting text data from images and PDFs.

[0702] "Natural language processing technology" refers to technology that analyzes extracted text data and identifies important elements of a contract, such as analyzing nouns and verbs and understanding the context.

[0703] "Means for analysis" refers to the ability to process text data using natural language processing and extract important information.

[0704] "Visual display means" refers to a function for displaying important analyzed information in a user-friendly format (such as a flow chart or diagram).

[0705] "Inappropriate clauses" refers to terms and conditions contained in the contract documents that are unfavorable to the user or inappropriate content.

[0706] "Proposed Amendments" refers to the proposed amendments to improve an inappropriate provision.

[0707] "Emotional Data" refers to data such as a user's facial expressions and voice tone collected using emotion recognition technology.

[0708] "Dynamic adjustment means" refers to a function that changes the way information is presented based on the user's emotional data, making it easier for the user to understand the contract.

[0709] "Means for providing feedback" refers to the ability to provide tailored information or suggested revisions to the user to assist them in understanding the contents of the contract.

[0710] The present invention is a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. This system not only visually displays the contents of contracts, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[0711] System configuration and program processing

[0712] 1. Enter contract data

[0713] The user first launches the smartphone application and then takes a photo of the contract or uploads a PDF file, which saves the contract data on the device.

[0714] 2. Data transmission and OCR processing

[0715] The device will send the captured image data or uploaded PDF file to the server, which will then use optical character recognition (OCR) technology to extract the text data. Tools such as Tesseract will be used for this OCR technology.

[0716] 3. Data Analysis with NLP

[0717] The server then analyzes the extracted text data using natural language processing (NLP) techniques, such as using NLP libraries like spaCy or BERT. The analysis results identify key elements of the contract (such as contract duration, fees, and cancellation conditions) and categorize each element.

[0718] 4. Visual Indications

[0719] The server converts the identified important information into a visually easy-to-understand format and sends it to the device. This visualization is done using Diagramly (Draw.io) or D3.js, for example. The device displays this visual information to the user, allowing them to intuitively understand the contents of the contract.

[0720] 5. Detecting inappropriate clauses and suggesting amendments

[0721] The server analyzes the contract clauses using AI models (e.g., GPT-3 or BERT) to check for inappropriate terms or clauses that are disadvantageous to users. If any are detected, the server generates amendments or improvement suggestions for the clauses.

[0722] 6. Emotion analysis and dynamic adjustment of information presentation

[0723] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, capturing emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. The server dynamically adjusts the way information is presented based on the emotional data and presents the information in a way that is easy for the user to understand.

[0724] 7. User Feedback

[0725] The terminal presents the adjusted visual information and suggested modifications received from the server to the user, allowing the user to better understand the contents of the contract and take appropriate action.

[0726] Specific examples

[0727] Consider a scenario in which a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology (such as Tesseract) to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology (such as spaCy or BERT) to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions."

[0728] The server then visualizes these elements as a flowchart and displays it to the user. At this time, it detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required."

[0729] If the device's emotion engine detects a confused expression on the user's face, the server will increase the details of the illustrations and provide a clearer explanation of the clauses, allowing the user to review the contract and take appropriate action to avoid unfavorable terms.

[0730] Prompt Sentence Examples

[0731] "There is a clause in the residential rental agreement that allows for termination with one month's notice. Could you please suggest a general amendment to this?"

[0732] In this way, the system of the present invention provides comprehensive support functions for quickly and accurately understanding the contents of contracts and detecting and correcting inappropriate clauses. Furthermore, by dynamically changing the way information is presented according to the user's emotions, the system can provide support that is more suited to each individual user.

[0733] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0734] Step 1:

[0735] The user launches the smartphone application and either takes a photo of the contract or uploads a PDF file. Through the user's operation, the device acquires the captured image data or uploaded PDF file and saves it in local storage. The input is the contract data (image or PDF file) provided by the user, and the output is the contract data saved on the device.

[0736] Step 2:

[0737] The terminal transmits the stored contract data to the server. The contract data transmitted from the terminal to the server is the input, and the data received by the server is the output. In this step, data is transferred using a communication means. Specific operational examples include an operation in which the terminal transmits data to the server via the Internet.

[0738] Step 3:

[0739] The server performs OCR processing on the received contract data. Tesseract is used as the OCR technology, and the input data is image data or a PDF file. Through OCR processing, text data is extracted from the contract, and this is the output. Specific examples of operation include the process of recognizing characters in an image and extracting them as digital text.

[0740] Step 4:

[0741] The server analyzes the extracted text data using natural language processing (NLP) technology. For this analysis, NLP libraries such as spaCy and BERT are used. The input data is the text data extracted using OCR, and after analysis, the data is classified into key elements of the contract (contract period, fee, cancellation conditions, etc.). The output is this classified data. Specifically, the process involves extracting important elements from the text and classifying them into a database.

[0742] Step 5:

[0743] The server converts the identified important information for visual display. For example, Diagramly (Draw.io) or D3.js is used for this visualization. The input is the classified important information, and the output is the visually converted data (flowcharts and diagrams). Specifically, data is generated to present the contract period and cancellation conditions to the user as graphs and diagrams.

[0744] Step 6:

[0745] The server analyzes the clauses in the contract using an AI model (e.g., GPT-3 or BERT) to detect inappropriate clauses. The input is the analyzed text data, and the output is the detected inappropriate clauses and their corresponding correction suggestions. Specific examples of operation include a process to highlight inappropriate clauses and generate correction suggestions.

[0746] Step 7:

[0747] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, obtaining emotional data. The input is the user's facial expressions and voice tones, and the output is the analyzed emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. Specific examples of operation include the process of capturing and evaluating the user's facial expressions with a camera.

[0748] Step 8:

[0749] The server dynamically adjusts the information presentation method based on the emotion data. The input is the user's emotion data, and the output is the adjusted information presentation method. If the user is confused, the server includes a process to provide simpler illustrations or additional annotations. As a specific example of operation, the server increases the details of the illustrations according to the user's emotion, making the explanation of the clauses easier to understand.

[0750] Step 9:

[0751] The terminal presents the adjusted visual information and correction proposals received from the server to the user. The input is the adjusted information and correction proposals, and the output is the information presented to the user. Specific operations include a process of displaying the adjusted visual information and correction proposals on the terminal screen so that the user can confirm them.

[0752] Through the above processing steps, the system efficiently understands the contents of the contract, detects and corrects inappropriate clauses, and provides optimal information to the user.

[0753] (Application example 2)

[0754] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0755] It is difficult for many people to accurately understand the contents of contract documents, and there is a risk of overlooking inappropriate clauses. Therefore, there is a need for a system that can accurately understand contract documents and effectively detect terms that are unfavorable to the user. In particular, a system that can present information according to the emotions of each user is necessary to deepen the user's understanding and support them in taking appropriate action.

[0756] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, means for recognizing the user's emotions, and means for adjusting the information presentation method based on the recognized emotion data. This makes it possible to easily understand the contents of the contract document, detect inappropriate clauses, and present information according to the user's emotions.

[0757] A "contract document" is a document that describes the details of a transaction or agreement and is legally binding.

[0758] "Reading means" refers to any device or technology used to read contract documents in digital form.

[0759] "Text data" refers to data that represents character information in a digital format.

[0760] "Means of extraction" refers to techniques or methods for extracting specific information from contract documents.

[0761] "Means of analysis" refers to techniques and methods for examining the extracted text data and finding important information from it.

[0762] "Important Information" is information in a contract document that is particularly relevant and must be understood.

[0763] "Visual display means" refers to techniques and methods for visually expressing extracted important information in the form of diagrams, flow charts, etc.

[0764] "Inappropriate clauses" refer to conditions contained in contract documents that are inherently undesirable or disadvantageous to the user.

[0765] The "means for suggesting corrections" refers to techniques or methods for providing appropriate corrections or improvement suggestions for detected inappropriate clauses.

[0766] "Means for recognizing emotions" refers to technologies and methods for detecting emotions from a user's facial expressions and voice.

[0767] "Means for adjusting information presentation" refers to techniques and methods that dynamically change the way information is displayed based on recognized emotions.

[0768] The present invention is a system that presents the contents of a contract document to a user in an easy-to-understand manner, detects inappropriate clauses, and provides suggestions for correcting them.

[0769] This system uses the following hardware and software. Hardware includes devices such as smartphones, tablets, and computers. Software includes Tesseract OCR for optical character recognition (OCR), the Natural Language Toolkit (NLTK) for natural language processing (NLP), and the transformers library for generative AI models. To recognize user emotions, the system uses emotion analysis APIs such as Microsoft Azure Face API and Google Cloud Speech-to-Text.

[0770] The system analyzes the contract document and presents the information to the user by following the steps below: First, the user launches the smartphone application and takes a picture of the contract document or uploads a PDF file, which prepares the device to send the contract document data to the server.

[0771] The server receives the image data or PDF file of the contract document from the terminal and extracts the text data from the image using optical character recognition (OCR) technology (using Tesseract OCR), thereby obtaining the content of the contract document as character string data.

[0772] The server then uses natural language processing (NLP) techniques (using NLTK) to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fee, cancellation conditions, etc.). At this stage, the server efficiently extracts relevant information from the text data.

[0773] The extracted important information is converted into a visually easy-to-understand format, such as a flowchart or diagram, to help users intuitively understand the contents of the contract document.

[0774] The server then analyzes the contract clauses to check for inappropriate conditions or clauses that are disadvantageous to the user. If an inappropriate clause is detected, it generates a proposal to amend or improve the clause and provides it to the user.

[0775] The emotion engine, a feature of the present invention, recognizes emotions by analyzing facial expressions and tone of voice while the user is reviewing the contract document. For example, if the user shows discomfort or confusion regarding the contract contents, the emotion engine detects this and notifies the server.

[0776] The server then dynamically adjusts the presentation of the contract based on the emotion data, providing simpler illustrations or additional annotations to aid understanding if the user is confused, and highlighting suggested amendments to inappropriate clauses if the user expresses surprise or discomfort.

[0777] Finally, the terminal will present the visual information and suggested modifications received from the server, as well as the adjusted presentation method, to the user, allowing the user to understand the contents of the contract document and take appropriate action to avoid unfavorable terms.

[0778] As a concrete example, consider a case where a user uploads a residential rental contract to the system. The device sends this image data to the server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[0779] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the illustrated details to make the clause easier to understand.

[0780] An example of a prompt is as follows:

[0781] "Extract the duration, fees, and cancellation conditions from a residential rental agreement and visualize them as a flowchart. Also, perform a sentiment analysis of the user and provide detailed explanations for any areas that cause confusion or anxiety."

[0782] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0783] Step 1:

[0784] The user launches the smartphone application and either takes a photo of the contract document or uploads a PDF file. The input is the contract document image or PDF file, and the output is that this data is saved on the device. Specifically, the user either takes a photo of the contract document using the app's camera function, or selects a PDF file using the file upload function.

[0785] Step 2:

[0786] The terminal prepares to send the contract document data to the server. The input is an image or PDF file of the contract document, and the output is the data converted into a format to be sent to the server. Specifically, the terminal encodes the image or PDF file and sends the data to the server over the network.

[0787] Step 3:

[0788] The server receives the image data or PDF file of the contract document and extracts the text data using optical character recognition (OCR) technology. The input is the contract document data, and the output is the extracted text data. Specifically, the server uses Tesseract OCR to detect characters in the image and convert them into text format.

[0789] Step 4:

[0790] The server uses natural language processing (NLP) techniques to analyze the extracted text data. The input is the extracted text data, and the output is the analyzed text data and key information extracted from it. Specifically, the server uses NLTK to tokenize and classify the text and identify key elements of the contract (e.g., contract term, fee, cancellation conditions, etc.).

[0791] Step 5:

[0792] The server converts the extracted key information into a visually understandable format. The input is the key information extracted from the parsed text data, and the output is a visually displayable format (e.g., a flowchart or diagram). Specifically, the server schematizes the data and renders it for the user interface.

[0793] Step 6:

[0794] The server analyzes the clauses in the contract document and detects inappropriate conditions or clauses that are disadvantageous to the user. The input is the full text of the contract document and extracted important information, and the output is the detected inappropriate clauses and suggested corrections. Specifically, the server analyzes the text using specific rules and generative AI models (e.g., transformers) and points out problems.

[0795] Step 7:

[0796] The device analyzes the user's facial expressions and voice using an emotion recognition API (for example, Microsoft Azure Face API or Google Cloud Speech-to-Text) to obtain emotion data. The input is the user's real-time facial and voice data, and the output is analyzed emotion data. Specifically, the device collects data using the camera and microphone, sends it to the emotion recognition API, and obtains the analysis results.

[0797] Step 8:

[0798] The server dynamically adjusts the information presentation method based on the emotional data. The input is the emotional data and extracted important information, and the output is the adjusted information presentation method. Specifically, the server evaluates the emotional data and adds explanations and annotations that match the user's emotions.

[0799] Step 9:

[0800] The device receives visual information and correction suggestions from the server and presents the presentation method adjusted based on the emotion to the user. The input is feedback data from the server, and the output is visual data displayed to the user. Specifically, the device renders the visual data on a user interface and displays it to the user.

[0801] Step 10:

[0802] The user reviews the contract document based on the information provided and takes appropriate action to avoid unfavorable terms. The input is visual information and suggested revisions, and the output is the user's actions and decisions. Specifically, the user understands the information provided and makes an appropriate decision regarding the contract.

[0803] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0804] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0805] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0806] [Third embodiment]

[0807] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0808] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0809] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0810] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0811] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0812] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0813] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0814] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0815] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0816] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0817] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0818] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0819] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[0820] User operations

[0821] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[0822] Sending and Reading Data

[0823] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[0824] Text data analysis

[0825] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[0826] Visual display of information

[0827] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[0828] Detecting inappropriate clauses and suggesting corrections

[0829] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[0830] User Feedback

[0831] The terminal presents the visual information and suggested amendments received from the server to the user, allowing the user to understand the contents of the contract and take action to avoid unfavorable terms.

[0832] Specific examples

[0833] Specific usage examples are shown below.

[0834] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[0835] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a modification: "Generally, this is reasonable, but three months' notice may be required." With this information, the user can review the contract and take appropriate action to avoid unfavorable conditions.

[0836] As described above, the present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and is a very useful tool for users.

[0837] The processing flow will be explained below.

[0838] Step 1:

[0839] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[0840] The device will activate the camera function and temporarily save the image taken by the user, and will also save the uploaded PDF file within the app.

[0841] Step 2:

[0842] The device sends an image or PDF file of the contract to the server.

[0843] The device converts the saved image or PDF file into the appropriate format and sends it to the server via the API.

[0844] Step 3:

[0845] The server extracts the text data of the contract using OCR technology.

[0846] The server inputs the image or PDF file into the OCR tool, performs character recognition, extracts the recognized text data, and saves it.

[0847] Step 4:

[0848] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[0849] The server inputs the text data into an NLP engine and extracts important elements such as contract period, rent, security deposit, and cancellation conditions.

[0850] Step 5:

[0851] The server converts the extracted important information into a visually understandable format.

[0852] The server uses tools (e.g., D3.js) to convert the extracted information into flowcharts and graphical diagrams.

[0853] Step 6:

[0854] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[0855] The server refers to predefined rules and databases to analyze the detected problems.

[0856] Step 7:

[0857] The server generates corrections and recommendations for inappropriate clauses.

[0858] The server creates a revised proposal and presents favorable terms to the user.

[0859] Step 8:

[0860] The terminal displays the visualized contract details and amendment proposals to the user.

[0861] The device displays flowcharts, diagrams, and suggested revisions on the app's UI for the user to review.

[0862] Step 9:

[0863] The user reviews the presented information and deepens their understanding.

[0864] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[0865] Example 1

[0866] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0867] Conventional contract reading systems lack the support required to easily understand the content and quickly identify inappropriate clauses. As a result, users spend a lot of time and effort trying to accurately understand the content of the contract, and especially those without legal expertise run the risk of signing unfavorable clauses without realizing it. Another issue is that the process of generating proposed amendments to contracts is time-consuming and inefficient.

[0868] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0869] In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, and means including a generative AI model for generating suggested amendments based on prompts entered by a user. This makes it possible to quickly and accurately understand the contents of the contract document, identify inappropriate clauses, and present suggested amendments.

[0870] "Contract documents" refers to documents that detail business or legal arrangements, including contracts, agreements, and memoranda of understanding.

[0871] "Means for reading" refers to devices or systems for obtaining the contents of contract documents as digital data, such as scanners and cameras.

[0872] "Extraction methods" refers to techniques or processes used to extract specific information from digital data, including optical character recognition (OCR).

[0873] "Text data" refers to data that represents the text information in a contract document in digital form.

[0874] "Means of analysis" refers to technologies and systems for analyzing text data and extracting meanings and patterns according to the purpose. Natural language processing (NLP) technology is a typical example.

[0875] "Material Information" refers to elements of the contract that are considered particularly important, such as the contract duration, fees, and cancellation conditions.

[0876] "Visual display means" refers to techniques or devices for displaying extracted information in a form that is easily understandable to the user, including, for example, flow charts or diagrams.

[0877] An "inappropriate clause" refers to a clause in a contract document that is disadvantageous to the user or that is considered unfair.

[0878] "Means for suggesting amendments" refers to a technology or system for automatically generating and providing to a user suggestions for amending inappropriate clauses.

[0879] A "generative AI model" refers to an artificial intelligence model that automatically generates document content and suggestions based on user input.

[0880] A "prompt" is a text instruction you provide to a generative AI model, which then generates a specific output.

[0881] MODE FOR CARRYING OUT THE INVENTION

[0882] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[0883] Hardware and software used

[0884] Users can start using the system by taking a photo of the contract using their smartphone or uploading a PDF file. The device (smartphone) acquires the image data or PDF file and sends it to the server. The server receives this data and extracts the contents of the contract as text data using optical character recognition (OCR) technology. Tesseract OCR can be used for OCR.

[0885] The server analyzes the extracted text data using natural language processing (NLP) techniques, such as frameworks like spaCy, to identify and classify key elements of the contract (such as contract duration, fees, and cancellation conditions).

[0886] The server then converts the extracted key information into a visually understandable format, which can be visualized using Matplotlib or other chart generators, such as flowcharts and diagrams.

[0887] Furthermore, the server analyzes the contract clauses and uses a rule-based engine or machine learning model (e.g., Scikit-learn) to detect whether they contain inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, a generative AI model (e.g., GPT-3) is used to generate suggested amendments. Based on the prompt text entered by the user, the AI ​​model provides appropriate amendments.

[0888] Specific examples

[0889] Consider a scenario where a user takes a photo of a residential rental contract and uploads it to the system. The user launches a smartphone application and either takes a photo of the contract or uploads a PDF file. The device then sends this image data or PDF file to a server. The server processes the received data with an OCR engine (Tesseract OCR) and extracts text information such as "Contract period: January 1, 2023 to December 31, 2023."

[0890] The server then uses an NLP engine (spaCy) to analyze this text data and identify key elements such as "contract term," "rent," "security deposit," "cancellation conditions," etc. The server then visualizes these elements as flowcharts and diagrams and displays them to the user.

[0891] Next, the server detects that the cancellation condition is "one month's notice" and uses a rule-based engine to determine that this is generally reasonable, but uses a generative AI model (GPT-3) to generate a revised suggestion that "three months' notice may be required."

[0892] As an example of a prompt sentence, a user can input the following into the generative AI model: "Regarding the termination conditions in a residential rental agreement, there is a clause requiring one month's notice. Please provide a general amendment to this clause.", and appropriate amendments will be generated.

[0893] The terminal presents the user with visual information and suggested amendments received from the server, and by reviewing this, the user is able to understand the contents of the contract and take specific action to avoid unfavorable terms.

[0894] As described above, this system provides comprehensive support for individuals to quickly and accurately understand contracts, and is a very useful tool for users.

[0895] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0896] Program processing flow

[0897] Step 1:

[0898] The user launches the smartphone application and takes a picture of the contract or uploads a PDF file.

[0899] Input: Image or PDF file of the contract

[0900] Specific behavior: After launching the application, a user interface will be displayed, offering the option to take a photo of the contract or select a PDF file. The user selects one of the options to complete the operation.

[0901] Step 2:

[0902] The device acquires the captured image data or uploaded PDF file and sends it to the server.

[0903] Input: Image or PDF file of the contract

[0904] Output: Image data or PDF file sent to the server

[0905] Specific behavior: The device sends the selected image data or PDF file to the server using a secure protocol (e.g., HTTPS).

[0906] Step 3:

[0907] The server extracts text data from the data it receives using an OCR (optical character recognition) engine.

[0908] Input: Image data or PDF file

[0909] Output: Extracted text data

[0910] Specific operation: The server inputs the received data into the Tesseract OCR engine, extracts the contents of the contract document as text data, and saves the extracted text as a string.

[0911] Step 4:

[0912] The server analyzes the extracted text data using natural language processing (NLP) technology.

[0913] Input: Extracted text data

[0914] Output: Key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.)

[0915] What it does: The server uses an NLP engine such as spaCy to parse the text data, identify key elements of the contract, and classify them, allowing for efficient extraction of relevant information.

[0916] Step 5:

[0917] The server converts the analyzed important information into a visually understandable format.

[0918] Input: Key elements of the contract

[0919] Output: Visualized information (e.g., flowcharts, diagrams)

[0920] Specific operation: The server uses a chart generator such as Matplotlib to generate flowcharts and diagrams to visually display the extracted key information.

[0921] Step 6:

[0922] The server analyzes the clauses in the contract document and checks for inappropriate clauses.

[0923] Input: Key elements of the contract

[0924] Output: Proposed amendments to inappropriate clauses

[0925] How it works: The server uses a rule-based engine and machine learning models to inspect the clauses in the contract document and detect inappropriate terms or clauses that are disadvantageous to the user.

[0926] Step 7:

[0927] The server uses a generative AI model to generate suggested amendments to inappropriate clauses.

[0928] Input: Incorrect clause

[0929] Output: Proposed amendments to inappropriate clauses

[0930] What it does: The server uses a generative AI model (e.g., GPT-3) to generate appropriate correction suggestions based on the prompt entered by the user.

[0931] Example: An example of a prompt might be, "My residential lease agreement has a one-month notice clause regarding termination terms. Please provide a general amendment to this."

[0932] Step 8:

[0933] The terminal presents the visual information and correction suggestions received from the server to the user.

[0934] Input: Visualized information, suggested corrections

[0935] Output: Information displayed in the user interface

[0936] Specific operation: The device displays the data received from the server, allowing the user to easily check the contents of the contract and proposed amendments.

[0937] Step 9:

[0938] The user reviews the information presented and takes the necessary action.

[0939] Input: Visualized information, suggested corrections

[0940] Output: User action (e.g., contract renegotiation, amendment adoption)

[0941] Specific Action: The user understands the contract based on the information presented and takes the necessary action to remove any unfavorable terms. In addition, this may include consulting a lawyer or renegotiating the contract.

[0942] Through these steps, the system is able to quickly and accurately understand the contents of a contract, identify inappropriate clauses, and suggest amendments.

[0943] (Application example 1)

[0944] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0945] In modern society, understanding contracts is extremely important, but many people find it difficult to understand them because they are technically complex. Furthermore, if an inappropriate clause is included, it is even more difficult to identify and correct it. This problem is particularly pronounced for users of electronic payment services, who face the challenge of finding appropriate action to avoid entering into contracts with unfavorable terms.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0947] In this invention, the server includes a means for reading the contract document, a means for extracting text data, a means for analyzing the text data using natural language processing technology and extracting important information, a means for generating amendments and improvement proposals for inappropriate clauses, and a means for visually presenting the analysis results and amendment proposals to the user in a dashboard format, thereby enabling the content of the contract to be quickly and accurately understood, inappropriate conditions to be detected, and amendment proposals to be presented.

[0948] A "contract document" is a written legal agreement that specifies the rights and obligations of the parties.

[0949] "Reading means" refers to a means for mechanically or electronically reading the contents of a contract document.

[0950] "Text data" is character string data extracted from a contract document, and is used for content analysis and display.

[0951] "Extraction means" refers to methods or techniques for extracting specific information or data from the original document or data.

[0952] "Important information" refers to the contents and key elements of a contract document that require particular attention.

[0953] The "visual display means" is a means for displaying extracted information in a form that can be directly seen and understood by the user.

[0954] "Inappropriate clauses" refer to clauses in a contract document that violate the law or contain terms that are disadvantageous to the user.

[0955] An "amendment" is a new, appropriate clause proposed to correct an inappropriate clause.

[0956] "Natural language processing technology" is a technology that enables machines to understand, interpret, and generate human language.

[0957] A "dashboard format" is a visual interface format in which multiple pieces of information are arranged so that they can be viewed at a glance.

[0958] "User" refers to a person who uses this system to review and analyze contract documents.

[0959] A "server" is a centralized computer system for processing and storing data.

[0960] The present invention relates to a system that helps users accurately understand contracts and detect and correct inappropriate clauses. The system reads contract documents, extracts text data, visually displays important information, detects inappropriate clauses, and suggests corrections.

[0961] System configuration

[0962] Hardware

[0963] Smartphone: Equipped with a camera function for taking photos of contract documents and internet connection function.

[0964] Server: A centralized computer system for processing and storing data.

[0965] software

[0966] OCR software: Use Tesseract OCR to extract text data from images.

[0967] NLP software: Uses Google Cloud Natural Language API to analyze extracted text data and identify key elements.

[0968] Front-end software: Flutter is used to provide an interface that visually displays the extracted information to the user.

[0969] Data processing and calculation

[0970] 1. Data Collection:

[0971] Users can either take a photo of the contract document using their smartphone's camera or upload an existing PDF file, and the data is captured on the smartphone and sent to the server.

[0972] 2. Text extraction:

[0973] The image data or PDF file of the contract document sent by the terminal to the server is converted into character string data using Tesseract OCR on the server.

[0974] 3. Data Analysis:

[0975] The converted text data is then analyzed using the Google Cloud Natural Language API to identify key elements of the contract document (e.g., contract term, fee, cancellation terms, etc.).

[0976] 4. Visual Indication:

[0977] The extracted important information is visually displayed in a dashboard format using Flutter, allowing users to intuitively understand the contents of the contract.

[0978] 5. Detecting inappropriate clauses and suggesting amendments:

[0979] The server uses NLP technology to analyze the clauses in the contract document and check for inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, the server generates suggestions for amendments or improvements to the clause.

[0980] 6. User Feedback:

[0981] The visual information and suggested amendments sent from the server are displayed on the user's smartphone, allowing the user to confirm the information and review the contract.

[0982] Specific examples

[0983] Example prompt sentence:

[0984] python

[0985] OCR processing

[0986] import pytesseract

[0987] from PIL import Image

[0988] def extract_text_from_image(image_path):

[0989] image = Image.open(image_path)

[0990] text = pytesseract.image_to_string(image, lang='jpn')

[0991] return text

[0992] Text analytics

[0993] from google.cloud import language_v1

[0994] def analyze_text(text):

[0995] client = language_v1.LanguageServiceClient()

[0996] document = language_v1.Document(content=text, type_=language_v1.Document.Type.PLAIN_TEXT)

[0997] response = client.analyze_entities(document=document)

[0998] return response

[0999] Extracting and visualizing key elements

[1000] import json

[1001] def extract_and_visualize(response):

[1002] elements = {}

[1003] for entity in response.entities:

[1004] elements[entity.name] = language_v1.Entity.Type(entity.type_).name

[1005] return json.dumps(elements, indent=2)

[1006] Specific example of the prompt sentence:

[1007] image_path = 'contract_image.jpg'

[1008] extracted_text = extract_text_from_image(image_path)

[1009] response = analyze_text(extracted_text)

[1010] visualization_data = extract_and_visualize(response)

[1011] print(visualization_data)

[1012] This system allows users to quickly understand the contents of contract documents, easily detect inappropriate clauses, and receive suggested amendments. This system is particularly useful for users of electronic payment services, and provides support for accurately understanding the contents of contracts.

[1013] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1014] Step 1:

[1015] Users can either take a photo of the contract document using their smartphone camera or upload an existing PDF file. The image or PDF file is then saved on the smartphone.

[1016] Step 2:

[1017] The terminal sends the image data or PDF file of the acquired contract document to the server, which then receives the contract document as input data.

[1018] Step 3:

[1019] The server converts the received image data or PDF file of the contract document into text data using OCR software. Specifically, it uses Tesseract OCR to extract text data from the image. This outputs the contents of the contract document as character string data.

[1020] Step 4:

[1021] The server then analyzes the converted text data using NLP software (Google Cloud Natural Language API). By analyzing this input text, important elements of the contract (such as contract period, fees, and cancellation conditions) are identified and classified.

[1022] Step 5:

[1023] The server uses front-end software (Flutter) to generate a visual interface in the form of a dashboard to visually display the analyzed important information, allowing the analysis results to be displayed in a user-friendly format.

[1024] Step 6:

[1025] The server also uses NLP technology to detect inappropriate clauses in the contract documents and generate amendments or improvement proposals for those clauses. Specifically, it checks each clause in the contract to see if it violates the law or contains terms that are disadvantageous to the user.

[1026] Step 7:

[1027] The visual information and suggested amendments sent from the server are displayed on the smartphone, allowing the user to review the information, examine the contract details, and take appropriate action.

[1028] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1029] The present invention relates to a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. In particular, the present invention not only visually displays the contents of a contract, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[1030] User operations

[1031] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[1032] Sending and Reading Data

[1033] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[1034] Text data analysis

[1035] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[1036] Visual display of information

[1037] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[1038] Detecting inappropriate clauses and suggesting corrections

[1039] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[1040] Emotion Engine Functions

[1041] A feature of the present invention is the incorporation of an emotion engine. The terminal recognizes emotions by analyzing the user's facial expressions and tone of voice while the user is reviewing the contract. For example, if the user shows signs of discomfort or confusion regarding the contents of the contract, the emotion engine detects this and notifies the server.

[1042] Adjusting how information is presented

[1043] The server dynamically adjusts the presentation of the contract based on the emotional data obtained from the emotion engine. For example, if the user is confused, it may provide simpler illustrations or additional annotations to aid understanding. If the user expresses surprise or displeasure, it may highlight suggested amendments to inappropriate clauses.

[1044] User Feedback

[1045] The device then presents the user with visual information and suggested revisions received from the server, as well as a presentation tailored to the user's emotions. By reviewing this information, the user can understand the contents of the contract and take action to avoid unfavorable terms.

[1046] Specific examples

[1047] Specific usage examples are shown below.

[1048] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[1049] Furthermore, the server detects that the cancellation condition is "one month's notice" and presents this along with a suggested amendment: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the details of the illustrations to make the clause easier to understand.

[1050] Based on this information, users can review their contracts and take appropriate measures to avoid unfavorable terms.

[1051] The present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and by dynamically responding to the user's emotions, it is a system that can provide information that is more suited to each individual user.

[1052] The processing flow will be explained below.

[1053] Step 1:

[1054] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[1055] The device will activate the camera function and save the image taken by the user, and will also save the uploaded PDF file within the app.

[1056] Step 2:

[1057] The terminal sends the image data or PDF file of the contract to the server.

[1058] The device converts the stored image or PDF file into the appropriate format and sends it to the server using a secure protocol.

[1059] Step 3:

[1060] The server extracts the text data of the contract using OCR technology.

[1061] The server inputs the transmitted image or PDF file into the OCR tool, performs character recognition, and saves the extracted text data.

[1062] Step 4:

[1063] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[1064] The server inputs the text data of the contract into an NLP engine and extracts important elements such as the contract period, rent, security deposit, and cancellation conditions. This extracted information is then classified and saved.

[1065] Step 5:

[1066] The server converts the extracted important information into a visually understandable format.

[1067] The server visualizes the extracted data as flowcharts and diagrams using graphical tools (e.g., D3.js).

[1068] Step 6:

[1069] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[1070] The server analyzes the content based on pre-set rules and databases to detect inappropriate clauses.

[1071] Step 7:

[1072] The server generates corrections and recommendations for inappropriate clauses.

[1073] The server creates a correction proposal for the detected inappropriate clause and presents advantageous terms to the user.

[1074] Step 8:

[1075] The device analyzes the user's facial expressions and tone of voice to recognize emotions.

[1076] The device uses a built-in camera and microphone to analyze the user's facial expressions and voice in real time, and the emotion engine uses this data to recognize the user's emotions (e.g., confusion, anger, interest).

[1077] Step 9:

[1078] The server dynamically adjusts the presentation of the contract content based on the recognized emotion data.

[1079] The server adjusts the visualization depending on the user's emotional state, such as increasing the level of detail or simplifying the presentation.

[1080] Step 10:

[1081] The terminal presents the adjusted information to the user.

[1082] Based on the adjustment information received from the server, the terminal visually displays the contract contents and suggests amendments to inappropriate clauses.

[1083] Step 11:

[1084] The user reviews the presented information and deepens their understanding.

[1085] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[1086] Example 2

[1087] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1088] Conventional contract management systems have the problem of making it difficult to accurately understand the contents of a contract, and in particular, making it easy to overlook inappropriate clauses. Furthermore, there are no dynamic systems that present information based on the user's emotions, which often leaves users confused or takes a long time to understand the contract. For this reason, there is a need for a system that allows users to quickly and accurately understand the contents of a contract and take appropriate action to avoid unfavorable terms.

[1089] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring input contract document data, a means for transmitting the acquired contract document data to the server, a means for using character recognition technology to extract text data from the transmitted contract document data, a means for analyzing the extracted text data using natural language processing technology, a means for extracting important information from the analyzed text data and visually displaying it, a means for detecting inappropriate clauses contained in the contract document and generating proposed amendments, a means for recognizing the user's emotions and dynamically adjusting the information presentation method based on the emotion data, and a means for providing the adjusted information and proposed amendments as feedback to the user. This allows the user to quickly and accurately understand the contents of the contract, easily detect inappropriate clauses, and obtain optimal proposed amendments. Furthermore, dynamically adjusting the information presentation method aids the user's understanding and allows them to more efficiently grasp the contents of the contract.

[1090] "Input contract document data" refers to the image data or electronic document file of the contract provided to the system by the user.

[1091] "Means of acquisition" refers to the function for collecting and storing contract document data photographed or uploaded by the user.

[1092] "Means for sending" refers to the function for sending contract document data from the terminal to the server.

[1093] "Character recognition technology" refers to optical character recognition (OCR) technology for extracting text data from images and PDFs.

[1094] "Natural language processing technology" refers to technology that analyzes extracted text data and identifies important elements of a contract, such as analyzing nouns and verbs and understanding the context.

[1095] "Means for analysis" refers to the ability to process text data using natural language processing and extract important information.

[1096] "Visual display means" refers to a function for displaying important analyzed information in a user-friendly format (such as a flow chart or diagram).

[1097] "Inappropriate clauses" refers to terms and conditions contained in the contract documents that are unfavorable to the user or inappropriate content.

[1098] "Proposed Amendments" refers to the proposed amendments to improve an inappropriate provision.

[1099] "Emotional Data" refers to data such as a user's facial expressions and voice tone collected using emotion recognition technology.

[1100] "Dynamic adjustment means" refers to a function that changes the way information is presented based on the user's emotional data, making it easier for the user to understand the contract.

[1101] "Means for providing feedback" refers to the ability to provide tailored information or suggested revisions to the user to assist them in understanding the contents of the contract.

[1102] The present invention is a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. This system not only visually displays the contents of contracts, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[1103] System configuration and program processing

[1104] 1. Enter contract data

[1105] The user first launches the smartphone application and then takes a photo of the contract or uploads a PDF file, which saves the contract data on the device.

[1106] 2. Data transmission and OCR processing

[1107] The device will send the captured image data or uploaded PDF file to the server, which will then use optical character recognition (OCR) technology to extract the text data. Tools such as Tesseract will be used for this OCR technology.

[1108] 3. Data Analysis with NLP

[1109] The server then analyzes the extracted text data using natural language processing (NLP) techniques, such as using NLP libraries like spaCy or BERT. The analysis results identify key elements of the contract (such as contract duration, fees, and cancellation conditions) and categorize each element.

[1110] 4. Visual Indications

[1111] The server converts the identified important information into a visually easy-to-understand format and sends it to the device. This visualization is done using Diagramly (Draw.io) or D3.js, for example. The device displays this visual information to the user, allowing them to intuitively understand the contents of the contract.

[1112] 5. Detecting inappropriate clauses and suggesting amendments

[1113] The server analyzes the contract clauses using AI models (e.g., GPT-3 or BERT) to check for inappropriate terms or clauses that are disadvantageous to users. If any are detected, the server generates amendments or improvement suggestions for the clauses.

[1114] 6. Emotion analysis and dynamic adjustment of information presentation

[1115] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, capturing emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. The server dynamically adjusts the way information is presented based on the emotional data and presents the information in a way that is easy for the user to understand.

[1116] 7. User Feedback

[1117] The terminal presents the adjusted visual information and suggested modifications received from the server to the user, allowing the user to better understand the contents of the contract and take appropriate action.

[1118] Specific examples

[1119] Consider a scenario in which a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology (such as Tesseract) to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology (such as spaCy or BERT) to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions."

[1120] The server then visualizes these elements as a flowchart and displays it to the user. At this time, it detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required."

[1121] If the device's emotion engine detects a confused expression on the user's face, the server will increase the details of the illustrations and provide a clearer explanation of the clauses, allowing the user to review the contract and take appropriate action to avoid unfavorable terms.

[1122] Prompt Sentence Examples

[1123] "There is a clause in the residential rental agreement that allows for termination with one month's notice. Could you please suggest a general amendment to this?"

[1124] In this way, the system of the present invention provides comprehensive support functions for quickly and accurately understanding the contents of contracts and detecting and correcting inappropriate clauses. Furthermore, by dynamically changing the way information is presented according to the user's emotions, the system can provide support that is more suited to each individual user.

[1125] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1126] Step 1:

[1127] The user launches the smartphone application and either takes a photo of the contract or uploads a PDF file. Through the user's operation, the device acquires the captured image data or uploaded PDF file and saves it in local storage. The input is the contract data (image or PDF file) provided by the user, and the output is the contract data saved on the device.

[1128] Step 2:

[1129] The terminal transmits the stored contract data to the server. The contract data transmitted from the terminal to the server is the input, and the data received by the server is the output. In this step, data is transferred using a communication means. Specific operational examples include an operation in which the terminal transmits data to the server via the Internet.

[1130] Step 3:

[1131] The server performs OCR processing on the received contract data. Tesseract is used as the OCR technology, and the input data is image data or a PDF file. Through OCR processing, text data is extracted from the contract, and this is the output. Specific examples of operation include the process of recognizing characters in an image and extracting them as digital text.

[1132] Step 4:

[1133] The server analyzes the extracted text data using natural language processing (NLP) technology. For this analysis, NLP libraries such as spaCy and BERT are used. The input data is the text data extracted using OCR, and after analysis, the data is classified into key elements of the contract (contract period, fee, cancellation conditions, etc.). The output is this classified data. Specifically, the process involves extracting important elements from the text and classifying them into a database.

[1134] Step 5:

[1135] The server converts the identified important information for visual display. For example, Diagramly (Draw.io) or D3.js is used for this visualization. The input is the classified important information, and the output is the visually converted data (flowcharts and diagrams). Specifically, data is generated to present the contract period and cancellation conditions to the user as graphs and diagrams.

[1136] Step 6:

[1137] The server analyzes the clauses in the contract using an AI model (e.g., GPT-3 or BERT) to detect inappropriate clauses. The input is the analyzed text data, and the output is the detected inappropriate clauses and their corresponding correction suggestions. Specific examples of operation include a process to highlight inappropriate clauses and generate correction suggestions.

[1138] Step 7:

[1139] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, obtaining emotional data. The input is the user's facial expressions and voice tones, and the output is the analyzed emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. Specific examples of operation include the process of capturing and evaluating the user's facial expressions with a camera.

[1140] Step 8:

[1141] The server dynamically adjusts the information presentation method based on the emotion data. The input is the user's emotion data, and the output is the adjusted information presentation method. If the user is confused, the server includes a process to provide simpler illustrations or additional annotations. As a specific example of operation, the server increases the details of the illustrations according to the user's emotion, making the explanation of the clauses easier to understand.

[1142] Step 9:

[1143] The terminal presents the adjusted visual information and correction proposals received from the server to the user. The input is the adjusted information and correction proposals, and the output is the information presented to the user. Specific operations include a process of displaying the adjusted visual information and correction proposals on the terminal screen so that the user can confirm them.

[1144] Through the above processing steps, the system efficiently understands the contents of the contract, detects and corrects inappropriate clauses, and provides optimal information to the user.

[1145] (Application example 2)

[1146] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1147] It is difficult for many people to accurately understand the contents of contract documents, and there is a risk of overlooking inappropriate clauses. Therefore, there is a need for a system that can accurately understand contract documents and effectively detect terms that are unfavorable to the user. In particular, a system that can present information according to the emotions of each user is necessary to deepen the user's understanding and support them in taking appropriate action.

[1148] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, means for recognizing the user's emotions, and means for adjusting the information presentation method based on the recognized emotion data. This makes it possible to easily understand the contents of the contract document, detect inappropriate clauses, and present information according to the user's emotions.

[1149] A "contract document" is a document that describes the details of a transaction or agreement and is legally binding.

[1150] "Reading means" refers to any device or technology used to read contract documents in digital form.

[1151] "Text data" refers to data that represents character information in a digital format.

[1152] "Means of extraction" refers to techniques or methods for extracting specific information from contract documents.

[1153] "Means of analysis" refers to techniques and methods for examining the extracted text data and finding important information from it.

[1154] "Important Information" is information in a contract document that is particularly relevant and must be understood.

[1155] "Visual display means" refers to techniques and methods for visually expressing extracted important information in the form of diagrams, flow charts, etc.

[1156] "Inappropriate clauses" refer to conditions contained in contract documents that are inherently undesirable or disadvantageous to the user.

[1157] The "means for suggesting corrections" refers to techniques or methods for providing appropriate corrections or improvement suggestions for detected inappropriate clauses.

[1158] "Means for recognizing emotions" refers to technologies and methods for detecting emotions from a user's facial expressions and voice.

[1159] "Means for adjusting information presentation" refers to techniques and methods that dynamically change the way information is displayed based on recognized emotions.

[1160] The present invention is a system that presents the contents of a contract document to a user in an easy-to-understand manner, detects inappropriate clauses, and provides suggestions for correcting them.

[1161] This system uses the following hardware and software. Hardware includes devices such as smartphones, tablets, and computers. Software includes Tesseract OCR for optical character recognition (OCR), the Natural Language Toolkit (NLTK) for natural language processing (NLP), and the transformers library for generative AI models. To recognize user emotions, the system uses emotion analysis APIs such as Microsoft Azure Face API and Google Cloud Speech-to-Text.

[1162] The system analyzes the contract document and presents the information to the user by following the steps below: First, the user launches the smartphone application and takes a picture of the contract document or uploads a PDF file, which prepares the device to send the contract document data to the server.

[1163] The server receives the image data or PDF file of the contract document from the terminal and extracts the text data from the image using optical character recognition (OCR) technology (using Tesseract OCR), thereby obtaining the content of the contract document as character string data.

[1164] The server then uses natural language processing (NLP) techniques (using NLTK) to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fee, cancellation conditions, etc.). At this stage, the server efficiently extracts relevant information from the text data.

[1165] The extracted important information is converted into a visually easy-to-understand format, such as a flowchart or diagram, to help users intuitively understand the contents of the contract document.

[1166] The server then analyzes the contract clauses to check for inappropriate conditions or clauses that are disadvantageous to the user. If an inappropriate clause is detected, it generates a proposal to amend or improve the clause and provides it to the user.

[1167] The emotion engine, a feature of the present invention, recognizes emotions by analyzing facial expressions and tone of voice while the user is reviewing the contract document. For example, if the user shows discomfort or confusion regarding the contract contents, the emotion engine detects this and notifies the server.

[1168] The server then dynamically adjusts the presentation of the contract based on the emotion data, providing simpler illustrations or additional annotations to aid understanding if the user is confused, and highlighting suggested amendments to inappropriate clauses if the user expresses surprise or discomfort.

[1169] Finally, the terminal will present the visual information and suggested modifications received from the server, as well as the adjusted presentation method, to the user, allowing the user to understand the contents of the contract document and take appropriate action to avoid unfavorable terms.

[1170] As a concrete example, consider a case where a user uploads a residential rental contract to the system. The device sends this image data to the server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[1171] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the illustrated details to make the clause easier to understand.

[1172] An example of a prompt is as follows:

[1173] "Extract the duration, fees, and cancellation conditions from a residential rental agreement and visualize them as a flowchart. Also, perform a sentiment analysis of the user and provide detailed explanations for any areas that cause confusion or anxiety."

[1174] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1175] Step 1:

[1176] The user launches the smartphone application and either takes a photo of the contract document or uploads a PDF file. The input is the contract document image or PDF file, and the output is that this data is saved on the device. Specifically, the user either takes a photo of the contract document using the app's camera function, or selects a PDF file using the file upload function.

[1177] Step 2:

[1178] The terminal prepares to send the contract document data to the server. The input is an image or PDF file of the contract document, and the output is the data converted into a format to be sent to the server. Specifically, the terminal encodes the image or PDF file and sends the data to the server over the network.

[1179] Step 3:

[1180] The server receives the image data or PDF file of the contract document and extracts the text data using optical character recognition (OCR) technology. The input is the contract document data, and the output is the extracted text data. Specifically, the server uses Tesseract OCR to detect characters in the image and convert them into text format.

[1181] Step 4:

[1182] The server uses natural language processing (NLP) techniques to analyze the extracted text data. The input is the extracted text data, and the output is the analyzed text data and key information extracted from it. Specifically, the server uses NLTK to tokenize and classify the text and identify key elements of the contract (e.g., contract term, fee, cancellation conditions, etc.).

[1183] Step 5:

[1184] The server converts the extracted key information into a visually understandable format. The input is the key information extracted from the parsed text data, and the output is a visually displayable format (e.g., a flowchart or diagram). Specifically, the server schematizes the data and renders it for the user interface.

[1185] Step 6:

[1186] The server analyzes the clauses in the contract document and detects inappropriate conditions or clauses that are disadvantageous to the user. The input is the full text of the contract document and extracted important information, and the output is the detected inappropriate clauses and suggested corrections. Specifically, the server analyzes the text using specific rules and generative AI models (e.g., transformers) and points out problems.

[1187] Step 7:

[1188] The device analyzes the user's facial expressions and voice using an emotion recognition API (for example, Microsoft Azure Face API or Google Cloud Speech-to-Text) to obtain emotion data. The input is the user's real-time facial and voice data, and the output is analyzed emotion data. Specifically, the device collects data using the camera and microphone, sends it to the emotion recognition API, and obtains the analysis results.

[1189] Step 8:

[1190] The server dynamically adjusts the information presentation method based on the emotional data. The input is the emotional data and extracted important information, and the output is the adjusted information presentation method. Specifically, the server evaluates the emotional data and adds explanations and annotations that match the user's emotions.

[1191] Step 9:

[1192] The device receives visual information and correction suggestions from the server and presents the presentation method adjusted based on the emotion to the user. The input is feedback data from the server, and the output is visual data displayed to the user. Specifically, the device renders the visual data on a user interface and displays it to the user.

[1193] Step 10:

[1194] The user reviews the contract document based on the information provided and takes appropriate action to avoid unfavorable terms. The input is visual information and suggested revisions, and the output is the user's actions and decisions. Specifically, the user understands the information provided and makes an appropriate decision regarding the contract.

[1195] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1196] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1197] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1198] [Fourth embodiment]

[1199] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1200] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1201] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1202] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1203] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1204] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1205] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1206] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1207] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1208] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1209] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1210] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1211] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1212] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[1213] User operations

[1214] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[1215] Sending and Reading Data

[1216] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[1217] Text data analysis

[1218] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[1219] Visual display of information

[1220] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[1221] Detecting inappropriate clauses and suggesting corrections

[1222] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[1223] User Feedback

[1224] The terminal presents the visual information and suggested amendments received from the server to the user, allowing the user to understand the contents of the contract and take action to avoid unfavorable terms.

[1225] Specific examples

[1226] Specific usage examples are shown below.

[1227] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[1228] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a modification: "Generally, this is reasonable, but three months' notice may be required." With this information, the user can review the contract and take appropriate action to avoid unfavorable conditions.

[1229] As described above, the present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and is a very useful tool for users.

[1230] The processing flow will be explained below.

[1231] Step 1:

[1232] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[1233] The device will activate the camera function and temporarily save the image taken by the user, and will also save the uploaded PDF file within the app.

[1234] Step 2:

[1235] The device sends an image or PDF file of the contract to the server.

[1236] The device converts the saved image or PDF file into the appropriate format and sends it to the server via the API.

[1237] Step 3:

[1238] The server extracts the text data of the contract using OCR technology.

[1239] The server inputs the image or PDF file into the OCR tool, performs character recognition, extracts the recognized text data, and saves it.

[1240] Step 4:

[1241] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[1242] The server inputs the text data into an NLP engine and extracts important elements such as contract period, rent, security deposit, and cancellation conditions.

[1243] Step 5:

[1244] The server converts the extracted important information into a visually understandable format.

[1245] The server uses tools (e.g., D3.js) to convert the extracted information into flowcharts and graphical diagrams.

[1246] Step 6:

[1247] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[1248] The server refers to predefined rules and databases to analyze the detected problems.

[1249] Step 7:

[1250] The server generates corrections and recommendations for inappropriate clauses.

[1251] The server creates a revised proposal and presents favorable terms to the user.

[1252] Step 8:

[1253] The terminal displays the visualized contract details and amendment proposals to the user.

[1254] The device displays flowcharts, diagrams, and suggested revisions on the app's UI for the user to review.

[1255] Step 9:

[1256] The user reviews the presented information and deepens their understanding.

[1257] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[1258] Example 1

[1259] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1260] Conventional contract reading systems lack the support required to easily understand the content and quickly identify inappropriate clauses. As a result, users spend a lot of time and effort trying to accurately understand the content of the contract, and especially those without legal expertise run the risk of signing unfavorable clauses without realizing it. Another issue is that the process of generating proposed amendments to contracts is time-consuming and inefficient.

[1261] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1262] In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, and means including a generative AI model for generating suggested amendments based on prompts entered by a user. This makes it possible to quickly and accurately understand the contents of the contract document, identify inappropriate clauses, and present suggested amendments.

[1263] "Contract documents" refers to documents that detail business or legal arrangements, including contracts, agreements, and memoranda of understanding.

[1264] "Means for reading" refers to devices or systems for obtaining the contents of contract documents as digital data, such as scanners and cameras.

[1265] "Extraction methods" refers to techniques or processes used to extract specific information from digital data, including optical character recognition (OCR).

[1266] "Text data" refers to data that represents the text information in a contract document in digital form.

[1267] "Means of analysis" refers to technologies and systems for analyzing text data and extracting meanings and patterns according to the purpose. Natural language processing (NLP) technology is a typical example.

[1268] "Material Information" refers to elements of the contract that are considered particularly important, such as the contract duration, fees, and cancellation conditions.

[1269] "Visual display means" refers to techniques or devices for displaying extracted information in a form that is easily understandable to the user, including, for example, flow charts or diagrams.

[1270] An "inappropriate clause" refers to a clause in a contract document that is disadvantageous to the user or that is considered unfair.

[1271] "Means for suggesting amendments" refers to a technology or system for automatically generating and providing to a user suggestions for amending inappropriate clauses.

[1272] A "generative AI model" refers to an artificial intelligence model that automatically generates document content and suggestions based on user input.

[1273] A "prompt" is a text instruction you provide to a generative AI model, which then generates a specific output.

[1274] MODE FOR CARRYING OUT THE INVENTION

[1275] The present invention relates to a system that assists individuals in accurately understanding contracts. The system includes functions for reading contract documents, extracting and analyzing text data, visually displaying important information, and detecting inappropriate clauses and suggesting corrections.

[1276] Hardware and software used

[1277] Users can start using the system by taking a photo of the contract using their smartphone or uploading a PDF file. The device (smartphone) acquires the image data or PDF file and sends it to the server. The server receives this data and extracts the contents of the contract as text data using optical character recognition (OCR) technology. Tesseract OCR can be used for OCR.

[1278] The server analyzes the extracted text data using natural language processing (NLP) techniques, such as frameworks like spaCy, to identify and classify key elements of the contract (such as contract duration, fees, and cancellation conditions).

[1279] The server then converts the extracted key information into a visually understandable format, which can be visualized using Matplotlib or other chart generators, such as flowcharts and diagrams.

[1280] Furthermore, the server analyzes the contract clauses and uses a rule-based engine or machine learning model (e.g., Scikit-learn) to detect whether they contain inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, a generative AI model (e.g., GPT-3) is used to generate suggested amendments. Based on the prompt text entered by the user, the AI ​​model provides appropriate amendments.

[1281] Specific examples

[1282] Consider a scenario where a user takes a photo of a residential rental contract and uploads it to the system. The user launches a smartphone application and either takes a photo of the contract or uploads a PDF file. The device then sends this image data or PDF file to a server. The server processes the received data with an OCR engine (Tesseract OCR) and extracts text information such as "Contract period: January 1, 2023 to December 31, 2023."

[1283] The server then uses an NLP engine (spaCy) to analyze this text data and identify key elements such as "contract term," "rent," "security deposit," "cancellation conditions," etc. The server then visualizes these elements as flowcharts and diagrams and displays them to the user.

[1284] Next, the server detects that the cancellation condition is "one month's notice" and uses a rule-based engine to determine that this is generally reasonable, but uses a generative AI model (GPT-3) to generate a revised suggestion that "three months' notice may be required."

[1285] As an example of a prompt sentence, a user can input the following into the generative AI model: "Regarding the termination conditions in a residential rental agreement, there is a clause requiring one month's notice. Please provide a general amendment to this clause.", and appropriate amendments will be generated.

[1286] The terminal presents the user with visual information and suggested amendments received from the server, and by reviewing this, the user is able to understand the contents of the contract and take specific action to avoid unfavorable terms.

[1287] As described above, this system provides comprehensive support for individuals to quickly and accurately understand contracts, and is a very useful tool for users.

[1288] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1289] Program processing flow

[1290] Step 1:

[1291] The user launches the smartphone application and takes a picture of the contract or uploads a PDF file.

[1292] Input: Image or PDF file of the contract

[1293] Specific behavior: After launching the application, a user interface will be displayed, offering the option to take a photo of the contract or select a PDF file. The user selects one of the options to complete the operation.

[1294] Step 2:

[1295] The device acquires the captured image data or uploaded PDF file and sends it to the server.

[1296] Input: Image or PDF file of the contract

[1297] Output: Image data or PDF file sent to the server

[1298] Specific behavior: The device sends the selected image data or PDF file to the server using a secure protocol (e.g., HTTPS).

[1299] Step 3:

[1300] The server extracts text data from the data it receives using an OCR (optical character recognition) engine.

[1301] Input: Image data or PDF file

[1302] Output: Extracted text data

[1303] Specific operation: The server inputs the received data into the Tesseract OCR engine, extracts the contents of the contract document as text data, and saves the extracted text as a string.

[1304] Step 4:

[1305] The server analyzes the extracted text data using natural language processing (NLP) technology.

[1306] Input: Extracted text data

[1307] Output: Key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.)

[1308] What it does: The server uses an NLP engine such as spaCy to parse the text data, identify key elements of the contract, and classify them, allowing for efficient extraction of relevant information.

[1309] Step 5:

[1310] The server converts the analyzed important information into a visually understandable format.

[1311] Input: Key elements of the contract

[1312] Output: Visualized information (e.g., flowcharts, diagrams)

[1313] Specific operation: The server uses a chart generator such as Matplotlib to generate flowcharts and diagrams to visually display the extracted key information.

[1314] Step 6:

[1315] The server analyzes the clauses in the contract document and checks for inappropriate clauses.

[1316] Input: Key elements of the contract

[1317] Output: Proposed amendments to inappropriate clauses

[1318] How it works: The server uses a rule-based engine and machine learning models to inspect the clauses in the contract document and detect inappropriate terms or clauses that are disadvantageous to the user.

[1319] Step 7:

[1320] The server uses a generative AI model to generate suggested amendments to inappropriate clauses.

[1321] Input: Incorrect clause

[1322] Output: Proposed amendments to inappropriate clauses

[1323] What it does: The server uses a generative AI model (e.g., GPT-3) to generate appropriate correction suggestions based on the prompt entered by the user.

[1324] Example: An example of a prompt might be, "My residential lease agreement has a one-month notice clause regarding termination terms. Please provide a general amendment to this."

[1325] Step 8:

[1326] The terminal presents the visual information and correction suggestions received from the server to the user.

[1327] Input: Visualized information, suggested corrections

[1328] Output: Information displayed in the user interface

[1329] Specific operation: The device displays the data received from the server, allowing the user to easily check the contents of the contract and proposed amendments.

[1330] Step 9:

[1331] The user reviews the information presented and takes the necessary action.

[1332] Input: Visualized information, suggested corrections

[1333] Output: User action (e.g., contract renegotiation, amendment adoption)

[1334] Specific Action: The user understands the contract based on the information presented and takes the necessary action to remove any unfavorable terms. In addition, this may include consulting a lawyer or renegotiating the contract.

[1335] Through these steps, the system is able to quickly and accurately understand the contents of a contract, identify inappropriate clauses, and suggest amendments.

[1336] (Application example 1)

[1337] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1338] In modern society, understanding contracts is extremely important, but many people find it difficult to understand them because they are technically complex. Furthermore, if an inappropriate clause is included, it is even more difficult to identify and correct it. This problem is particularly pronounced for users of electronic payment services, who face the challenge of finding appropriate action to avoid entering into contracts with unfavorable terms.

[1339] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1340] In this invention, the server includes a means for reading the contract document, a means for extracting text data, a means for analyzing the text data using natural language processing technology and extracting important information, a means for generating amendments and improvement proposals for inappropriate clauses, and a means for visually presenting the analysis results and amendment proposals to the user in a dashboard format, thereby enabling the content of the contract to be quickly and accurately understood, inappropriate conditions to be detected, and amendment proposals to be presented.

[1341] A "contract document" is a written legal agreement that specifies the rights and obligations of the parties.

[1342] "Reading means" refers to a means for mechanically or electronically reading the contents of a contract document.

[1343] "Text data" is character string data extracted from a contract document, and is used for content analysis and display.

[1344] "Extraction means" refers to methods or techniques for extracting specific information or data from the original document or data.

[1345] "Important information" refers to the contents and key elements of a contract document that require particular attention.

[1346] The "visual display means" is a means for displaying extracted information in a form that can be directly seen and understood by the user.

[1347] "Inappropriate clauses" refer to clauses in a contract document that violate the law or contain terms that are disadvantageous to the user.

[1348] An "amendment" is a new, appropriate clause proposed to correct an inappropriate clause.

[1349] "Natural language processing technology" is a technology that enables machines to understand, interpret, and generate human language.

[1350] A "dashboard format" is a visual interface format in which multiple pieces of information are arranged so that they can be viewed at a glance.

[1351] "User" refers to a person who uses this system to review and analyze contract documents.

[1352] A "server" is a centralized computer system for processing and storing data.

[1353] The present invention relates to a system that helps users accurately understand contracts and detect and correct inappropriate clauses. The system reads contract documents, extracts text data, visually displays important information, detects inappropriate clauses, and suggests corrections.

[1354] System configuration

[1355] Hardware

[1356] Smartphone: Equipped with a camera function for taking photos of contract documents and internet connection function.

[1357] Server: A centralized computer system for processing and storing data.

[1358] software

[1359] OCR software: Use Tesseract OCR to extract text data from images.

[1360] NLP software: Uses Google Cloud Natural Language API to analyze extracted text data and identify key elements.

[1361] Front-end software: Flutter is used to provide an interface that visually displays the extracted information to the user.

[1362] Data processing and calculation

[1363] 1. Data Collection:

[1364] Users can either take a photo of the contract document using their smartphone's camera or upload an existing PDF file, and the data is captured on the smartphone and sent to the server.

[1365] 2. Text extraction:

[1366] The image data or PDF file of the contract document sent by the terminal to the server is converted into character string data using Tesseract OCR on the server.

[1367] 3. Data Analysis:

[1368] The converted text data is then analyzed using the Google Cloud Natural Language API to identify key elements of the contract document (e.g., contract term, fee, cancellation terms, etc.).

[1369] 4. Visual Indication:

[1370] The extracted important information is visually displayed in a dashboard format using Flutter, allowing users to intuitively understand the contents of the contract.

[1371] 5. Detecting inappropriate clauses and suggesting amendments:

[1372] The server uses NLP technology to analyze the clauses in the contract document and check for inappropriate terms or clauses that are disadvantageous to the user. If an inappropriate clause is detected, the server generates suggestions for amendments or improvements to the clause.

[1373] 6. User Feedback:

[1374] The visual information and suggested amendments sent from the server are displayed on the user's smartphone, allowing the user to confirm the information and review the contract.

[1375] Specific examples

[1376] Example prompt sentence:

[1377] python

[1378] OCR processing

[1379] import pytesseract

[1380] from PIL import Image

[1381] def extract_text_from_image(image_path):

[1382] image = Image.open(image_path)

[1383] text = pytesseract.image_to_string(image, lang='jpn')

[1384] return text

[1385] Text analytics

[1386] from google.cloud import language_v1

[1387] def analyze_text(text):

[1388] client = language_v1.LanguageServiceClient()

[1389] document = language_v1.Document(content=text, type_=language_v1.Document.Type.PLAIN_TEXT)

[1390] response = client.analyze_entities(document=document)

[1391] return response

[1392] Extracting and visualizing key elements

[1393] import json

[1394] def extract_and_visualize(response):

[1395] elements = {}

[1396] for entity in response.entities:

[1397] elements[entity.name] = language_v1.Entity.Type(entity.type_).name

[1398] return json.dumps(elements, indent=2)

[1399] Specific example of the prompt sentence:

[1400] image_path = 'contract_image.jpg'

[1401] extracted_text = extract_text_from_image(image_path)

[1402] response = analyze_text(extracted_text)

[1403] visualization_data = extract_and_visualize(response)

[1404] print(visualization_data)

[1405] This system allows users to quickly understand the contents of contract documents, easily detect inappropriate clauses, and receive suggested amendments. This system is particularly useful for users of electronic payment services, and provides support for accurately understanding the contents of contracts.

[1406] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1407] Step 1:

[1408] Users can either take a photo of the contract document using their smartphone camera or upload an existing PDF file. The image or PDF file is then saved on the smartphone.

[1409] Step 2:

[1410] The terminal sends the image data or PDF file of the acquired contract document to the server, which then receives the contract document as input data.

[1411] Step 3:

[1412] The server converts the received image data or PDF file of the contract document into text data using OCR software. Specifically, it uses Tesseract OCR to extract text data from the image. This outputs the contents of the contract document as character string data.

[1413] Step 4:

[1414] The server then analyzes the converted text data using NLP software (Google Cloud Natural Language API). By analyzing this input text, important elements of the contract (such as contract period, fees, and cancellation conditions) are identified and classified.

[1415] Step 5:

[1416] The server uses front-end software (Flutter) to generate a visual interface in the form of a dashboard to visually display the analyzed important information, allowing the analysis results to be displayed in a user-friendly format.

[1417] Step 6:

[1418] The server also uses NLP technology to detect inappropriate clauses in the contract documents and generate amendments or improvement proposals for those clauses. Specifically, it checks each clause in the contract to see if it violates the law or contains terms that are disadvantageous to the user.

[1419] Step 7:

[1420] The visual information and suggested amendments sent from the server are displayed on the smartphone, allowing the user to review the information, examine the contract details, and take appropriate action.

[1421] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1422] The present invention relates to a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. In particular, the present invention not only visually displays the contents of a contract, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[1423] User operations

[1424] The first step for a user to use this system is to launch the smartphone application and take a picture of the contract or upload a PDF file. When the user takes a picture of the contract, the device acquires the image data and prepares to send it to the server.

[1425] Sending and Reading Data

[1426] The device sends the image data or PDF file of the contract to the server. When the server receives this data, it uses optical character recognition (OCR) technology to extract text data from the image, thereby obtaining the content of the contract as a string of characters.

[1427] Text data analysis

[1428] The server uses natural language processing (NLP) techniques to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fees, cancellation conditions, etc.). At this stage, the server can efficiently extract relevant information from the text data.

[1429] Visual display of information

[1430] The server then converts the extracted important information into a visually understandable format, such as a flowchart or diagram, allowing users to intuitively understand the contents of the contract.

[1431] Detecting inappropriate clauses and suggesting corrections

[1432] The server then analyzes the contract clauses to check for inappropriate terms or clauses that are unfavorable to the user. If an inappropriate clause is detected, the server generates a proposal to amend or improve the clause.

[1433] Emotion Engine Functions

[1434] A feature of the present invention is the incorporation of an emotion engine. The terminal recognizes emotions by analyzing the user's facial expressions and tone of voice while the user is reviewing the contract. For example, if the user shows signs of discomfort or confusion regarding the contents of the contract, the emotion engine detects this and notifies the server.

[1435] Adjusting how information is presented

[1436] The server dynamically adjusts the presentation of the contract based on the emotional data obtained from the emotion engine. For example, if the user is confused, it may provide simpler illustrations or additional annotations to aid understanding. If the user expresses surprise or displeasure, it may highlight suggested amendments to inappropriate clauses.

[1437] User Feedback

[1438] The device then presents the user with visual information and suggested revisions received from the server, as well as a presentation tailored to the user's emotions. By reviewing this information, the user can understand the contents of the contract and take action to avoid unfavorable terms.

[1439] Specific examples

[1440] Specific usage examples are shown below.

[1441] For example, consider a case where a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[1442] Furthermore, the server detects that the cancellation condition is "one month's notice" and presents this along with a suggested amendment: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the details of the illustrations to make the clause easier to understand.

[1443] Based on this information, users can review their contracts and take appropriate measures to avoid unfavorable terms.

[1444] The present invention is a comprehensive support system for quickly and easily understanding the contents of a contract, and by dynamically responding to the user's emotions, it is a system that can provide information that is more suited to each individual user.

[1445] The processing flow will be explained below.

[1446] Step 1:

[1447] The user launches the smartphone app and takes a photo of the contract or uploads a PDF file.

[1448] The device will activate the camera function and save the image taken by the user, and will also save the uploaded PDF file within the app.

[1449] Step 2:

[1450] The terminal sends the image data or PDF file of the contract to the server.

[1451] The device converts the stored image or PDF file into the appropriate format and sends it to the server using a secure protocol.

[1452] Step 3:

[1453] The server extracts the text data of the contract using OCR technology.

[1454] The server inputs the transmitted image or PDF file into the OCR tool, performs character recognition, and saves the extracted text data.

[1455] Step 4:

[1456] The server analyzes the extracted text data using natural language processing (NLP) techniques.

[1457] The server inputs the text data of the contract into an NLP engine and extracts important elements such as the contract period, rent, security deposit, and cancellation conditions. This extracted information is then classified and saved.

[1458] Step 5:

[1459] The server converts the extracted important information into a visually understandable format.

[1460] The server visualizes the extracted data as flowcharts and diagrams using graphical tools (e.g., D3.js).

[1461] Step 6:

[1462] The server checks the terms of the contract for inappropriate conditions or clauses that are unfavorable to the user.

[1463] The server analyzes the content based on pre-set rules and databases to detect inappropriate clauses.

[1464] Step 7:

[1465] The server generates corrections and recommendations for inappropriate clauses.

[1466] The server creates a correction proposal for the detected inappropriate clause and presents advantageous terms to the user.

[1467] Step 8:

[1468] The device analyzes the user's facial expressions and tone of voice to recognize emotions.

[1469] The device uses a built-in camera and microphone to analyze the user's facial expressions and voice in real time, and the emotion engine uses this data to recognize the user's emotions (e.g., confusion, anger, interest).

[1470] Step 9:

[1471] The server dynamically adjusts the presentation of the contract content based on the recognized emotion data.

[1472] The server adjusts the visualization depending on the user's emotional state, such as increasing the level of detail or simplifying the presentation.

[1473] Step 10:

[1474] The terminal presents the adjusted information to the user.

[1475] Based on the adjustment information received from the server, the terminal visually displays the contract contents and suggests amendments to inappropriate clauses.

[1476] Step 11:

[1477] The user reviews the presented information and deepens their understanding.

[1478] The user reviews the contents of the contract based on the illustrated information and proposed amendments, and proposes amendments to the contracting party as necessary.

[1479] Example 2

[1480] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1481] Conventional contract management systems have the problem of making it difficult to accurately understand the contents of a contract, and in particular, making it easy to overlook inappropriate clauses. Furthermore, there are no dynamic systems that present information based on the user's emotions, which often leaves users confused or takes a long time to understand the contract. For this reason, there is a need for a system that allows users to quickly and accurately understand the contents of a contract and take appropriate action to avoid unfavorable terms.

[1482] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring input contract document data, a means for transmitting the acquired contract document data to the server, a means for using character recognition technology to extract text data from the transmitted contract document data, a means for analyzing the extracted text data using natural language processing technology, a means for extracting important information from the analyzed text data and visually displaying it, a means for detecting inappropriate clauses contained in the contract document and generating proposed amendments, a means for recognizing the user's emotions and dynamically adjusting the information presentation method based on the emotion data, and a means for providing the adjusted information and proposed amendments as feedback to the user. This allows the user to quickly and accurately understand the contents of the contract, easily detect inappropriate clauses, and obtain optimal proposed amendments. Furthermore, dynamically adjusting the information presentation method aids the user's understanding and allows them to more efficiently grasp the contents of the contract.

[1483] "Input contract document data" refers to the image data or electronic document file of the contract provided to the system by the user.

[1484] "Means of acquisition" refers to the function for collecting and storing contract document data photographed or uploaded by the user.

[1485] "Means for sending" refers to the function for sending contract document data from the terminal to the server.

[1486] "Character recognition technology" refers to optical character recognition (OCR) technology for extracting text data from images and PDFs.

[1487] "Natural language processing technology" refers to technology that analyzes extracted text data and identifies important elements of a contract, such as analyzing nouns and verbs and understanding the context.

[1488] "Means for analysis" refers to the ability to process text data using natural language processing and extract important information.

[1489] "Visual display means" refers to a function for displaying important analyzed information in a user-friendly format (such as a flow chart or diagram).

[1490] "Inappropriate clauses" refers to terms and conditions contained in the contract documents that are unfavorable to the user or inappropriate content.

[1491] "Proposed Amendments" refers to the proposed amendments to improve an inappropriate provision.

[1492] "Emotional Data" refers to data such as a user's facial expressions and voice tone collected using emotion recognition technology.

[1493] "Dynamic adjustment means" refers to a function that changes the way information is presented based on the user's emotional data, making it easier for the user to understand the contract.

[1494] "Means for providing feedback" refers to the ability to provide tailored information or suggested revisions to the user to assist them in understanding the contents of the contract.

[1495] The present invention is a system that helps individuals accurately understand contracts and easily detect inappropriate clauses. This system not only visually displays the contents of contracts, but also has the ability to recognize the user's emotions and dynamically adjust the way information is presented based on those emotions.

[1496] System configuration and program processing

[1497] 1. Enter contract data

[1498] The user first launches the smartphone application and then takes a photo of the contract or uploads a PDF file, which saves the contract data on the device.

[1499] 2. Data transmission and OCR processing

[1500] The device will send the captured image data or uploaded PDF file to the server, which will then use optical character recognition (OCR) technology to extract the text data. Tools such as Tesseract will be used for this OCR technology.

[1501] 3. Data Analysis with NLP

[1502] The server then analyzes the extracted text data using natural language processing (NLP) techniques, such as using NLP libraries like spaCy or BERT. The analysis results identify key elements of the contract (such as contract duration, fees, and cancellation conditions) and categorize each element.

[1503] 4. Visual Indications

[1504] The server converts the identified important information into a visually easy-to-understand format and sends it to the device. This visualization is done using Diagramly (Draw.io) or D3.js, for example. The device displays this visual information to the user, allowing them to intuitively understand the contents of the contract.

[1505] 5. Detecting inappropriate clauses and suggesting amendments

[1506] The server analyzes the contract clauses using AI models (e.g., GPT-3 or BERT) to check for inappropriate terms or clauses that are disadvantageous to users. If any are detected, the server generates amendments or improvement suggestions for the clauses.

[1507] 6. Emotion analysis and dynamic adjustment of information presentation

[1508] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, capturing emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. The server dynamically adjusts the way information is presented based on the emotional data and presents the information in a way that is easy for the user to understand.

[1509] 7. User Feedback

[1510] The terminal presents the adjusted visual information and suggested modifications received from the server to the user, allowing the user to better understand the contents of the contract and take appropriate action.

[1511] Specific examples

[1512] Consider a scenario in which a user takes a photo of a residential rental contract and uploads it to the system. The device sends this image data to a server, which uses OCR technology (such as Tesseract) to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology (such as spaCy or BERT) to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions."

[1513] The server then visualizes these elements as a flowchart and displays it to the user. At this time, it detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required."

[1514] If the device's emotion engine detects a confused expression on the user's face, the server will increase the details of the illustrations and provide a clearer explanation of the clauses, allowing the user to review the contract and take appropriate action to avoid unfavorable terms.

[1515] Prompt Sentence Examples

[1516] "There is a clause in the residential rental agreement that allows for termination with one month's notice. Could you please suggest a general amendment to this?"

[1517] In this way, the system of the present invention provides comprehensive support functions for quickly and accurately understanding the contents of contracts and detecting and correcting inappropriate clauses. Furthermore, by dynamically changing the way information is presented according to the user's emotions, the system can provide support that is more suited to each individual user.

[1518] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1519] Step 1:

[1520] The user launches the smartphone application and either takes a photo of the contract or uploads a PDF file. Through the user's operation, the device acquires the captured image data or uploaded PDF file and saves it in local storage. The input is the contract data (image or PDF file) provided by the user, and the output is the contract data saved on the device.

[1521] Step 2:

[1522] The terminal transmits the stored contract data to the server. The contract data transmitted from the terminal to the server is the input, and the data received by the server is the output. In this step, data is transferred using a communication means. Specific operational examples include an operation in which the terminal transmits data to the server via the Internet.

[1523] Step 3:

[1524] The server performs OCR processing on the received contract data. Tesseract is used as the OCR technology, and the input data is image data or a PDF file. Through OCR processing, text data is extracted from the contract, and this is the output. Specific examples of operation include the process of recognizing characters in an image and extracting them as digital text.

[1525] Step 4:

[1526] The server analyzes the extracted text data using natural language processing (NLP) technology. For this analysis, NLP libraries such as spaCy and BERT are used. The input data is the text data extracted using OCR, and after analysis, the data is classified into key elements of the contract (contract period, fee, cancellation conditions, etc.). The output is this classified data. Specifically, the process involves extracting important elements from the text and classifying them into a database.

[1527] Step 5:

[1528] The server converts the identified important information for visual display. For example, Diagramly (Draw.io) or D3.js is used for this visualization. The input is the classified important information, and the output is the visually converted data (flowcharts and diagrams). Specifically, data is generated to present the contract period and cancellation conditions to the user as graphs and diagrams.

[1529] Step 6:

[1530] The server analyzes the clauses in the contract using an AI model (e.g., GPT-3 or BERT) to detect inappropriate clauses. The input is the analyzed text data, and the output is the detected inappropriate clauses and their corresponding correction suggestions. Specific examples of operation include a process to highlight inappropriate clauses and generate correction suggestions.

[1531] Step 7:

[1532] The device uses a camera and microphone to analyze facial expressions and voice tones while the user is reviewing the contract, obtaining emotional data. The input is the user's facial expressions and voice tones, and the output is the analyzed emotional data. This emotion analysis uses OpenFace and Microsoft Emotion API. Specific examples of operation include the process of capturing and evaluating the user's facial expressions with a camera.

[1533] Step 8:

[1534] The server dynamically adjusts the information presentation method based on the emotion data. The input is the user's emotion data, and the output is the adjusted information presentation method. If the user is confused, the server includes a process to provide simpler illustrations or additional annotations. As a specific example of operation, the server increases the details of the illustrations according to the user's emotion, making the explanation of the clauses easier to understand.

[1535] Step 9:

[1536] The terminal presents the adjusted visual information and correction proposals received from the server to the user. The input is the adjusted information and correction proposals, and the output is the information presented to the user. Specific operations include a process of displaying the adjusted visual information and correction proposals on the terminal screen so that the user can confirm them.

[1537] Through the above processing steps, the system efficiently understands the contents of the contract, detects and corrects inappropriate clauses, and provides optimal information to the user.

[1538] (Application example 2)

[1539] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1540] It is difficult for many people to accurately understand the contents of contract documents, and there is a risk of overlooking inappropriate clauses. Therefore, there is a need for a system that can accurately understand contract documents and effectively detect terms that are unfavorable to the user. In particular, a system that can present information according to the emotions of each user is necessary to deepen the user's understanding and support them in taking appropriate action.

[1541] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for reading a contract document, means for extracting text data from the read contract document, means for analyzing the extracted text data and extracting important information, means for visually displaying the extracted important information, means for detecting inappropriate clauses in the contract document and presenting suggested amendments, means for recognizing the user's emotions, and means for adjusting the information presentation method based on the recognized emotion data. This makes it possible to easily understand the contents of the contract document, detect inappropriate clauses, and present information according to the user's emotions.

[1542] A "contract document" is a document that describes the details of a transaction or agreement and is legally binding.

[1543] "Reading means" refers to any device or technology used to read contract documents in digital form.

[1544] "Text data" refers to data that represents character information in a digital format.

[1545] "Means of extraction" refers to techniques or methods for extracting specific information from contract documents.

[1546] "Means of analysis" refers to techniques and methods for examining the extracted text data and finding important information from it.

[1547] "Important Information" is information in a contract document that is particularly relevant and must be understood.

[1548] "Visual display means" refers to techniques and methods for visually expressing extracted important information in the form of diagrams, flow charts, etc.

[1549] "Inappropriate clauses" refer to conditions contained in contract documents that are inherently undesirable or disadvantageous to the user.

[1550] The "means for suggesting corrections" refers to techniques or methods for providing appropriate corrections or improvement suggestions for detected inappropriate clauses.

[1551] "Means for recognizing emotions" refers to technologies and methods for detecting emotions from a user's facial expressions and voice.

[1552] "Means for adjusting information presentation" refers to techniques and methods that dynamically change the way information is displayed based on recognized emotions.

[1553] The present invention is a system that presents the contents of a contract document to a user in an easy-to-understand manner, detects inappropriate clauses, and provides suggestions for correcting them.

[1554] This system uses the following hardware and software. Hardware includes devices such as smartphones, tablets, and computers. Software includes Tesseract OCR for optical character recognition (OCR), the Natural Language Toolkit (NLTK) for natural language processing (NLP), and the transformers library for generative AI models. To recognize user emotions, the system uses emotion analysis APIs such as Microsoft Azure Face API and Google Cloud Speech-to-Text.

[1555] The system analyzes the contract document and presents the information to the user by following the steps below: First, the user launches the smartphone application and takes a picture of the contract document or uploads a PDF file, which prepares the device to send the contract document data to the server.

[1556] The server receives the image data or PDF file of the contract document from the terminal and extracts the text data from the image using optical character recognition (OCR) technology (using Tesseract OCR), thereby obtaining the content of the contract document as character string data.

[1557] The server then uses natural language processing (NLP) techniques (using NLTK) to analyze the extracted text data. This analysis identifies and classifies key elements of the contract (e.g., contract duration, fee, cancellation conditions, etc.). At this stage, the server efficiently extracts relevant information from the text data.

[1558] The extracted important information is converted into a visually easy-to-understand format, such as a flowchart or diagram, to help users intuitively understand the contents of the contract document.

[1559] The server then analyzes the contract clauses to check for inappropriate conditions or clauses that are disadvantageous to the user. If an inappropriate clause is detected, it generates a proposal to amend or improve the clause and provides it to the user.

[1560] The emotion engine, a feature of the present invention, recognizes emotions by analyzing facial expressions and tone of voice while the user is reviewing the contract document. For example, if the user shows discomfort or confusion regarding the contract contents, the emotion engine detects this and notifies the server.

[1561] The server then dynamically adjusts the presentation of the contract based on the emotion data, providing simpler illustrations or additional annotations to aid understanding if the user is confused, and highlighting suggested amendments to inappropriate clauses if the user expresses surprise or discomfort.

[1562] Finally, the terminal will present the visual information and suggested modifications received from the server, as well as the adjusted presentation method, to the user, allowing the user to understand the contents of the contract document and take appropriate action to avoid unfavorable terms.

[1563] As a concrete example, consider a case where a user uploads a residential rental contract to the system. The device sends this image data to the server, which uses OCR technology to extract text information such as "Contract period: January 1, 2023 to December 31, 2023." The server then uses NLP technology to analyze this text data and identify key elements such as "contract period," "rent," "security deposit," and "cancellation conditions." The server then visualizes these elements as a flowchart and displays it to the user.

[1564] Furthermore, the server detects that the cancellation condition is "one month's notice" and suggests a correction: "Generally, this is reasonable, but three months' notice may be required." If the device's emotion engine detects a confused expression on the user's face, the server increases the illustrated details to make the clause easier to understand.

[1565] An example of a prompt is as follows:

[1566] "Extract the duration, fees, and cancellation conditions from a residential rental agreement and visualize them as a flowchart. Also, perform a sentiment analysis of the user and provide detailed explanations for any areas that cause confusion or anxiety."

[1567] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1568] Step 1:

[1569] The user launches the smartphone application and either takes a photo of the contract document or uploads a PDF file. The input is the contract document image or PDF file, and the output is that this data is saved on the device. Specifically, the user either takes a photo of the contract document using the app's camera function, or selects a PDF file using the file upload function.

[1570] Step 2:

[1571] The terminal prepares to send the contract document data to the server. The input is an image or PDF file of the contract document, and the output is the data converted into a format to be sent to the server. Specifically, the terminal encodes the image or PDF file and sends the data to the server over the network.

[1572] Step 3:

[1573] The server receives the image data or PDF file of the contract document and extracts the text data using optical character recognition (OCR) technology. The input is the contract document data, and the output is the extracted text data. Specifically, the server uses Tesseract OCR to detect characters in the image and convert them into text format.

[1574] Step 4:

[1575] The server uses natural language processing (NLP) techniques to analyze the extracted text data. The input is the extracted text data, and the output is the analyzed text data and key information extracted from it. Specifically, the server uses NLTK to tokenize and classify the text and identify key elements of the contract (e.g., contract term, fee, cancellation conditions, etc.).

[1576] Step 5:

[1577] The server converts the extracted key information into a visually understandable format. The input is the key information extracted from the parsed text data, and the output is a visually displayable format (e.g., a flowchart or diagram). Specifically, the server schematizes the data and renders it for the user interface.

[1578] Step 6:

[1579] The server analyzes the clauses in the contract document and detects inappropriate conditions or clauses that are disadvantageous to the user. The input is the full text of the contract document and extracted important information, and the output is the detected inappropriate clauses and suggested corrections. Specifically, the server analyzes the text using specific rules and generative AI models (e.g., transformers) and points out problems.

[1580] Step 7:

[1581] The device analyzes the user's facial expressions and voice using an emotion recognition API (for example, Microsoft Azure Face API or Google Cloud Speech-to-Text) to obtain emotion data. The input is the user's real-time facial and voice data, and the output is analyzed emotion data. Specifically, the device collects data using the camera and microphone, sends it to the emotion recognition API, and obtains the analysis results.

[1582] Step 8:

[1583] The server dynamically adjusts the information presentation method based on the emotional data. The input is the emotional data and extracted important information, and the output is the adjusted information presentation method. Specifically, the server evaluates the emotional data and adds explanations and annotations that match the user's emotions.

[1584] Step 9:

[1585] The device receives visual information and correction suggestions from the server and presents the presentation method adjusted based on the emotion to the user. The input is feedback data from the server, and the output is visual data displayed to the user. Specifically, the device renders the visual data on a user interface and displays it to the user.

[1586] Step 10:

[1587] The user reviews the contract document based on the information provided and takes appropriate action to avoid unfavorable terms. The input is visual information and suggested revisions, and the output is the user's actions and decisions. Specifically, the user understands the information provided and makes an appropriate decision regarding the contract.

[1588] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1589] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1590] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1591] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1592] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1593] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1594] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1595] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1596] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1597] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1598] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1599] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1600] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1601] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1602] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1603] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1604] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1605] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1606] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1607] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1608] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1609] The following is further disclosed regarding the above embodiment.

[1610] (Claim 1)

[1611] A means for reading a contract document;

[1612] A means for extracting text data from the read contract document;

[1613] A means for analyzing the extracted text data and extracting important information;

[1614] a means for visually displaying the extracted important information;

[1615] A means for detecting inappropriate clauses in contract documents and suggesting corrections;

[1616] A system including:

[1617] (Claim 2)

[1618] 2. The system of claim 1, wherein the contract document is a photographed image or a PDF file.

[1619] (Claim 3)

[1620] 2. The system of claim 1, wherein the means for extracting text data from the scanned contract document utilizes optical character recognition technology.

[1621] (Claim 4)

[1622] 2. The system of claim 1, wherein the means for analyzing the extracted important information utilizes natural language processing technology.

[1623] (Claim 5)

[1624] 10. The system of claim 1, wherein the visual displaying means generates a flowchart or diagram.

[1625] "Example 1"

[1626] (Claim 1)

[1627] a means for reading contract documents;

[1628] A means for extracting text data from the read contract document;

[1629] A means for analyzing the extracted text data and extracting important information;

[1630] a means for visually displaying the extracted important information;

[1631] A means for detecting inappropriate clauses in contract documents and suggesting corrections;

[1632] a means including a generative AI model that generates suggested revisions based on prompts entered by a user;

[1633] A system including:

[1634] (Claim 2)

[1635] 2. The system of claim 1, wherein the contract document is in the form of a photographed image or an electronic file.

[1636] (Claim 3)

[1637] 2. The system of claim 1, wherein the means for extracting text data from the scanned contract document utilizes optical character recognition technology.

[1638] "Application Example 1"

[1639] (Claim 1)

[1640] A means for reading a contract document;

[1641] A means for extracting text data from the read contract document;

[1642] A means for analyzing the extracted text data and extracting important information;

[1643] a means for visually displaying the extracted important information;

[1644] A means for detecting inappropriate clauses in contract documents and suggesting corrections;

[1645] A means of utilizing natural language processing technology to generate amendments and improvement suggestions for inappropriate clauses;

[1646] A means to visually present the analysis results and suggested modifications to the user in a dashboard format;

[1647] A system including:

[1648] (Claim 2)

[1649] 2. The system of claim 1, wherein the contract document is a photographed image or a PDF file.

[1650] (Claim 3)

[1651] 2. The system of claim 1, wherein the means for extracting text data from the scanned contract document utilizes optical character recognition technology.

[1652] "Example 2: Combining Emotion Engines"

[1653] (Claim 1)

[1654] A means for acquiring input contract document data;

[1655] means for transmitting the acquired contract document data to a server;

[1656] A means for utilizing character recognition technology to extract text data from the transmitted contract document data;

[1657] A means for analyzing the extracted text data using natural language processing technology;

[1658] A means for extracting important information from the analyzed text data and visually displaying it;

[1659] A means for detecting inappropriate clauses contained in contract documents and generating amendment proposals;

[1660] means for recognizing a user's emotion and dynamically adjusting information presentation methods based on the emotion data;

[1661] a means for providing feedback of the adjusted information and suggested modifications to the user;

[1662] A system including:

[1663] (Claim 2)

[1664] 2. The system according to claim 1, wherein the contract document is photographed image data or an electronic document file.

[1665] (Claim 3)

[1666] 10. The system of claim 1, wherein the means for extracting text data from the acquired contract document data utilizes optical character recognition technology.

[1667] "Application example 2 when combining emotion engines"

[1668] (Claim 1)

[1669] A means for reading a contract document;

[1670] A means for extracting text data from the read contract document;

[1671] A means for analyzing the extracted text data and extracting important information;

[1672] a means for visually displaying the extracted important information;

[1673] A means for detecting inappropriate clauses in contract documents and suggesting corrections;

[1674] means for recognizing a user's emotion;

[1675] means for adjusting the information presentation method based on the recognized emotion data;

[1676] A system including:

[1677] (Claim 2)

[1678] 2. The system of claim 1, wherein the contract document is a photographed image or a PDF file.

[1679] (Claim 3)

[1680] 2. The system of claim 1, wherein the means for extracting text data from the scanned contract document utilizes optical character recognition technology. [Explanation of symbols]

[1681] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for reading a contract document; A means for extracting text data from the read contract document; A means for analyzing the extracted text data and extracting important information; a means for visually displaying the extracted important information; A means for detecting inappropriate clauses in contract documents and suggesting corrections; A system including:

2. 2. The system of claim 1, wherein the contract document is a photographed image or a PDF file.

3. 10. The system of claim 1, wherein the means for extracting text data from the scanned contract document utilizes optical character recognition technology.

4. 2. The system of claim 1, wherein the means for analyzing the extracted important information utilizes natural language processing techniques.

5. 10. The system of claim 1, wherein the visual display means generates a flowchart or diagram.

Images