system

Abstract

Provide a system. 【Solution means】 Receiving means for inputting contract information, Conversion means for converting to a data format corresponding to the case where the contract information is image or document data, Analysis means for extracting text from the converted data, Identification means for detecting points to be noted based on the extracted text, Evaluation means for evaluating the risk of the identified points and generating an evaluation result, Generation means for visually highlighting and providing the evaluation result to the user, Output means for outputting the generated feedback to the user, Function for inputting a contract through the image acquisition means of the terminal, Risk identification function using natural language processing technology, Function for visually displaying a warning on a smart device, A system including

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Contract documents and terms of use usually contain many technical terms and legal clauses, which are difficult for ordinary people to understand. As a result, there are cases where people sign without fully checking the content and suffer unexpected disadvantages. It is an issue to improve such a situation and provide an environment where users can easily understand the contract content and make careful judgments.

Means for Solving the Problems

[0005] This invention solves this problem by analyzing text data contained in contracts and terms of service and detecting clauses that require attention. Users can input contract information as image or document data, which is then converted to an appropriate data format by a conversion means. Subsequently, an analysis means extracts the text, and an identification means identifies risky sections using natural language processing technology. An evaluation means evaluates these sections, and a generation means generates feedback in a format that is easy for the user to understand. An output means visually presents this feedback to the user and realizes a system that suggests necessary actions.

[0006] "Contract information" refers to document data and image data contained in contracts and terms of service.

[0007] "Reception means" refers to the interface or device used by the user to input contract information.

[0008] "Conversion means" refers to a mechanism that transforms contract information, when it is in the form of image or document data, into a format that is easier to process.

[0009] "Analysis means" refers to the technology or process of extracting character information from data generated by the conversion means.

[0010] "Identification means" refers to systems or algorithms used to identify clauses that require attention from analyzed text.

[0011] "Evaluation method" refers to the process of analyzing the risks of a clause based on identified information and generating the results.

[0012] "Generation means" refers to functions and systems that assemble feedback for users based on evaluation results.

[0013] "Output means" refers to the means or devices used to present the generated feedback to the user.

[0014] "Natural language processing technology" refers to a series of technologies that enable computers to process and analyze the language that humans use on a daily basis. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention provides a system that helps users easily understand contract information and avoid signing risky contracts. The system includes a series of processes, starting with the user inputting contracts or terms of service, effectively analyzing that information, and providing feedback.

[0037] First, the user uses a terminal to input contract information as an image or text file, or provides the system with a web page URL. This contract information is then acquired by the receiving device. The terminal then sends the contract information to the server, which converts the received data into an appropriate data format. If it is image data, text is extracted using OCR technology; if it is text data, it proceeds directly to the analysis process.

[0038] The server examines the text data extracted by the analysis tool using an identification tool to find clauses that require attention. Natural language processing technology is used in this process to identify risky clauses and specialized legal terms. Subsequently, the evaluation tool performs a risk analysis on these clauses and generates feedback that suggests some action to the user.

[0039] The feedback generated by the generation mechanism is then sent to the terminal by the output mechanism and presented to the user. This feedback visually highlights risky clauses and is displayed in an easy-to-understand format. This allows the user to fully understand the contents of the contract or terms of service and make important decisions before signing.

[0040] As a concrete example, consider a scenario where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement's webpage into this system. The server analyzes the page and generates warnings about automatic renewal clauses and hefty penalties for early termination. Based on this information, the user can reconfirm the terms before signing. This allows the user to avoid disadvantages and make safe and informed decisions.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user enters contract information from their device. This is done by taking a picture, selecting an existing file, or entering a web page URL.

[0044] Step 2:

[0045] The terminal sends the entered contract information to the server. Images are encoded in Base64 format, and URLs are sent as JSON data.

[0046] Step 3:

[0047] The server processes the received contract information appropriately. In the case of images, text is extracted from the image through OCR processing. In the case of URLs, web scraping technology is used to obtain the page content as text data.

[0048] Step 4:

[0049] The server analyzes the text data extracted by the analysis tools using natural language processing. This identifies risky clauses and important contractual terms.

[0050] Step 5:

[0051] The server evaluates the risk level based on the analyzed data using evaluation tools. It then generates evaluation results that form the basis for feedback.

[0052] Step 6:

[0053] The generation mechanism creates a user feedback message based on these evaluation results. This message highlights and explains risky clauses and suggests necessary actions for the user.

[0054] Step 7:

[0055] The feedback is sent to the terminal via an output device. The terminal visually displays the feedback to the user, highlighting important elements of the contract.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] When users try to understand contracts and terms of service, they may overlook technical terms or risky clauses, which poses a risk of misunderstanding and disadvantage. This makes it difficult to make safe and informed decisions.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes information acquisition means for inputting contract information, data manipulation means for converting the contract information into a data format, and data analysis means for extracting text from the converted data. This makes it possible to accurately understand the content of the contract information and effectively identify risky clauses.

[0061] "Information acquisition means" refers to means that have the function of receiving contract information as input from the user.

[0062] "Data manipulation means" refers to means of processing acquired contract information to convert it into an analyzable format.

[0063] A "data analysis tool" is a tool equipped with the function of extracting and analyzing text from converted data.

[0064] An "item identification means" is a means that has the function of identifying clauses that require attention or content that poses a risk from the extracted text data.

[0065] A "generative AI model" refers to artificial intelligence technology that uses previously learned data to identify important patterns and specific items from target data.

[0066] A "risk assessment tool" is a tool that has the function of evaluating the risk level of an identified clause and generating the result.

[0067] An "information generation means" is a means equipped with the function of generating feedback to be provided to the user based on the results of a risk assessment.

[0068] An "information presentation tool" is a means that has the function of visually presenting generated feedback to the user and aiding in their understanding.

[0069] This invention is a system for providing contract information in a format that is easy for users to understand, and it operates primarily with three elements: a server, a terminal, and a user.

[0070] The user provides contract and terms of service information using their device. This information can be provided as an image file, selected as a text file, or by entering the URL of the webpage containing the contract. The information retrieval system on the device accepts this input.

[0071] The received information is transmitted from the terminal to the server. The server uses data manipulation tools to convert the received contract information into an appropriate format. For example, if it is image data, optical character recognition (OCR) is used to extract the text. The converted text data is then analyzed in detail by data analysis tools, and item identification tools are used to identify important clauses and risky content. Generative AI models are used in this process to accurately identify complex legal terminology and clauses that may affect end users.

[0072] Next, the server's risk assessment means evaluates the risk level based on the identified information, and the information generation means creates feedback for the user. The generated feedback is sent to the terminal through the information presentation means and displayed to the user with visual emphasis. For example, high-risk clauses are highlighted with color-coded icons to draw the user's attention.

[0073] A concrete example is a situation where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement page into their device. The server analyzes the page and generates warnings about automatic renewal clauses for subscription services and high penalties. By reviewing this information and reconsidering the contract before signing, the user can avoid disadvantages and make a safe and wise decision.

[0074] An example of a prompt message would be: "Please identify and visually highlight any risky clauses in the following contract. In particular, please provide details regarding the automatic renewal clause and penalties."

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The user enters contract information through a terminal. This information is provided as an image file, text file, or webpage URL. The terminal's information acquisition mechanism receives this input and passes the contract information to the system as data. In this step, the input is the contract information provided by the user, and the output is the contract information as it has been incorporated into the system.

[0078] Step 2:

[0079] The terminal sends the captured contract information to the server. The server uses data manipulation tools to perform a conversion process: if the input information is image data, it extracts text using OCR technology; if it is text data, it sends it directly for analysis. The input for this step is the raw contract information provided by the user, and the output is the contract information converted into text data.

[0080] Step 3:

[0081] The server receives text data and performs a detailed analysis using data analysis tools. Specifically, it processes the data to identify important clauses and risky items within the contract. In this process, a generative AI model uses pre-trained data to identify clauses that require attention. The input for this step is contract information converted into text data, and the output is the identified important clauses and risk information.

[0082] Step 4:

[0083] The server's risk assessment means evaluates the risk level based on the identified clauses. Based on the results of the risk assessment, the information generation means generates feedback. This feedback includes detailed information and recommended actions regarding the clauses that the user should pay attention to. The input to this step is the identified risk information, and the output is the materialized feedback.

[0084] Step 5:

[0085] The server-generated feedback is sent to the terminal via an information display device and presented to the user. The terminal visually highlights the feedback, drawing the user's attention by using colors and icons to highlight particularly high-risk clauses. The user reviews the provided information and makes a final decision. The input for this step is the generated feedback, and the output is the visually highlighted feedback received by the user.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] In the process of understanding contracts and terms of service, users often overlook complex clauses or unfavorable conditions, resulting in the risk of entering into unfavorable agreements. In particular, electronic payment services may contain contract terms that require careful attention, and users often find it difficult to accurately understand these terms. This invention aims to help users easily understand the content of contract information, identify risky areas, and make informed decisions.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes receiving means for inputting contract information, conversion means for converting the contract information into a corresponding data format if it is image or document data, and analysis means for extracting text from the converted data. This allows users to easily understand risky contract terms by taking a picture of or uploading a contract or terms of service with a smart device, and the system analyzing the content and visually highlighting areas that require attention.

[0091] "Contract information" refers to data that includes the contents of terms of service, contracts, and other similar documents. This information should be provided in a format that is easy for users to understand.

[0092] "Acceptance method" refers to the function for obtaining contract information from a terminal. It is the first step in the user entering contract data into the system.

[0093] "Conversion means" refers to the function of converting contract information in image or document data format into an appropriate data format. This is the process of making text information extractable using OCR technology, etc.

[0094] "Analysis means" refers to the function that extracts text from the converted data and analyzes the contract content. It plays a role in preparing the information for further processing.

[0095] "Identification means" refers to a function that identifies areas requiring attention based on analyzed text data. This is necessary to identify risky clauses and warn users.

[0096] The "evaluation method" refers to a function that assesses the risk of identified areas and generates evaluation results. This provides users with information-based feedback.

[0097] "Generation method" refers to the function that generates feedback for the user based on the evaluation results. It is a process for providing information in a way that helps the user understand, such as through visual highlighting.

[0098] "Output mechanism" refers to a function that provides generated feedback to the user. It effectively presents information to help the user understand the contract terms.

[0099] To implement this invention, the system employs a configuration combining multiple hardware and software components. The user inputs contract information in image or text format using a smart device. The acquired contract information is transmitted to the system via a receiving means, where it undergoes a process of converting the data into text using OCR technology. This process utilizes an OCR library such as pytesseract.

[0100] The converted text is sent to a server and analyzed using natural language processing (NLP) technology. This analysis is performed using a NLP library such as spaCy to identify risky clauses and specialized legal terminology. The identified risky sections are evaluated in detail using evaluation tools to generate appropriate feedback for the user. The evaluation results are processed using generation tools for visual highlighting on smart devices.

[0101] For example, consider a scenario where a user is about to sign a contract for a new electronic payment service. If the user takes a photo of the contract and uploads the image to the system, the system can identify automatic withdrawal conditions and high penalty clauses, and warn the user. This allows the user to review unfavorable contract terms in advance and make an informed decision.

[0102] An example of a prompt to input into a generative AI model is, "Teach me how to develop an app that highlights risk clauses in contracts." This prompt allows the model to learn techniques for specifically identifying risky contract terms.

[0103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0104] Step 1:

[0105] The user uses a terminal to input contract information as an image or URL. This input is received by the system via a receiving mechanism. The input data includes photos of contracts and URLs of web pages. The terminal prepares to send this data to the server.

[0106] Step 2:

[0107] To analyze the received data, the server first converts the image data into text data. This process utilizes OCR technology such as pytesseract. By extracting text from the input image data, the contract details can be treated as textual information. The extracted text is then output.

[0108] Step 3:

[0109] The server uses natural language processing techniques to evaluate the data and analyze the converted text. This process utilizes libraries such as spaCy. From the text data received as input, it identifies risky clauses and technical terms, and identifies points that require attention. The identified risky clauses are then output.

[0110] Step 4:

[0111] The server evaluates the identified risk areas and generates feedback based on the results. It uses evaluation tools to generate evaluation results and organizes information to determine the risks to the user. Detailed risk assessment information is output as feedback.

[0112] Step 5:

[0113] The terminal presents the generated feedback to the user. It processes the feedback to visually highlight risky areas and displays them on the terminal screen using a generation method. The user can then review this feedback to deepen their understanding of the contract and make a final decision. The output is the visually processed feedback.

[0114] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0115] This invention is a system that allows users to easily understand information related to contracts and terms of service, and adjusts feedback while also considering the user's emotional state at the time. In addition to the basic functions of inputting and analyzing contract information, this system also includes a function that recognizes the user's emotions using an emotion engine and generates feedback based on those emotions.

[0116] First, the user uses their device to input contract and terms of service information as images or URLs. The device sends this contract information to the server, which extracts the text data using OCR technology. Next, the server analyzes the text data using natural language processing to identify contract clauses that require attention.

[0117] This process also incorporates an emotion engine, which evaluates the user's emotions in response to the data extracted by the analysis tools. This system uses non-verbal data obtained from the user via the device, such as facial recognition and voice data, to determine emotions. If the user is feeling anxious, the emotion engine takes that emotion into account and adjusts the feedback to be more detailed or highlights important points.

[0118] The generation mechanism prepares user-optimized feedback based on emotional data provided by the emotion engine and risk information identified by the identification mechanism. This feedback visually displays the contract's risk level and is presented in a format that is easily understandable to the user.

[0119] For example, when a user is checking the terms and conditions of a new online service, if the system detects the user's unease, it will provide a detailed explanation of any special clauses or points to note regarding automatic renewal included in the contract, and offer specific suggestions to help the user make a decision with confidence. In this way, users can gain a deep understanding of the contract and make decisions with confidence.

[0120] Thus, as an interactive system that combines emotion recognition technology, the present invention provides users with better support for making contractual decisions.

[0121] The following describes the processing flow.

[0122] Step 1:

[0123] The user enters specific contract information on their device. Input methods include taking a picture, selecting an existing file, or directly entering a URL.

[0124] Step 2:

[0125] The terminal sends the entered contract information to the server. Image data is converted to Base64 format, and text data and URLs are sent in JSON format.

[0126] Step 3:

[0127] The server analyzes the received contract information. If image data is sent, the server uses OCR to extract text from the image. In the case of a URL, the page content is obtained in text format via web scraping.

[0128] Step 4:

[0129] The server uses analysis tools to analyze text data using natural language processing and identify clauses that require attention. Based on these analysis results, the risk points of the contract are identified.

[0130] Step 5:

[0131] The emotion engine activates and performs non-verbal information acquisition from the user's device, such as facial expression analysis via the camera and voice tone analysis via the microphone. Based on this information, the user's emotional state is evaluated.

[0132] Step 6:

[0133] The server combines identified risk information with the results of the emotion engine to generate feedback for the user. This feedback is tailored to the user's emotional state. For example, if the user is feeling anxious, the feedback will be more detailed and reassuring.

[0134] Step 7:

[0135] The device receives feedback sent from the server and presents it in a visual and user-friendly format. This highlights the contract's risk level, making it intuitively understandable to the user.

[0136] (Example 2)

[0137] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0138] Conventional contract information analysis systems have faced challenges in providing optimal feedback to users because it is difficult for them to fully understand the details of contracts and terms of service, and because the user's emotional state is not taken into consideration. In particular, when users are feeling anxious, they may overlook important information, so there is a need to provide contract information that takes the user's emotional state into account.

[0139] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0140] In this invention, the server includes a function for inputting contract information, a function for converting the contract information into a corresponding data format, and a function for detecting areas of interest based on the extracted text. This makes it possible to appropriately analyze the contract information of interest while taking into account the user's emotional state and provide the user with optimal feedback.

[0141] "Contract information" refers to data contained in contracts and terms of service, and is the information necessary for a user to consent to a particular service or product.

[0142] The "data format conversion function" refers to a function that converts the entered contract information into a parseable format, including the process of converting image and PDF data into text data.

[0143] The "text extraction function" refers to the process of extracting character information from the converted data and obtaining text data that will serve as the basis for analysis.

[0144] The "function to detect areas requiring attention" refers to the process of identifying information that is important to the user or content that may contain potential risks from the extracted text data.

[0145] The "risk assessment function" is a process that evaluates potential risks from detected information and determines their importance and impact on users.

[0146] The "function for evaluating emotions" refers to the process of analyzing non-verbal data such as the user's facial expressions and voice to identify the user's psychological state and emotions.

[0147] The "feedback generation function" is the process of constructing information to provide to the user based on assessed risk and sentiment data, and includes formats such as text and visual displays.

[0148] "Output function" refers to a function that presents the generated feedback in a format that is easy for the user to understand, and includes displaying it on the device.

[0149] This invention relates to a system that easily analyzes contract information and presents its contents in a format that is easy for users to understand. Users input contract and terms of service information using a terminal in image or URL format. The terminal temporarily stores this information and sends it to a server.

[0150] The server performs analysis on the received information. First, it uses OCR technology to convert image data into text data. Specifically, general OCR software is used here. Next, it uses a generative AI model to process the extracted text data and identify clauses that require attention or items that carry risks. For example, natural language processing toolkits and machine learning algorithms are utilized.

[0151] In addition, the server analyzes non-verbal data obtained from the user's terminal. By acquiring the user's facial expressions and voice and performing emotion analysis, it determines the user's emotional state. Software tools are used as emotion recognition technology in this process.

[0152] The server integrates analyzed contract information with data on the user's emotional state and generates feedback based on this. This feedback is presented to the user in the form of visualized risk levels and highlighting of important clauses. For example, if the system detects a user's anxious expression while reviewing an online service contract, it will provide detailed explanations regarding special clauses and automatic renewal considerations. An example of a prompt related to this operation might be, "Please provide a clear explanation of the contract clauses the user is concerned about," which could be input to the generating AI model.

[0153] In this way, by incorporating emotion recognition technology, users can gain a deeper understanding of the contract terms and create an environment where they can make decisions with confidence. The system as a whole provides an interactive analytical tool that combines detailed contract analysis with an evaluation of the user's emotional state.

[0154] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0155] Step 1:

[0156] The user enters contract information in image or URL format using a terminal. This becomes the initial input data for the system. The terminal temporarily stores this information and prepares to send it to the server. At this time, the input is completed via the interface on the user's terminal, and the destination server IP address is set.

[0157] Step 2:

[0158] The terminal sends the saved contract information to the server. The server receives the input data and converts it into a format for text analysis. Here, the server uses OCR technology to convert image files into text data. Specifically, it performs calculations to recognize characters from the image and convert them into digital text. The output of this process is text data containing the contract information.

[0159] Step 3:

[0160] The server feeds the text data extracted by OCR into a natural language processing engine. A generative AI model analyzes this text data to determine whether there are any clauses that require attention or any particular risks. Specifically, the generative AI model analyzes the text and performs data calculations to extract keywords and expressions related to risk. The output at this stage is a list of identified important clauses.

[0161] Step 4:

[0162] Simultaneously, the device collects nonverbal data such as the user's facial expressions and voice, and sends it to the server. The server analyzes this data using an emotion recognition algorithm to evaluate the user's emotional state. Here, the server analyzes the user's nonverbal signs and determines the emotion they indicate (e.g., anxiety, confusion). The output identifies the user's emotional state.

[0163] Step 5:

[0164] The server generates optimal feedback based on the key contract clauses identified by the generation AI model and the user's assessed emotional state. Feedback generation involves data processing to visually indicate the contract's risk level and highlight points the user should pay attention to. The generated feedback is then output and ready to use.

[0165] Step 6:

[0166] The terminal receives feedback information sent from the server and presents it to the user. Specifically, a well-formatted dashboard is displayed on the terminal, designed to make it easy for the user to understand the contract information. In this process, the output is provided in the form of visualized feedback for the user.

[0167] (Application Example 2)

[0168] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0169] Contract information and terms of service are often difficult for many users to understand, and as a result, they may accept contracts without fully understanding their contents. This situation can lead to disadvantages later on, so there is a need for a system that supports users in understanding contract contents and using them with peace of mind.

[0170] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0171] In this invention, the server includes a device for inputting contract information, a conversion device for converting the contract information into an appropriate information format if it is image or document information, and an analysis device for extracting text information from the converted information. This makes it possible to detect important points in the contract content and analyze the user's emotions, thereby providing more understandable and reliable feedback.

[0172] A "device for inputting contract information" is a device that has an interface for users to provide contract information to a system.

[0173] A "conversion device" is a device equipped with the function of converting received contract information into a format that is easy for the system to analyze.

[0174] An "analysis device" is a device that extracts necessary textual information from converted information and analyzes the contract details.

[0175] An "identification device" is a device that has the function of identifying areas of interest within contract information based on analyzed textual information.

[0176] An "evaluation device" is a device that evaluates the risk level of a contract based on identified sections and generates the evaluation results.

[0177] An "emotion analysis device" is a device that uses nonverbal data obtained from a user to analyze their emotions.

[0178] A "generating device" is a device equipped with the function of generating information to be provided to the user based on evaluation results and sentiment analysis results.

[0179] An "output device" is a device that visually displays the generated feedback to the user, highlighting and providing information.

[0180] In embodiments of this invention, the system includes means for effectively analyzing contract information and providing optimized feedback to the user. The specific configuration and operation are described below.

[0181] When a user first enters their contract information, they use a device such as a smartphone or tablet. The device receives the contract information by either taking a picture of the contract using its camera or by entering a URL, and then sends it to the server.

[0182] The server uses Tesseract OCR to extract text data from these image data. The extracted text is then analyzed using a natural language processing library such as spaCy to identify important clauses and risks within the contract information.

[0183] Furthermore, using OpenCV-based facial recognition technology and TENSORFLOW® emotion analysis models, the system analyzes the user's emotions from their facial expressions and voice. This allows the system to understand the user's level of anxiety and adjust the content and emphasis of the feedback accordingly.

[0184] The generated feedback is provided to the user through an intuitive interface developed using Flutter®. This feedback highlights points of particular importance and presents them in a visually clear manner, allowing users to better understand the contract and make informed decisions.

[0185] As a concrete example, when a user registers for a new mobile payment service, this system allows them to take a picture of the terms of service displayed during the registration process. The system detects anxiety from the user's facial expressions and provides feedback that explains the terms in detail, especially those concerning automatic renewal and fees.

[0186] An example of a prompt for the generative AI model regarding the use of this system is: "If a user is reading a new service agreement and feels uneasy, how would you use AI to adjust the feedback? As a specific example, consider an application that combines facial recognition and natural language processing."

[0187] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0188] Step 1:

[0189] The user enters contract information. Using a smartphone or tablet, the contract information is collected on the device by taking a picture of the contract or entering a URL. At this stage, the input data is either an image or a URL.

[0190] Step 2:

[0191] The terminal sends the collected contract information to the server. The server uses Tesseract OCR to extract text data from the image data. This process takes image data as input and outputs text data.

[0192] Step 3:

[0193] The server analyzes the extracted text data. Using natural language processing libraries such as spaCy, it identifies important clauses and risks. The input here is text data, and the output is risk information.

[0194] Step 4:

[0195] The server analyzes the user's emotions. It uses OpenCV for facial recognition and a TensorFlow model to determine the emotion. Here, non-verbal data (images and audio) acquired by the camera is used as input data, and the output is the user's emotional state.

[0196] Step 5:

[0197] The server generates feedback based on the evaluation results and sentiment data. The content of the feedback is adjusted according to the evaluated risk information and the user's sentiment. The inputs to this process are risk information and sentiment data, and the output is feedback data.

[0198] Step 6:

[0199] The device provides the user with generated feedback. This is visually displayed through a Flutter-based UI, highlighting points that require particular attention. The input is feedback data, and the output is visual information presented to the user.

[0200] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0201] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0202] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0203] [Second Embodiment]

[0204] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0205] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0206] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0207] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0208] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0209] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0210] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0211] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0212] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0213] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0214] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0215] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0216] This invention provides a system that helps users easily understand contract information and avoid signing risky contracts. The system includes a series of processes, starting with the user inputting contracts or terms of service, effectively analyzing that information, and providing feedback.

[0217] First, the user uses a terminal to input contract information as an image or text file, or provides the system with a web page URL. This contract information is then acquired by the receiving device. The terminal then sends the contract information to the server, which converts the received data into an appropriate data format. If it is image data, text is extracted using OCR technology; if it is text data, it proceeds directly to the analysis process.

[0218] The server examines the text data extracted by the analysis tool using an identification tool to find clauses that require attention. Natural language processing technology is used in this process to identify risky clauses and specialized legal terms. Subsequently, the evaluation tool performs a risk analysis on these clauses and generates feedback that suggests some action to the user.

[0219] The feedback generated by the generation mechanism is then sent to the terminal by the output mechanism and presented to the user. This feedback visually highlights risky clauses and is displayed in an easy-to-understand format. This allows the user to fully understand the contents of the contract or terms of service and make important decisions before signing.

[0220] As a concrete example, consider a scenario where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement's webpage into this system. The server analyzes the page and generates warnings about automatic renewal clauses and hefty penalties for early termination. Based on this information, the user can reconfirm the terms before signing. This allows the user to avoid disadvantages and make safe and informed decisions.

[0221] The following describes the processing flow.

[0222] Step 1:

[0223] The user enters contract information from their device. This is done by taking a picture, selecting an existing file, or entering a web page URL.

[0224] Step 2:

[0225] The terminal sends the entered contract information to the server. Images are encoded in Base64 format, and URLs are sent as JSON data.

[0226] Step 3:

[0227] The server processes the received contract information appropriately. In the case of images, text is extracted from the image through OCR processing. In the case of URLs, web scraping technology is used to obtain the page content as text data.

[0228] Step 4:

[0229] The server analyzes the text data extracted by the analysis tools using natural language processing. This identifies risky clauses and important contractual terms.

[0230] Step 5:

[0231] The server evaluates the risk level based on the analyzed data using evaluation tools. It then generates evaluation results that form the basis for feedback.

[0232] Step 6:

[0233] The generation mechanism creates a user feedback message based on these evaluation results. This message highlights and explains risky clauses and suggests necessary actions for the user.

[0234] Step 7:

[0235] The feedback is sent to the terminal via an output device. The terminal visually displays the feedback to the user, highlighting important elements of the contract.

[0236] (Example 1)

[0237] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0238] When users try to understand contracts and terms of service, they may overlook technical terms or risky clauses, which poses a risk of misunderstanding and disadvantage. This makes it difficult to make safe and informed decisions.

[0239] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0240] In this invention, the server includes information acquisition means for inputting contract information, data manipulation means for converting the contract information into a data format, and data analysis means for extracting text from the converted data. This makes it possible to accurately understand the content of the contract information and effectively identify risky clauses.

[0241] "Information acquisition means" refers to means that have the function of receiving contract information as input from the user.

[0242] "Data manipulation means" refers to means of processing acquired contract information to convert it into an analyzable format.

[0243] A "data analysis tool" is a tool equipped with the function of extracting and analyzing text from converted data.

[0244] An "item identification means" is a means that has the function of identifying clauses that require attention or content that poses a risk from the extracted text data.

[0245] A "generative AI model" refers to artificial intelligence technology that uses previously learned data to identify important patterns and specific items from target data.

[0246] A "risk assessment tool" is a tool that has the function of evaluating the risk level of an identified clause and generating the result.

[0247] An "information generation means" is a means equipped with the function of generating feedback to be provided to the user based on the results of a risk assessment.

[0248] An "information presentation tool" is a means that visually presents generated feedback to the user and has the function of aiding understanding.

[0249] This invention is a system for providing contract information in a format that is easy for users to understand, and it operates primarily with three elements: a server, a terminal, and a user.

[0250] The user provides contract and terms of service information using their device. This information can be uploaded as an image file, selected as a text file, or entered as a URL to the webpage containing the contract. The device's information retrieval system accepts this input.

[0251] The received information is transmitted from the terminal to the server. The server uses data manipulation tools to convert the received contract information into an appropriate format. For example, if it is image data, optical character recognition (OCR) is used to extract the text. The converted text data is then analyzed in detail by data analysis tools, and item identification tools are used to identify important clauses and risky content. Generative AI models are used in this process to accurately identify complex legal terminology and clauses that may affect end users.

[0252] Next, the server's risk assessment means evaluates the risk level based on the identified information, and the information generation means creates feedback for the user. The generated feedback is sent to the terminal through the information presentation means and displayed to the user with visual emphasis. For example, high-risk clauses are highlighted with color-coded icons to draw the user's attention.

[0253] A concrete example is a situation where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement page into their device. The server analyzes the page and generates warnings about automatic renewal clauses for subscription services and high penalties. By reviewing this information and reconsidering the contract before signing, the user can avoid disadvantages and make a safe and wise decision.

[0254] An example of a prompt message would be: "Please identify and visually highlight any risky clauses in the following contract. In particular, please provide details regarding the automatic renewal clause and penalties."

[0255] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0256] Step 1:

[0257] The user enters contract information through a terminal. This information is provided as an image file, text file, or webpage URL. The terminal's information acquisition mechanism receives this input and passes the contract information to the system as data. In this step, the input is the contract information provided by the user, and the output is the contract information as it has been incorporated into the system.

[0258] Step 2:

[0259] The terminal sends the captured contract information to the server. The server uses data manipulation tools to perform a conversion process: if the input information is image data, it extracts text using OCR technology; if it is text data, it sends it directly for analysis. The input for this step is the raw contract information provided by the user, and the output is the contract information converted into text data.

[0260] Step 3:

[0261] The server receives text data and performs a detailed analysis using data analysis tools. Specifically, it processes the data to identify important clauses and risky items within the contract. In this process, a generative AI model uses pre-trained data to identify clauses that require attention. The input for this step is contract information converted into text data, and the output is the identified important clauses and risk information.

[0262] Step 4:

[0263] The server's risk assessment means evaluates the risk level based on the identified clauses. Based on the results of the risk assessment, the information generation means generates feedback. This feedback includes detailed information and recommended actions regarding the clauses that the user should pay attention to. The input to this step is the identified risk information, and the output is the materialized feedback.

[0264] Step 5:

[0265] The server-generated feedback is sent to the terminal via an information display device and presented to the user. The terminal visually highlights the feedback, drawing the user's attention by using colors and icons to highlight particularly high-risk clauses. The user reviews the provided information and makes a final decision. The input for this step is the generated feedback, and the output is the visually highlighted feedback received by the user.

[0266] (Application Example 1)

[0267] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0268] In the process of understanding contracts and terms of service, users often overlook complex clauses or unfavorable conditions, resulting in the risk of entering into unfavorable agreements. In particular, electronic payment services may contain contract terms that require careful attention, and users often find it difficult to accurately understand these terms. This invention aims to help users easily understand the content of contract information, identify risky areas, and make informed decisions.

[0269] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0270] In this invention, the server includes receiving means for inputting contract information, conversion means for converting the contract information into a corresponding data format if it is image or document data, and analysis means for extracting text from the converted data. This allows users to easily understand risky contract terms by taking a picture of or uploading a contract or terms of service with a smart device, and the system analyzing the content and visually highlighting areas that require attention.

[0271] "Contract information" refers to data that includes the contents of terms of service, contracts, and other similar documents. This information should be provided in a format that is easy for users to understand.

[0272] "Acceptance method" refers to the function for obtaining contract information from a terminal. It is the first step in the user entering contract data into the system.

[0273] "Conversion means" refers to the function of converting contract information in image or document data format into an appropriate data format. This is the process of making text information extractable using OCR technology, etc.

[0274] "Analysis means" refers to the function that extracts text from the converted data and analyzes the contract content. It plays a role in preparing the information for further processing.

[0275] "Identification means" refers to a function that identifies areas requiring attention based on analyzed text data. This is necessary to identify risky clauses and warn users.

[0276] The "evaluation method" refers to a function that assesses the risk of identified areas and generates evaluation results. This provides users with information-based feedback.

[0277] "Generation method" refers to the function that generates feedback for the user based on the evaluation results. It is a process for providing information in a way that helps the user understand, such as through visual highlighting.

[0278] "Output mechanism" refers to a function that provides generated feedback to the user. It effectively presents information to help the user understand the contract terms.

[0279] To implement this invention, the system employs a configuration combining multiple hardware and software components. The user inputs contract information in image or text format using a smart device. The acquired contract information is transmitted to the system via a receiving means, where it undergoes a process of converting the data into text using OCR technology. This process utilizes an OCR library such as pytesseract.

[0280] The converted text is sent to a server and analyzed using natural language processing (NLP) technology. This analysis is performed using a NLP library such as spaCy to identify risky clauses and specialized legal terminology. The identified risky sections are evaluated in detail using evaluation tools to generate appropriate feedback for the user. The evaluation results are processed using generation tools for visual highlighting on smart devices.

[0281] For example, consider the case where a user is trying to sign up for a new electronic payment service. When the user takes a photo of the contract and uploads the image to the system, the system identifies automatic debit conditions and high penalty clauses for breach of contract and warns the user. This allows the user to check for unfavorable contract conditions in advance and make a decision.

[0282] As an example of a prompt sentence to input into the generative AI model, there is "Teach me how to develop an app that highlights the risk clauses in a contract." With this prompt, it is possible to learn a method for specifically identifying risky contract conditions.

[0283] The flow of the specific processing in Application Example 1 will be described using FIG. 12.

[0284] Step 1:

[0285] The user uses the terminal to input contract information as an image or URL. This input is taken into the system by the receiving means. The data to be input is a photo of the contract or the URL of a web page. The terminal prepares to send this data to the server.

[0286] Step 2:

[0287] First, the server converts the received image data into text data in order to analyze the received data. At this time, OCR technology such as pytesseract is used. By extracting text from the input image data, the contract content can be treated as character information. The extracted text is output.

[0288] Step 3:

[0289] The server uses natural language processing technology to evaluate the data in order to analyze the converted text. In this process, libraries such as spaCy are used. From the text data received as input, risky clauses and technical terms are identified, and points to be noted are identified. The identified risky clauses are output.

[0290] Step 4:

[0291] The server evaluates the identified risk areas and generates feedback based on the results. It uses evaluation tools to generate evaluation results and organizes information to determine the risks to the user. Detailed risk assessment information is output as feedback.

[0292] Step 5:

[0293] The terminal presents the generated feedback to the user. It processes the feedback to visually highlight risky areas and displays them on the terminal screen using a generation method. The user can then review this feedback to deepen their understanding of the contract and make a final decision. The output is the visually processed feedback.

[0294] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0295] This invention is a system that allows users to easily understand information related to contracts and terms of service, and adjusts feedback while also considering the user's emotional state at the time. In addition to the basic functions of inputting and analyzing contract information, this system also includes a function that recognizes the user's emotions using an emotion engine and generates feedback based on those emotions.

[0296] First, the user uses their device to input contract and terms of service information as images or URLs. The device sends this contract information to the server, which extracts the text data using OCR technology. Next, the server analyzes the text data using natural language processing to identify contract clauses that require attention.

[0297] This process also incorporates an emotion engine, which evaluates the user's emotions in response to the data extracted by the analysis tools. This system uses non-verbal data obtained from the user via the device, such as facial recognition and voice data, to determine emotions. If the user is feeling anxious, the emotion engine takes that emotion into account and adjusts the feedback to be more detailed or highlights important points.

[0298] The generation mechanism prepares user-optimized feedback based on emotional data provided by the emotion engine and risk information identified by the identification mechanism. This feedback visually displays the contract's risk level and is presented in a format that is easily understandable to the user.

[0299] For example, when a user is checking the terms and conditions of a new online service, if the system detects the user's unease, it will provide a detailed explanation of any special clauses or points to note regarding automatic renewal included in the contract, and offer specific suggestions to help the user make a decision with confidence. In this way, users can gain a deep understanding of the contract and make decisions with confidence.

[0300] Thus, as an interactive system that combines emotion recognition technology, the present invention provides users with better support for making contractual decisions.

[0301] The following describes the processing flow.

[0302] Step 1:

[0303] The user enters specific contract information on their device. Input methods include taking a picture, selecting an existing file, or directly entering a URL.

[0304] Step 2:

[0305] The terminal sends the input contract information to the server. The image data is converted into Base64 format, and the text data and URL are sent in JSON format.

[0306] Step 3:

[0307] The server analyzes the received contract information. If image data is sent, the server uses OCR to extract text from the image. In the case of a URL, the page content is obtained in text form via web scraping.

[0308] Step 4:

[0309] The server uses analysis means to analyze the text data through natural language processing and identify the content that needs attention from the terms. Based on this analysis result, the risk points of the contract are confirmed.

[0310] Step 5:

[0311] The emotion engine activates and conducts non-verbal information obtained from the user's terminal, such as facial expression analysis through the camera and voice tone analysis by the microphone. Based on this information, the user's emotional state is evaluated.

[0312] Step 6:

[0313] The server combines the identified risk information and the results of the emotion engine to generate feedback provided to the user. This feedback is adjusted according to the user's emotional state. For example, when the user is feeling anxious, the feedback is adjusted to be more detailed and reassuring.

[0314] Step 7:

[0315] The terminal receives the feedback sent from the server and presents it in a visual and user-friendly format. Thereby, the risk level of the contract is highlighted and the user can intuitively understand it.

[0316] (Example 2)

[0317] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0318] Conventional contract information analysis systems have faced challenges in providing optimal feedback to users because it is difficult for them to fully understand the details of contracts and terms of service, and because the user's emotional state is not taken into consideration. In particular, when users are feeling anxious, they may overlook important information, so there is a need to provide contract information that takes the user's emotional state into account.

[0319] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0320] In this invention, the server includes a function for inputting contract information, a function for converting the contract information into a corresponding data format, and a function for detecting areas of interest based on the extracted text. This makes it possible to appropriately analyze the contract information of interest while taking into account the user's emotional state and provide the user with optimal feedback.

[0321] "Contract information" refers to data contained in contracts and terms of service, and is the information necessary for a user to consent to a particular service or product.

[0322] The "data format conversion function" refers to a function that converts the entered contract information into a parseable format, including the process of converting image and PDF data into text data.

[0323] The "text extraction function" refers to the process of extracting character information from the converted data and obtaining text data that will serve as the basis for analysis.

[0324] The "function to detect areas requiring attention" refers to the process of identifying information that is important to the user or content that may contain potential risks from the extracted text data.

[0325] The "risk assessment function" is a process that evaluates potential risks from detected information and determines their importance and impact on users.

[0326] The "function for evaluating emotions" refers to the process of analyzing non-verbal data such as the user's facial expressions and voice to identify the user's psychological state and emotions.

[0327] The "feedback generation function" is the process of constructing information to provide to the user based on assessed risk and sentiment data, and includes formats such as text and visual displays.

[0328] "Output function" refers to a function that presents the generated feedback in a format that is easy for the user to understand, and includes displaying it on the device.

[0329] This invention relates to a system that easily analyzes contract information and presents its contents in a format that is easy for users to understand. Users input contract and terms of service information using a terminal in image or URL format. The terminal temporarily stores this information and sends it to a server.

[0330] The server performs analysis on the received information. First, it uses OCR technology to convert image data into text data. Specifically, general OCR software is used here. Next, it uses a generative AI model to process the extracted text data and identify clauses that require attention or items that carry risks. For example, natural language processing toolkits and machine learning algorithms are utilized.

[0331] In addition, the server analyzes non-verbal data obtained from the user's terminal. By acquiring the user's facial expressions and voice and performing emotion analysis, it determines the user's emotional state. Software tools are used as emotion recognition technology in this process.

[0332] The server integrates analyzed contract information with data on the user's emotional state and generates feedback based on this. This feedback is presented to the user in the form of visualized risk levels and highlighting of important clauses. For example, if the system detects a user's anxious expression while reviewing an online service contract, it will provide detailed explanations regarding special clauses and automatic renewal considerations. An example of a prompt related to this operation might be, "Please provide a clear explanation of the contract clauses the user is concerned about," which could be input to the generating AI model.

[0333] In this way, by incorporating emotion recognition technology, users can gain a deeper understanding of the contract terms and create an environment where they can make decisions with confidence. The system as a whole provides an interactive analytical tool that combines detailed contract analysis with an evaluation of the user's emotional state.

[0334] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0335] Step 1:

[0336] The user enters contract information in image or URL format using a terminal. This becomes the initial input data for the system. The terminal temporarily stores this information and prepares to send it to the server. At this time, the input is completed via the interface on the user's terminal, and the destination server IP address is set.

[0337] Step 2:

[0338] The terminal sends the saved contract information to the server. The server receives the input data and converts it into a format for text analysis. Here, the server uses OCR technology to convert image files into text data. Specifically, it performs calculations to recognize characters from the image and convert them into digital text. The output of this process is text data containing the contract information.

[0339] Step 3:

[0340] The server feeds the text data extracted by OCR into a natural language processing engine. A generative AI model analyzes this text data to determine whether there are any clauses that require attention or any particular risks. Specifically, the generative AI model analyzes the text and performs data calculations to extract keywords and expressions related to risk. The output at this stage is a list of identified important clauses.

[0341] Step 4:

[0342] Simultaneously, the device collects nonverbal data such as the user's facial expressions and voice, and sends it to the server. The server analyzes this data using an emotion recognition algorithm to evaluate the user's emotional state. Here, the server analyzes the user's nonverbal signs and determines the emotion they indicate (e.g., anxiety, confusion). The output identifies the user's emotional state.

[0343] Step 5:

[0344] The server generates optimal feedback based on the key contract clauses identified by the generation AI model and the user's assessed emotional state. Feedback generation involves data processing to visually indicate the contract's risk level and highlight points the user should pay attention to. The generated feedback is then output and ready to use.

[0345] Step 6:

[0346] The terminal receives feedback information sent from the server and presents it to the user. Specifically, a well-formatted dashboard is displayed on the terminal, designed to make it easy for the user to understand the contract information. In this process, the output is provided in the form of visualized feedback for the user.

[0347] (Application Example 2)

[0348] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0349] Contract information and terms of service are often difficult for many users to understand, and as a result, they may accept contracts without fully understanding their contents. This situation can lead to disadvantages later on, so there is a need for a system that supports users in understanding contract contents and using them with peace of mind.

[0350] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0351] In this invention, the server includes a device for inputting contract information, a conversion device for converting the contract information into an appropriate information format if it is image or document information, and an analysis device for extracting text information from the converted information. This makes it possible to detect important points in the contract content and analyze the user's emotions, thereby providing more understandable and reliable feedback.

[0352] A "device for inputting contract information" is a device that has an interface for users to provide contract information to a system.

[0353] A "conversion device" is a device equipped with the function of converting received contract information into a format that is easy for the system to analyze.

[0354] An "analysis device" is a device that extracts necessary textual information from converted information and analyzes the contract details.

[0355] An "identification device" is a device that has the function of identifying areas of interest within contract information based on analyzed textual information.

[0356] An "evaluation device" is a device that evaluates the risk level of a contract based on identified sections and generates the evaluation results.

[0357] An "emotion analysis device" is a device that uses nonverbal data obtained from a user to analyze their emotions.

[0358] A "generating device" is a device equipped with the function of generating information to be provided to the user based on evaluation results and sentiment analysis results.

[0359] An "output device" is a device that visually displays the generated feedback to the user, highlighting and providing information.

[0360] In embodiments of this invention, the system includes means for effectively analyzing contract information and providing optimized feedback to the user. The specific configuration and operation are described below.

[0361] When a user first enters their contract information, they use a device such as a smartphone or tablet. The device receives the contract information by either taking a picture of the contract using its camera or by entering a URL, and then sends it to the server.

[0362] The server uses Tesseract OCR to extract text data from these image data. The extracted text is then analyzed using a natural language processing library such as spaCy to identify important clauses and risks within the contract information.

[0363] Furthermore, using OpenCV-based facial recognition technology and TensorFlow's emotion analysis model, the system analyzes the user's emotions from their facial expressions and voice. This allows the system to understand the user's level of anxiety and adjust the content and emphasis of the feedback accordingly.

[0364] The generated feedback is provided to the user through an intuitive interface developed using Flutter. This feedback highlights points of particular importance and presents them in a visually clear manner, allowing users to better understand the contract and make informed decisions.

[0365] As a concrete example, when a user registers for a new mobile payment service, this system allows them to take a picture of the terms of service displayed during the registration process. The system detects anxiety from the user's facial expressions and provides feedback that explains the terms in detail, especially those concerning automatic renewal and fees.

[0366] An example of a prompt for the generative AI model regarding the use of this system is: "If a user is reading a new service agreement and feels uneasy, how would you use AI to adjust the feedback? As a specific example, consider an application that combines facial recognition and natural language processing."

[0367] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0368] Step 1:

[0369] The user enters contract information. Using a smartphone or tablet, the contract information is collected on the device by taking a picture of the contract or entering a URL. At this stage, the input data is either an image or a URL.

[0370] Step 2:

[0371] The terminal sends the collected contract information to the server. The server uses Tesseract OCR to extract text data from the image data. This process takes image data as input and outputs text data.

[0372] Step 3:

[0373] The server analyzes the extracted text data. Using natural language processing libraries such as spaCy, it identifies important clauses and risks. The input here is text data, and the output is risk information.

[0374] Step 4:

[0375] The server analyzes the user's emotions. It uses OpenCV for facial recognition and a TensorFlow model to determine the emotion. Here, non-verbal data (images and audio) acquired by the camera is used as input data, and the output is the user's emotional state.

[0376] Step 5:

[0377] The server generates feedback based on the evaluation results and sentiment data. The content of the feedback is adjusted according to the evaluated risk information and the user's sentiment. The inputs to this process are risk information and sentiment data, and the output is feedback data.

[0378] Step 6:

[0379] The device provides the user with generated feedback. This is visually displayed through a Flutter-based UI, highlighting points that require particular attention. The input is feedback data, and the output is visual information presented to the user.

[0380] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0381] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0382] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0383] [Third Embodiment]

[0384] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0385] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0386] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0387] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0388] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0389] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0390] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0391] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0392] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0393] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0394] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0395] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0396] This invention provides a system that helps users easily understand contract information and avoid signing risky contracts. The system includes a series of processes, starting with the user inputting contracts or terms of service, effectively analyzing that information, and providing feedback.

[0397] First, the user uses a terminal to input contract information as an image or text file, or provides the system with a web page URL. This contract information is then acquired by the receiving device. The terminal then sends the contract information to the server, which converts the received data into an appropriate data format. If it is image data, text is extracted using OCR technology; if it is text data, it proceeds directly to the analysis process.

[0398] The server examines the text data extracted by the analysis tool using an identification tool to find clauses that require attention. Natural language processing technology is used in this process to identify risky clauses and specialized legal terms. Subsequently, the evaluation tool performs a risk analysis on these clauses and generates feedback that suggests some action to the user.

[0399] The feedback generated by the generation mechanism is then sent to the terminal by the output mechanism and presented to the user. This feedback visually highlights risky clauses and is displayed in an easy-to-understand format. This allows the user to fully understand the contents of the contract or terms of service and make important decisions before signing.

[0400] As a concrete example, consider a scenario where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement's webpage into this system. The server analyzes the page and generates warnings about automatic renewal clauses and hefty penalties for early termination. Based on this information, the user can reconfirm the terms before signing. This allows the user to avoid disadvantages and make safe and informed decisions.

[0401] The following describes the processing flow.

[0402] Step 1:

[0403] The user enters contract information from their device. This is done by taking a picture, selecting an existing file, or entering a web page URL.

[0404] Step 2:

[0405] The terminal sends the entered contract information to the server. Images are encoded in Base64 format, and URLs are sent as JSON data.

[0406] Step 3:

[0407] The server processes the received contract information appropriately. In the case of images, text is extracted from the image through OCR processing. In the case of URLs, web scraping technology is used to obtain the page content as text data.

[0408] Step 4:

[0409] The server analyzes the text data extracted by the analysis tools using natural language processing. This identifies risky clauses and important contractual terms.

[0410] Step 5:

[0411] The server evaluates the risk level based on the analyzed data using evaluation tools. It then generates evaluation results that form the basis for feedback.

[0412] Step 6:

[0413] The generation mechanism creates a user feedback message based on these evaluation results. This message highlights and explains risky clauses and suggests necessary actions for the user.

[0414] Step 7:

[0415] The feedback is sent to the terminal via an output device. The terminal visually displays the feedback to the user, highlighting important elements of the contract.

[0416] (Example 1)

[0417] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0418] When users try to understand contracts and terms of service, they may overlook technical terms or risky clauses, which poses a risk of misunderstanding and disadvantage. This makes it difficult to make safe and informed decisions.

[0419] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0420] In this invention, the server includes information acquisition means for inputting contract information, data manipulation means for converting the contract information into a data format, and data analysis means for extracting text from the converted data. This makes it possible to accurately understand the content of the contract information and effectively identify risky clauses.

[0421] "Information acquisition means" refers to means that have the function of receiving contract information as input from the user.

[0422] "Data manipulation means" refers to means of processing acquired contract information to convert it into an analyzable format.

[0423] A "data analysis tool" is a tool equipped with the function of extracting and analyzing text from converted data.

[0424] An "item identification means" is a means that has the function of identifying clauses that require attention or content that poses a risk from the extracted text data.

[0425] A "generative AI model" refers to artificial intelligence technology that uses previously learned data to identify important patterns and specific items from target data.

[0426] A "risk assessment tool" is a tool that has the function of evaluating the risk level of an identified clause and generating the result.

[0427] An "information generation means" is a means equipped with the function of generating feedback to be provided to the user based on the results of a risk assessment.

[0428] An "information presentation tool" is a means that has the function of visually presenting generated feedback to the user and aiding in their understanding.

[0429] This invention is a system for providing contract information in a format that is easy for users to understand, and it operates primarily with three elements: a server, a terminal, and a user.

[0430] The user provides contract and terms of service information using their device. This information can be provided as an image file, selected as a text file, or by entering the URL of the webpage containing the contract. The information retrieval system on the device accepts this input.

[0431] The received information is transmitted from the terminal to the server. The server uses data manipulation tools to convert the received contract information into an appropriate format. For example, if it is image data, optical character recognition (OCR) is used to extract the text. The converted text data is then analyzed in detail by data analysis tools, and item identification tools are used to identify important clauses and risky content. Generative AI models are used in this process to accurately identify complex legal terminology and clauses that may affect end users.

[0432] Next, the server's risk assessment means evaluates the risk level based on the identified information, and the information generation means creates feedback for the user. The generated feedback is sent to the terminal through the information presentation means and displayed to the user with visual emphasis. For example, high-risk clauses are highlighted with color-coded icons to draw the user's attention.

[0433] A concrete example is a situation where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement page into their device. The server analyzes the page and generates warnings about automatic renewal clauses for subscription services and high penalties. By reviewing this information and reconsidering the contract before signing, the user can avoid disadvantages and make a safe and wise decision.

[0434] An example of a prompt message would be: "Please identify and visually highlight any risky clauses in the following contract. In particular, please provide details regarding the automatic renewal clause and penalties."

[0435] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0436] Step 1:

[0437] The user enters contract information through a terminal. This information is provided as an image file, text file, or webpage URL. The terminal's information acquisition mechanism receives this input and passes the contract information to the system as data. In this step, the input is the contract information provided by the user, and the output is the contract information as it has been incorporated into the system.

[0438] Step 2:

[0439] The terminal sends the captured contract information to the server. The server uses data manipulation tools to perform a conversion process: if the input information is image data, it extracts text using OCR technology; if it is text data, it sends it directly for analysis. The input for this step is the raw contract information provided by the user, and the output is the contract information converted into text data.

[0440] Step 3:

[0441] The server receives text data and performs a detailed analysis using data analysis tools. Specifically, it processes the data to identify important clauses and risky items within the contract. In this process, a generative AI model uses pre-trained data to identify clauses that require attention. The input for this step is contract information converted into text data, and the output is the identified important clauses and risk information.

[0442] Step 4:

[0443] The server's risk assessment means evaluates the risk level based on the identified clauses. Based on the results of the risk assessment, the information generation means generates feedback. This feedback includes detailed information and recommended actions regarding the clauses that the user should pay attention to. The input to this step is the identified risk information, and the output is the materialized feedback.

[0444] Step 5:

[0445] The server-generated feedback is sent to the terminal via an information display device and presented to the user. The terminal visually highlights the feedback, drawing the user's attention by using colors and icons to highlight particularly high-risk clauses. The user reviews the provided information and makes a final decision. The input for this step is the generated feedback, and the output is the visually highlighted feedback received by the user.

[0446] (Application Example 1)

[0447] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0448] In the process of understanding contracts and terms of service, users often overlook complex clauses or unfavorable conditions, resulting in the risk of entering into unfavorable agreements. In particular, electronic payment services may contain contract terms that require careful attention, and users often find it difficult to accurately understand these terms. This invention aims to help users easily understand the content of contract information, identify risky areas, and make informed decisions.

[0449] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0450] In this invention, the server includes receiving means for inputting contract information, conversion means for converting the contract information into a corresponding data format if it is image or document data, and analysis means for extracting text from the converted data. This allows users to easily understand risky contract terms by taking a picture of or uploading a contract or terms of service with a smart device, and the system analyzing the content and visually highlighting areas that require attention.

[0451] "Contract information" refers to data that includes the contents of terms of service, contracts, and other similar documents. This information should be provided in a format that is easy for users to understand.

[0452] "Acceptance method" refers to the function for obtaining contract information from a terminal. It is the first step in the user entering contract data into the system.

[0453] "Conversion means" refers to the function of converting contract information in image or document data format into an appropriate data format. This is the process of making text information extractable using OCR technology, etc.

[0454] "Analysis means" refers to the function that extracts text from the converted data and analyzes the contract content. It plays a role in preparing the information for further processing.

[0455] "Identification means" refers to a function that identifies areas requiring attention based on analyzed text data. This is necessary to identify risky clauses and warn users.

[0456] The "evaluation method" refers to a function that assesses the risk of identified areas and generates evaluation results. This provides users with information-based feedback.

[0457] "Generation method" refers to the function that generates feedback for the user based on the evaluation results. It is a process for providing information in a way that helps the user understand, such as through visual highlighting.

[0458] "Output mechanism" refers to a function that provides generated feedback to the user. It effectively presents information to help the user understand the contract terms.

[0459] To implement this invention, the system employs a configuration combining multiple hardware and software components. The user inputs contract information in image or text format using a smart device. The acquired contract information is transmitted to the system via a receiving means, where it undergoes a process of converting the data into text using OCR technology. This process utilizes an OCR library such as pytesseract.

[0460] The converted text is sent to a server and analyzed using natural language processing (NLP) technology. This analysis is performed using a NLP library such as spaCy to identify risky clauses and specialized legal terminology. The identified risky sections are evaluated in detail using evaluation tools to generate appropriate feedback for the user. The evaluation results are processed using generation tools for visual highlighting on smart devices.

[0461] For example, consider a scenario where a user is about to sign a contract for a new electronic payment service. If the user takes a photo of the contract and uploads the image to the system, the system can identify automatic withdrawal conditions and high penalty clauses, and warn the user. This allows the user to review unfavorable contract terms in advance and make an informed decision.

[0462] An example of a prompt to input into a generative AI model is, "Teach me how to develop an app that highlights risk clauses in contracts." This prompt allows the model to learn techniques for specifically identifying risky contract terms.

[0463] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0464] Step 1:

[0465] The user uses a terminal to input contract information as an image or URL. This input is received by the system via a receiving mechanism. The input data includes photos of contracts and URLs of web pages. The terminal prepares to send this data to the server.

[0466] Step 2:

[0467] To analyze the received data, the server first converts the image data into text data. This process utilizes OCR technology such as pytesseract. By extracting text from the input image data, the contract details can be treated as textual information. The extracted text is then output.

[0468] Step 3:

[0469] The server uses natural language processing techniques to evaluate the data and analyze the converted text. This process utilizes libraries such as spaCy. From the text data received as input, it identifies risky clauses and technical terms, and identifies points that require attention. The identified risky clauses are then output.

[0470] Step 4:

[0471] The server evaluates the identified risk areas and generates feedback based on the results. It uses evaluation tools to generate evaluation results and organizes information to determine the risks to the user. Detailed risk assessment information is output as feedback.

[0472] Step 5:

[0473] The terminal presents the generated feedback to the user. It processes the feedback to visually highlight risky areas and displays them on the terminal screen using a generation method. The user can then review this feedback to deepen their understanding of the contract and make a final decision. The output is the visually processed feedback.

[0474] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0475] This invention is a system that allows users to easily understand information related to contracts and terms of service, and adjusts feedback while also considering the user's emotional state at the time. In addition to the basic functions of inputting and analyzing contract information, this system also includes a function that recognizes the user's emotions using an emotion engine and generates feedback based on those emotions.

[0476] First, the user uses their device to input contract and terms of service information as images or URLs. The device sends this contract information to the server, which extracts the text data using OCR technology. Next, the server analyzes the text data using natural language processing to identify contract clauses that require attention.

[0477] This process also incorporates an emotion engine, which evaluates the user's emotions in response to the data extracted by the analysis tools. This system uses non-verbal data obtained from the user via the device, such as facial recognition and voice data, to determine emotions. If the user is feeling anxious, the emotion engine takes that emotion into account and adjusts the feedback to be more detailed or highlights important points.

[0478] The generation mechanism prepares user-optimized feedback based on emotional data provided by the emotion engine and risk information identified by the identification mechanism. This feedback visually displays the contract's risk level and is presented in a format that is easily understandable to the user.

[0479] For example, when a user is checking the terms and conditions of a new online service, if the system detects the user's unease, it will provide a detailed explanation of any special clauses or points to note regarding automatic renewal included in the contract, and offer specific suggestions to help the user make a decision with confidence. In this way, users can gain a deep understanding of the contract and make decisions with confidence.

[0480] Thus, as an interactive system that combines emotion recognition technology, the present invention provides users with better support for making contractual decisions.

[0481] The following describes the processing flow.

[0482] Step 1:

[0483] The user enters specific contract information on their device. Input methods include taking a picture, selecting an existing file, or directly entering a URL.

[0484] Step 2:

[0485] The terminal sends the entered contract information to the server. Image data is converted to Base64 format, and text data and URLs are sent in JSON format.

[0486] Step 3:

[0487] The server analyzes the received contract information. If image data is sent, the server uses OCR to extract text from the image. In the case of a URL, the page content is obtained in text format via web scraping.

[0488] Step 4:

[0489] The server uses analysis tools to analyze text data using natural language processing and identify clauses that require attention. Based on these analysis results, the risk points of the contract are identified.

[0490] Step 5:

[0491] The emotion engine activates and performs non-verbal information acquisition from the user's device, such as facial expression analysis via the camera and voice tone analysis via the microphone. Based on this information, the user's emotional state is evaluated.

[0492] Step 6:

[0493] The server combines identified risk information with the results of the emotion engine to generate feedback for the user. This feedback is tailored to the user's emotional state. For example, if the user is feeling anxious, the feedback will be more detailed and reassuring.

[0494] Step 7:

[0495] The device receives feedback sent from the server and presents it in a visual and user-friendly format. This highlights the contract's risk level, making it intuitively understandable to the user.

[0496] (Example 2)

[0497] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0498] Conventional contract information analysis systems have faced challenges in providing optimal feedback to users because it is difficult for them to fully understand the details of contracts and terms of service, and because the user's emotional state is not taken into consideration. In particular, when users are feeling anxious, they may overlook important information, so there is a need to provide contract information that takes the user's emotional state into account.

[0499] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0500] In this invention, the server includes a function for inputting contract information, a function for converting the contract information into a corresponding data format, and a function for detecting areas of interest based on the extracted text. This makes it possible to appropriately analyze the contract information of interest while taking into account the user's emotional state and provide the user with optimal feedback.

[0501] "Contract information" refers to data contained in contracts and terms of service, and is the information necessary for a user to consent to a particular service or product.

[0502] The "data format conversion function" refers to a function that converts the entered contract information into a parseable format, including the process of converting image and PDF data into text data.

[0503] The "text extraction function" refers to the process of extracting character information from the converted data and obtaining text data that will serve as the basis for analysis.

[0504] The "function to detect areas requiring attention" refers to the process of identifying information that is important to the user or content that may contain potential risks from the extracted text data.

[0505] The "risk assessment function" is a process that evaluates potential risks from detected information and determines their importance and impact on users.

[0506] The "function for evaluating emotions" refers to the process of analyzing non-verbal data such as the user's facial expressions and voice to identify the user's psychological state and emotions.

[0507] The "feedback generation function" is the process of constructing information to provide to the user based on assessed risk and sentiment data, and includes formats such as text and visual displays.

[0508] "Output function" refers to a function that presents the generated feedback in a format that is easy for the user to understand, and includes displaying it on the device.

[0509] This invention relates to a system that easily analyzes contract information and presents its contents in a format that is easy for users to understand. Users input contract and terms of service information using a terminal in image or URL format. The terminal temporarily stores this information and sends it to a server.

[0510] The server performs analysis on the received information. First, it uses OCR technology to convert image data into text data. Specifically, general OCR software is used here. Next, it uses a generative AI model to process the extracted text data and identify clauses that require attention or items that carry risks. For example, natural language processing toolkits and machine learning algorithms are utilized.

[0511] In addition, the server analyzes non-verbal data obtained from the user's terminal. By acquiring the user's facial expressions and voice and performing emotion analysis, it determines the user's emotional state. Software tools are used as emotion recognition technology in this process.

[0512] The server integrates analyzed contract information with data on the user's emotional state and generates feedback based on this. This feedback is presented to the user in the form of visualized risk levels and highlighting of important clauses. For example, if the system detects a user's anxious expression while reviewing an online service contract, it will provide detailed explanations regarding special clauses and automatic renewal considerations. An example of a prompt related to this operation might be, "Please provide a clear explanation of the contract clauses the user is concerned about," which could be input to the generating AI model.

[0513] In this way, by incorporating emotion recognition technology, users can gain a deeper understanding of the contract terms and create an environment where they can make decisions with confidence. The system as a whole provides an interactive analytical tool that combines detailed contract analysis with an evaluation of the user's emotional state.

[0514] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0515] Step 1:

[0516] The user enters contract information in image or URL format using a terminal. This becomes the initial input data for the system. The terminal temporarily stores this information and prepares to send it to the server. At this time, the input is completed via the interface on the user's terminal, and the destination server IP address is set.

[0517] Step 2:

[0518] The terminal sends the saved contract information to the server. The server receives the input data and converts it into a format for text analysis. Here, the server uses OCR technology to convert image files into text data. Specifically, it performs calculations to recognize characters from the image and convert them into digital text. The output of this process is text data containing the contract information.

[0519] Step 3:

[0520] The server feeds the text data extracted by OCR into a natural language processing engine. A generative AI model analyzes this text data to determine whether there are any clauses that require attention or any particular risks. Specifically, the generative AI model analyzes the text and performs data calculations to extract keywords and expressions related to risk. The output at this stage is a list of identified important clauses.

[0521] Step 4:

[0522] Simultaneously, the device collects nonverbal data such as the user's facial expressions and voice, and sends it to the server. The server analyzes this data using an emotion recognition algorithm to evaluate the user's emotional state. Here, the server analyzes the user's nonverbal signs and determines the emotion they indicate (e.g., anxiety, confusion). The output identifies the user's emotional state.

[0523] Step 5:

[0524] The server generates optimal feedback based on the key contract clauses identified by the generation AI model and the user's assessed emotional state. Feedback generation involves data processing to visually indicate the contract's risk level and highlight points the user should pay attention to. The generated feedback is then output and ready to use.

[0525] Step 6:

[0526] The terminal receives feedback information sent from the server and presents it to the user. Specifically, a well-formatted dashboard is displayed on the terminal, designed to make it easy for the user to understand the contract information. In this process, the output is provided in the form of visualized feedback for the user.

[0527] (Application Example 2)

[0528] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0529] Contract information and terms of service are often difficult for many users to understand, and as a result, they may accept contracts without fully understanding their contents. This situation can lead to disadvantages later on, so there is a need for a system that supports users in understanding contract contents and using them with peace of mind.

[0530] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0531] In this invention, the server includes a device for inputting contract information, a conversion device for converting the contract information into an appropriate information format if it is image or document information, and an analysis device for extracting text information from the converted information. This makes it possible to detect important points in the contract content and analyze the user's emotions, thereby providing more understandable and reliable feedback.

[0532] A "device for inputting contract information" is a device that has an interface for users to provide contract information to a system.

[0533] A "conversion device" is a device equipped with the function of converting received contract information into a format that is easy for the system to analyze.

[0534] An "analysis device" is a device that extracts necessary textual information from converted information and analyzes the contract details.

[0535] An "identification device" is a device that has the function of identifying areas of interest within contract information based on analyzed textual information.

[0536] An "evaluation device" is a device that evaluates the risk level of a contract based on identified sections and generates the evaluation results.

[0537] An "emotion analysis device" is a device that uses nonverbal data obtained from a user to analyze their emotions.

[0538] A "generating device" is a device equipped with the function of generating information to be provided to the user based on evaluation results and sentiment analysis results.

[0539] An "output device" is a device that visually displays the generated feedback to the user, highlighting and providing information.

[0540] In embodiments of this invention, the system includes means for effectively analyzing contract information and providing optimized feedback to the user. The specific configuration and operation are described below.

[0541] When a user first enters their contract information, they use a device such as a smartphone or tablet. The device receives the contract information by either taking a picture of the contract using its camera or by entering a URL, and then sends it to the server.

[0542] The server uses Tesseract OCR to extract text data from these image data. The extracted text is then analyzed using a natural language processing library such as spaCy to identify important clauses and risks within the contract information.

[0543] Furthermore, using OpenCV-based facial recognition technology and TensorFlow's emotion analysis model, the system analyzes the user's emotions from their facial expressions and voice. This allows the system to understand the user's level of anxiety and adjust the content and emphasis of the feedback accordingly.

[0544] The generated feedback is provided to the user through an intuitive interface developed using Flutter. This feedback highlights points of particular importance and presents them in a visually clear manner, allowing users to better understand the contract and make informed decisions.

[0545] As a concrete example, when a user registers for a new mobile payment service, this system allows them to take a picture of the terms of service displayed during the registration process. The system detects anxiety from the user's facial expressions and provides feedback that explains the terms in detail, especially those concerning automatic renewal and fees.

[0546] An example of a prompt for the generative AI model regarding the use of this system is: "If a user is reading a new service agreement and feels uneasy, how would you use AI to adjust the feedback? As a specific example, consider an application that combines facial recognition and natural language processing."

[0547] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0548] Step 1:

[0549] The user enters contract information. Using a smartphone or tablet, the contract information is collected on the device by taking a picture of the contract or entering a URL. At this stage, the input data is either an image or a URL.

[0550] Step 2:

[0551] The terminal sends the collected contract information to the server. The server uses Tesseract OCR to extract text data from the image data. This process takes image data as input and outputs text data.

[0552] Step 3:

[0553] The server analyzes the extracted text data. Using natural language processing libraries such as spaCy, it identifies important clauses and risks. The input here is text data, and the output is risk information.

[0554] Step 4:

[0555] The server analyzes the user's emotions. It uses OpenCV for facial recognition and a TensorFlow model to determine the emotion. Here, non-verbal data (images and audio) acquired by the camera is used as input data, and the output is the user's emotional state.

[0556] Step 5:

[0557] The server generates feedback based on the evaluation results and sentiment data. The content of the feedback is adjusted according to the evaluated risk information and the user's sentiment. The inputs to this process are risk information and sentiment data, and the output is feedback data.

[0558] Step 6:

[0559] The device provides the user with generated feedback. This is visually displayed through a Flutter-based UI, highlighting points that require particular attention. The input is feedback data, and the output is visual information presented to the user.

[0560] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0561] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0562] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0563] [Fourth Embodiment]

[0564] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0565] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0566] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0567] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0568] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0569] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0570] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0571] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0572] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0573] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0574] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0575] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0576] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0577] This invention provides a system that helps users easily understand contract information and avoid signing risky contracts. The system includes a series of processes, starting with the user inputting contracts or terms of service, effectively analyzing that information, and providing feedback.

[0578] First, the user uses a terminal to input contract information as an image or text file, or provides the system with a web page URL. This contract information is then acquired by the receiving device. The terminal then sends the contract information to the server, which converts the received data into an appropriate data format. If it is image data, text is extracted using OCR technology; if it is text data, it proceeds directly to the analysis process.

[0579] The server examines the text data extracted by the analysis tool using an identification tool to find clauses that require attention. Natural language processing technology is used in this process to identify risky clauses and specialized legal terms. Subsequently, the evaluation tool performs a risk analysis on these clauses and generates feedback that suggests some action to the user.

[0580] The feedback generated by the generation mechanism is then sent to the terminal by the output mechanism and presented to the user. This feedback visually highlights risky clauses and is displayed in an easy-to-understand format. This allows the user to fully understand the contents of the contract or terms of service and make important decisions before signing.

[0581] As a concrete example, consider a scenario where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement's webpage into this system. The server analyzes the page and generates warnings about automatic renewal clauses and hefty penalties for early termination. Based on this information, the user can reconfirm the terms before signing. This allows the user to avoid disadvantages and make safe and informed decisions.

[0582] The following describes the processing flow.

[0583] Step 1:

[0584] The user enters contract information from their device. This is done by taking a picture, selecting an existing file, or entering a web page URL.

[0585] Step 2:

[0586] The terminal sends the entered contract information to the server. Images are encoded in Base64 format, and URLs are sent as JSON data.

[0587] Step 3:

[0588] The server processes the received contract information appropriately. In the case of images, text is extracted from the image through OCR processing. In the case of URLs, web scraping technology is used to obtain the page content as text data.

[0589] Step 4:

[0590] The server analyzes the text data extracted by the analysis tools using natural language processing. This identifies risky clauses and important contractual terms.

[0591] Step 5:

[0592] The server evaluates the risk level based on the analyzed data using evaluation tools. It then generates evaluation results that form the basis for feedback.

[0593] Step 6:

[0594] The generation mechanism creates a user feedback message based on these evaluation results. This message highlights and explains risky clauses and suggests necessary actions for the user.

[0595] Step 7:

[0596] The feedback is sent to the terminal via an output device. The terminal visually displays the feedback to the user, highlighting important elements of the contract.

[0597] (Example 1)

[0598] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0599] When users try to understand contracts and terms of service, they may overlook technical terms or risky clauses, which poses a risk of misunderstanding and disadvantage. This makes it difficult to make safe and informed decisions.

[0600] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0601] In this invention, the server includes information acquisition means for inputting contract information, data manipulation means for converting the contract information into a data format, and data analysis means for extracting text from the converted data. This makes it possible to accurately understand the content of the contract information and effectively identify risky clauses.

[0602] "Information acquisition means" refers to means that have the function of receiving contract information as input from the user.

[0603] "Data manipulation means" refers to means of processing acquired contract information to convert it into an analyzable format.

[0604] A "data analysis tool" is a tool equipped with the function of extracting and analyzing text from converted data.

[0605] An "item identification means" is a means that has the function of identifying clauses that require attention or content that poses a risk from the extracted text data.

[0606] A "generative AI model" refers to artificial intelligence technology that uses previously learned data to identify important patterns and specific items from target data.

[0607] A "risk assessment tool" is a tool that has the function of evaluating the risk level of an identified clause and generating the result.

[0608] An "information generation means" is a means equipped with the function of generating feedback to be provided to the user based on the results of a risk assessment.

[0609] An "information presentation tool" is a means that has the function of visually presenting generated feedback to the user and aiding in their understanding.

[0610] This invention is a system for providing contract information in a format that is easy for users to understand, and it operates primarily with three elements: a server, a terminal, and a user.

[0611] The user provides contract and terms of service information using their device. This information can be provided as an image file, selected as a text file, or by entering the URL of the webpage containing the contract. The information retrieval system on the device accepts this input.

[0612] The received information is transmitted from the terminal to the server. The server uses data manipulation tools to convert the received contract information into an appropriate format. For example, if it is image data, optical character recognition (OCR) is used to extract the text. The converted text data is then analyzed in detail by data analysis tools, and item identification tools are used to identify important clauses and risky content. Generative AI models are used in this process to accurately identify complex legal terminology and clauses that may affect end users.

[0613] Next, the server's risk assessment means evaluates the risk level based on the identified information, and the information generation means creates feedback for the user. The generated feedback is sent to the terminal through the information presentation means and displayed to the user with visual emphasis. For example, high-risk clauses are highlighted with color-coded icons to draw the user's attention.

[0614] A concrete example is a situation where a user is trying to understand the license agreement for new software. The user enters the URL of the license agreement page into their device. The server analyzes the page and generates warnings about automatic renewal clauses for subscription services and high penalties. By reviewing this information and reconsidering the contract before signing, the user can avoid disadvantages and make a safe and wise decision.

[0615] An example of a prompt message would be: "Please identify and visually highlight any risky clauses in the following contract. In particular, please provide details regarding the automatic renewal clause and penalties."

[0616] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0617] Step 1:

[0618] The user enters contract information through a terminal. This information is provided as an image file, text file, or webpage URL. The terminal's information acquisition mechanism receives this input and passes the contract information to the system as data. In this step, the input is the contract information provided by the user, and the output is the contract information as it has been incorporated into the system.

[0619] Step 2:

[0620] The terminal sends the captured contract information to the server. The server uses data manipulation tools to perform a conversion process: if the input information is image data, it extracts text using OCR technology; if it is text data, it sends it directly for analysis. The input for this step is the raw contract information provided by the user, and the output is the contract information converted into text data.

[0621] Step 3:

[0622] The server receives text data and performs a detailed analysis using data analysis tools. Specifically, it processes the data to identify important clauses and risky items within the contract. In this process, a generative AI model uses pre-trained data to identify clauses that require attention. The input for this step is contract information converted into text data, and the output is the identified important clauses and risk information.

[0623] Step 4:

[0624] The server's risk assessment means evaluates the risk level based on the identified clauses. Based on the results of the risk assessment, the information generation means generates feedback. This feedback includes detailed information and recommended actions regarding the clauses that the user should pay attention to. The input to this step is the identified risk information, and the output is the materialized feedback.

[0625] Step 5:

[0626] The server-generated feedback is sent to the terminal via an information display device and presented to the user. The terminal visually highlights the feedback, drawing the user's attention by using colors and icons to highlight particularly high-risk clauses. The user reviews the provided information and makes a final decision. The input for this step is the generated feedback, and the output is the visually highlighted feedback received by the user.

[0627] (Application Example 1)

[0628] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] In the process of understanding contracts and terms of service, users often overlook complex clauses or unfavorable conditions, resulting in the risk of entering into unfavorable agreements. In particular, electronic payment services may contain contract terms that require careful attention, and users often find it difficult to accurately understand these terms. This invention aims to help users easily understand the content of contract information, identify risky areas, and make informed decisions.

[0630] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0631] In this invention, the server includes receiving means for inputting contract information, conversion means for converting the contract information into a corresponding data format if it is image or document data, and analysis means for extracting text from the converted data. This allows users to easily understand risky contract terms by taking a picture of or uploading a contract or terms of service with a smart device, and the system analyzing the content and visually highlighting areas that require attention.

[0632] "Contract information" refers to data that includes the contents of terms of service, contracts, and other similar documents. This information should be provided in a format that is easy for users to understand.

[0633] "Acceptance method" refers to the function for obtaining contract information from a terminal. It is the first step in the user entering contract data into the system.

[0634] "Conversion means" refers to the function of converting contract information in image or document data format into an appropriate data format. This is the process of making text information extractable using OCR technology, etc.

[0635] "Analysis means" refers to the function that extracts text from the converted data and analyzes the contract content. It plays a role in preparing the information for further processing.

[0636] "Identification means" refers to a function that identifies areas requiring attention based on analyzed text data. This is necessary to identify risky clauses and warn users.

[0637] The "evaluation method" refers to a function that assesses the risk of identified areas and generates evaluation results. This provides users with information-based feedback.

[0638] "Generation method" refers to the function that generates feedback for the user based on the evaluation results. It is a process for providing information in a way that helps the user understand, such as through visual highlighting.

[0639] "Output mechanism" refers to a function that provides generated feedback to the user. It effectively presents information to help the user understand the contract terms.

[0640] To implement this invention, the system employs a configuration combining multiple hardware and software components. The user inputs contract information in image or text format using a smart device. The acquired contract information is transmitted to the system via a receiving means, where it undergoes a process of converting the data into text using OCR technology. This process utilizes an OCR library such as pytesseract.

[0641] The converted text is sent to a server and analyzed using natural language processing (NLP) technology. This analysis is performed using a NLP library such as spaCy to identify risky clauses and specialized legal terminology. The identified risky sections are evaluated in detail using evaluation tools to generate appropriate feedback for the user. The evaluation results are processed using generation tools for visual highlighting on smart devices.

[0642] For example, consider a scenario where a user is about to sign a contract for a new electronic payment service. If the user takes a photo of the contract and uploads the image to the system, the system can identify automatic withdrawal conditions and high penalty clauses, and warn the user. This allows the user to review unfavorable contract terms in advance and make an informed decision.

[0643] An example of a prompt to input into a generative AI model is, "Teach me how to develop an app that highlights risk clauses in contracts." This prompt allows the model to learn techniques for specifically identifying risky contract terms.

[0644] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0645] Step 1:

[0646] The user uses a terminal to input contract information as an image or URL. This input is received by the system via a receiving mechanism. The input data includes photos of contracts and URLs of web pages. The terminal prepares to send this data to the server.

[0647] Step 2:

[0648] To analyze the received data, the server first converts the image data into text data. This process utilizes OCR technology such as pytesseract. By extracting text from the input image data, the contract details can be treated as textual information. The extracted text is then output.

[0649] Step 3:

[0650] The server uses natural language processing techniques to evaluate the data and analyze the converted text. This process utilizes libraries such as spaCy. From the text data received as input, it identifies risky clauses and technical terms, and identifies points that require attention. The identified risky clauses are then output.

[0651] Step 4:

[0652] The server evaluates the identified risk areas and generates feedback based on the results. It uses evaluation tools to generate evaluation results and organizes information to determine the risks to the user. Detailed risk assessment information is output as feedback.

[0653] Step 5:

[0654] The terminal presents the generated feedback to the user. It processes the feedback to visually highlight risky areas and displays them on the terminal screen using a generation method. The user can then review this feedback to deepen their understanding of the contract and make a final decision. The output is the visually processed feedback.

[0655] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0656] This invention is a system that allows users to easily understand information related to contracts and terms of service, and adjusts feedback while also considering the user's emotional state at the time. In addition to the basic functions of inputting and analyzing contract information, this system also includes a function that recognizes the user's emotions using an emotion engine and generates feedback based on those emotions.

[0657] First, the user uses their device to input contract and terms of service information as images or URLs. The device sends this contract information to the server, which extracts the text data using OCR technology. Next, the server analyzes the text data using natural language processing to identify contract clauses that require attention.

[0658] This process also incorporates an emotion engine, which evaluates the user's emotions in response to the data extracted by the analysis tools. This system uses non-verbal data obtained from the user via the device, such as facial recognition and voice data, to determine emotions. If the user is feeling anxious, the emotion engine takes that emotion into account and adjusts the feedback to be more detailed or highlights important points.

[0659] The generation mechanism prepares user-optimized feedback based on emotional data provided by the emotion engine and risk information identified by the identification mechanism. This feedback visually displays the contract's risk level and is presented in a format that is easily understandable to the user.

[0660] For example, when a user is checking the terms and conditions of a new online service, if the system detects the user's unease, it will provide a detailed explanation of any special clauses or points to note regarding automatic renewal included in the contract, and offer specific suggestions to help the user make a decision with confidence. In this way, users can gain a deep understanding of the contract and make decisions with confidence.

[0661] Thus, as an interactive system that combines emotion recognition technology, the present invention provides users with better support for making contractual decisions.

[0662] The following describes the processing flow.

[0663] Step 1:

[0664] The user enters specific contract information on their device. Input methods include taking a picture, selecting an existing file, or directly entering a URL.

[0665] Step 2:

[0666] The terminal sends the entered contract information to the server. Image data is converted to Base64 format, and text data and URLs are sent in JSON format.

[0667] Step 3:

[0668] The server analyzes the received contract information. If image data is sent, the server uses OCR to extract text from the image. In the case of a URL, the page content is obtained in text format via web scraping.

[0669] Step 4:

[0670] The server uses analysis tools to analyze text data using natural language processing and identify clauses that require attention. Based on these analysis results, the risk points of the contract are identified.

[0671] Step 5:

[0672] The emotion engine activates and performs non-verbal information acquisition from the user's device, such as facial expression analysis via the camera and voice tone analysis via the microphone. Based on this information, the user's emotional state is evaluated.

[0673] Step 6:

[0674] The server combines identified risk information with the results of the emotion engine to generate feedback for the user. This feedback is tailored to the user's emotional state. For example, if the user is feeling anxious, the feedback will be more detailed and reassuring.

[0675] Step 7:

[0676] The device receives feedback sent from the server and presents it in a visual and user-friendly format. This highlights the contract's risk level, making it intuitively understandable to the user.

[0677] (Example 2)

[0678] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0679] Conventional contract information analysis systems have faced challenges in providing optimal feedback to users because it is difficult for them to fully understand the details of contracts and terms of service, and because the user's emotional state is not taken into consideration. In particular, when users are feeling anxious, they may overlook important information, so there is a need to provide contract information that takes the user's emotional state into account.

[0680] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0681] In this invention, the server includes a function for inputting contract information, a function for converting the contract information into a corresponding data format, and a function for detecting areas of interest based on the extracted text. This makes it possible to appropriately analyze the contract information of interest while taking into account the user's emotional state and provide the user with optimal feedback.

[0682] "Contract information" refers to data contained in contracts and terms of service, and is the information necessary for a user to consent to a particular service or product.

[0683] The "data format conversion function" refers to a function that converts the entered contract information into a parseable format, including the process of converting image and PDF data into text data.

[0684] The "text extraction function" refers to the process of extracting character information from the converted data and obtaining text data that will serve as the basis for analysis.

[0685] The "function to detect areas requiring attention" refers to the process of identifying information that is important to the user or content that may contain potential risks from the extracted text data.

[0686] The "risk assessment function" is a process that evaluates potential risks from detected information and determines their importance and impact on users.

[0687] The "function for evaluating emotions" refers to the process of analyzing non-verbal data such as the user's facial expressions and voice to identify the user's psychological state and emotions.

[0688] The "feedback generation function" is the process of constructing information to provide to the user based on assessed risk and sentiment data, and includes formats such as text and visual displays.

[0689] "Output function" refers to a function that presents the generated feedback in a format that is easy for the user to understand, and includes displaying it on the device.

[0690] This invention relates to a system that easily analyzes contract information and presents its contents in a format that is easy for users to understand. Users input contract and terms of service information using a terminal in image or URL format. The terminal temporarily stores this information and sends it to a server.

[0691] The server performs analysis on the received information. First, it uses OCR technology to convert image data into text data. Specifically, general OCR software is used here. Next, it uses a generative AI model to process the extracted text data and identify clauses that require attention or items that carry risks. For example, natural language processing toolkits and machine learning algorithms are utilized.

[0692] In addition, the server analyzes non-verbal data obtained from the user's terminal. By acquiring the user's facial expressions and voice and performing emotion analysis, it determines the user's emotional state. Software tools are used as emotion recognition technology in this process.

[0693] The server integrates analyzed contract information with data on the user's emotional state and generates feedback based on this. This feedback is presented to the user in the form of visualized risk levels and highlighting of important clauses. For example, if the system detects a user's anxious expression while reviewing an online service contract, it will provide detailed explanations regarding special clauses and automatic renewal considerations. An example of a prompt related to this operation might be, "Please provide a clear explanation of the contract clauses the user is concerned about," which could be input to the generating AI model.

[0694] In this way, by incorporating emotion recognition technology, users can gain a deeper understanding of the contract terms and create an environment where they can make decisions with confidence. The system as a whole provides an interactive analytical tool that combines detailed contract analysis with an evaluation of the user's emotional state.

[0695] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0696] Step 1:

[0697] The user enters contract information in image or URL format using a terminal. This becomes the initial input data for the system. The terminal temporarily stores this information and prepares to send it to the server. At this time, the input is completed via the interface on the user's terminal, and the destination server IP address is set.

[0698] Step 2:

[0699] The terminal sends the saved contract information to the server. The server receives the input data and converts it into a format for text analysis. Here, the server uses OCR technology to convert image files into text data. Specifically, it performs calculations to recognize characters from the image and convert them into digital text. The output of this process is text data containing the contract information.

[0700] Step 3:

[0701] The server feeds the text data extracted by OCR into a natural language processing engine. A generative AI model analyzes this text data to determine whether there are any clauses that require attention or any particular risks. Specifically, the generative AI model analyzes the text and performs data calculations to extract keywords and expressions related to risk. The output at this stage is a list of identified important clauses.

[0702] Step 4:

[0703] Simultaneously, the device collects nonverbal data such as the user's facial expressions and voice, and sends it to the server. The server analyzes this data using an emotion recognition algorithm to evaluate the user's emotional state. Here, the server analyzes the user's nonverbal signs and determines the emotion they indicate (e.g., anxiety, confusion). The output identifies the user's emotional state.

[0704] Step 5:

[0705] The server generates optimal feedback based on the key contract clauses identified by the generation AI model and the user's assessed emotional state. Feedback generation involves data processing to visually indicate the contract's risk level and highlight points the user should pay attention to. The generated feedback is then output and ready to use.

[0706] Step 6:

[0707] The terminal receives feedback information sent from the server and presents it to the user. Specifically, a well-formatted dashboard is displayed on the terminal, designed to make it easy for the user to understand the contract information. In this process, the output is provided in the form of visualized feedback for the user.

[0708] (Application Example 2)

[0709] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0710] Contract information and terms of service are often difficult for many users to understand, and as a result, they may accept contracts without fully understanding their contents. This situation can lead to disadvantages later on, so there is a need for a system that supports users in understanding contract contents and using them with peace of mind.

[0711] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0712] In this invention, the server includes a device for inputting contract information, a conversion device for converting the contract information into an appropriate information format if it is image or document information, and an analysis device for extracting text information from the converted information. This makes it possible to detect important points in the contract content and analyze the user's emotions, thereby providing more understandable and reliable feedback.

[0713] A "device for inputting contract information" is a device that has an interface for users to provide contract information to a system.

[0714] A "conversion device" is a device equipped with the function of converting received contract information into a format that is easy for the system to analyze.

[0715] An "analysis device" is a device that extracts necessary textual information from converted information and analyzes the contract details.

[0716] An "identification device" is a device that has the function of identifying areas of interest within contract information based on analyzed textual information.

[0717] An "evaluation device" is a device that evaluates the risk level of a contract based on identified sections and generates the evaluation results.

[0718] An "emotion analysis device" is a device that uses nonverbal data obtained from a user to analyze their emotions.

[0719] A "generating device" is a device equipped with the function of generating information to be provided to the user based on evaluation results and sentiment analysis results.

[0720] An "output device" is a device that visually displays the generated feedback to the user, highlighting and providing information.

[0721] In embodiments of this invention, the system includes means for effectively analyzing contract information and providing optimized feedback to the user. The specific configuration and operation are described below.

[0722] When a user first enters their contract information, they use a device such as a smartphone or tablet. The device receives the contract information by either taking a picture of the contract using its camera or by entering a URL, and then sends it to the server.

[0723] The server uses Tesseract OCR to extract text data from these image data. The extracted text is then analyzed using a natural language processing library such as spaCy to identify important clauses and risks within the contract information.

[0724] Furthermore, using OpenCV-based facial recognition technology and TensorFlow's emotion analysis model, the system analyzes the user's emotions from their facial expressions and voice. This allows the system to understand the user's level of anxiety and adjust the content and emphasis of the feedback accordingly.

[0725] The generated feedback is provided to the user through an intuitive interface developed using Flutter. This feedback highlights points of particular importance and presents them in a visually clear manner, allowing users to better understand the contract and make informed decisions.

[0726] As a concrete example, when a user registers for a new mobile payment service, this system allows them to take a picture of the terms of service displayed during the registration process. The system detects anxiety from the user's facial expressions and provides feedback that explains the terms in detail, especially those concerning automatic renewal and fees.

[0727] An example of a prompt for the generative AI model regarding the use of this system is: "If a user is reading a new service agreement and feels uneasy, how would you use AI to adjust the feedback? As a specific example, consider an application that combines facial recognition and natural language processing."

[0728] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0729] Step 1:

[0730] The user enters contract information. Using a smartphone or tablet, the contract information is collected on the device by taking a picture of the contract or entering a URL. At this stage, the input data is either an image or a URL.

[0731] Step 2:

[0732] The terminal sends the collected contract information to the server. The server uses Tesseract OCR to extract text data from the image data. This process takes image data as input and outputs text data.

[0733] Step 3:

[0734] The server analyzes the extracted text data. Using natural language processing libraries such as spaCy, it identifies important clauses and risks. The input here is text data, and the output is risk information.

[0735] Step 4:

[0736] The server analyzes the user's emotions. It uses OpenCV for facial recognition and a TensorFlow model to determine the emotion. Here, non-verbal data (images and audio) acquired by the camera is used as input data, and the output is the user's emotional state.

[0737] Step 5:

[0738] The server generates feedback based on the evaluation results and sentiment data. The content of the feedback is adjusted according to the evaluated risk information and the user's sentiment. The inputs to this process are risk information and sentiment data, and the output is feedback data.

[0739] Step 6:

[0740] The device provides the user with generated feedback. This is visually displayed through a Flutter-based UI, highlighting points that require particular attention. The input is feedback data, and the output is visual information presented to the user.

[0741] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0742] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0743] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0744] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0745] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0746] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0747] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0748] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0749] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0750] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0751] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0752] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0753] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0754] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0755] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0756] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0757] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0758] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0759] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0760] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0761] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0762] The following is further disclosed regarding the embodiments described above.

[0763] (Claim 1)

[0764] A means of receiving contract information,

[0765] A conversion means for converting the contract information into an image or document data format if the contract information is in image or document data,

[0766] An analysis means for extracting text from the converted data,

[0767] An identification means for detecting areas of concern based on the extracted text,

[0768] An evaluation means for evaluating the risk of the identified location and generating an evaluation result,

[0769] A generation means for generating user feedback based on the aforementioned evaluation results,

[0770] A system including an output means for outputting the generated feedback to the user.

[0771] (Claim 2)

[0772] The system according to claim 1, wherein the identification means identifies clauses requiring attention using natural language processing technology.

[0773] (Claim 3)

[0774] The system according to claim 1, wherein the output means visually highlights and provides the evaluation result to the user.

[0775] "Example 1"

[0776] (Claim 1)

[0777] A means of obtaining information for entering contract information,

[0778] A data manipulation means for converting the contract information into an image or document data format if the contract information is in image or document data,

[0779] A data analysis means for extracting text from the converted data,

[0780] An item identification means for detecting areas requiring attention based on the extracted text,

[0781] The process by which the item identification means identifies risky clauses using a generated AI model,

[0782] A risk assessment means for evaluating the risk of the identified location and generating an assessment result,

[0783] Information generation means for generating user feedback based on the aforementioned evaluation results,

[0784] A system including an information presentation means for outputting the generated feedback to the user.

[0785] (Claim 2)

[0786] The system according to claim 1, wherein the item identification means identifies clauses requiring attention using natural language processing technology.

[0787] (Claim 3)

[0788] The system according to claim 1, wherein the information presentation means provides the evaluation results to the user by visually highlighting them.

[0789] "Application Example 1"

[0790] (Claim 1)

[0791] A means of receiving contract information,

[0792] A conversion means for converting the contract information into an image or document data format if the contract information is in image or document data,

[0793] An analysis means for extracting text from the converted data,

[0794] An identification means for detecting areas of concern based on the extracted text,

[0795] An evaluation means for evaluating the risk of the identified location and generating an evaluation result,

[0796] A generation means that visually highlights and provides the evaluation results to the user,

[0797] Output means for outputting the generated feedback to the user,

[0798] A function to input contract information via the device's image acquisition method,

[0799] Risk identification function using natural language processing technology,

[0800] A function that visually displays warnings in smart devices,

[0801] A system that includes this.

[0802] (Claim 2)

[0803] The system according to claim 1, wherein the identification means identifies clauses requiring attention using natural language processing technology.

[0804] (Claim 3)

[0805] The system according to claim 1, wherein the output means has a function for providing the evaluation results to the user by visually highlighting them on a smart device.

[0806] "Example 2 of combining an emotion engine"

[0807] (Claim 1)

[0808] A function for entering contract information,

[0809] A function to convert the aforementioned contract information into a corresponding data format,

[0810] A function to extract text from the converted data,

[0811] A function to detect areas of concern based on the extracted text,

[0812] A function to evaluate the risk of the identified location,

[0813] A function that analyzes the user's nonverbal data to evaluate their emotions,

[0814] A function that generates user feedback based on the aforementioned risk and emotional assessment results,

[0815] A system including a function to output the generated feedback to the user.

[0816] (Claim 2)

[0817] The system according to claim 1, wherein the identification function uses language processing technology to identify areas that require attention.

[0818] (Claim 3)

[0819] The system according to claim 1, wherein the output function visually displays and provides the evaluation results to the user.

[0820] "Application example 2 when combining with an emotional engine"

[0821] (Claim 1)

[0822] A device for entering contract information,

[0823] A conversion device that converts the contract information into an appropriate information format if the contract information is image or document information,

[0824] An analysis device for extracting character information from the converted information,

[0825] An identification device that detects areas requiring attention based on the extracted character information,

[0826] An evaluation device that evaluates the degree of risk of the identified location and generates an evaluation result,

[0827] An emotion analysis device that evaluates a user's emotions using the user's nonverbal data,

[0828] A generation device that generates feedback to the user based on the aforementioned evaluation results and emotional evaluation,

[0829] A system including an output device that visually displays and highlights the generated feedback and outputs it to the user.

[0830] (Claim 2)

[0831] The system according to claim 1, wherein the identification device identifies items requiring attention using natural language processing technology.

[0832] (Claim 3)

[0833] The system according to claim 1, wherein the emotion analysis device determines the user's emotions from the user's facial expression data and voice data. [Explanation of symbols]

[0834] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving contract information, A conversion means for converting the contract information into an image or document data format if the contract information is in image or document data, An analysis means for extracting text from the converted data, An identification means for detecting areas of concern based on the extracted text, An evaluation means for evaluating the risk of the identified location and generating an evaluation result, A generation means that visually highlights and provides the evaluation results to the user, Output means for outputting the generated feedback to the user, A function to input contract information via the device's image acquisition method, Risk identification function using natural language processing technology, A function that visually displays warnings in smart devices, A system that includes this.

2. The system according to claim 1, wherein the identification means identifies clauses requiring attention using natural language processing technology.

3. The system according to claim 1, wherein the output means has a function for providing the evaluation results to the user by visually highlighting them on a smart device.

Images