system

Abstract

This system uses OCR technology and generative AI to automatically extract contract information from electronic documents related to contracts. [Solution] An information processing device for automated document management, comprising: means for executing electronic document reception processing; means for extracting information by visual character recognition processing based on the received electronic document; means for creating associations and generating a document structure based on the extracted information; means for receiving inquiries in natural language and searching for related electronic documents; means for evaluating risk factors and generating an analysis report based on the analyzed information; means for monitoring the expiration date of a document and providing notification when the deadline approaches; and means for generating a document template based on a request for contract document generation.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Contract management in modern enterprises relies on analog methods such as assigning identification IDs and storing them in folders, which has problems requiring a lot of human labor and time. Also, in updating contract information and risk assessment, timely and efficient management is difficult, and the quality of management may decline. Due to such problems, there is a demand for a system that can automate contract management and perform it efficiently and accurately.

Means for Solving the Problems

[0005] This invention provides a system that automatically extracts contract information from contract-related electronic documents using OCR technology and generation AI. This enables rapid editing and updating of important information in contracts. Furthermore, it improves the accuracy and efficiency of contract management by providing features such as natural language search, risk analysis report generation, and contract renewal deadline reminder notifications. In addition, the automatic contract document generation function simplifies the contract creation process, allowing for smoother negotiations and reviews.

[0006] An "information processing device" is a device that includes hardware and software for automated document management.

[0007] An "electronic document" is a document that is represented in digital format and can be processed by a computer or other digital device.

[0008] "Visual character recognition processing" is a technology that automatically detects character information from images and PDFs and extracts it as text.

[0009] "Association" refers to identifying the relationships between multiple documents based on extracted information and organizing them into a single structure.

[0010] "Document structure" refers to the hierarchical or network-like arrangement formed by a group of related documents.

[0011] "Natural language querying" refers to asking questions to a system using human language.

[0012] "Risk factors" are elements that introduce problems or uncertainties inherent in contracts or documents.

[0013] An "analysis report" is a document that summarizes the results of information gathering and research conducted for a specific purpose.

[0014] "Update Deadline" refers to the date when the next confirmation or update is required in the contract document.

[0015] "Contract Document Generation" is a process that automatically creates a draft of the contract based on the input information.

Brief Description of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of the data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of the data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of the data processing device and the smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of the data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of the data processing device and the headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of the data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of the data processing device and the robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The AI ​​contract management system of the present invention involves a server, terminals, and users working together to perform a series of processes for automated contract document management.

[0038] The server first receives PDF or image files of contracts uploaded by users. The server then performs optical character recognition (OCR) processing on the received files to extract text information from the documents. This process utilizes OCR technology to accurately read both printed and handwritten text.

[0039] The extracted information consists of key data such as contract title, signatory, and signing date. The server uses this data to create associations and generate a document structure. The server also stores the information in a database to prepare for efficient search processing later.

[0040] The terminal receives search queries from the user in natural language format through its interface and sends them to the server. The server searches the database based on the query, identifies the most relevant contract documents, and returns the results to the terminal. The terminal displays the search results in an easy-to-understand format for the user.

[0041] Furthermore, the server analyzes the contract details in real time, evaluates the risk factors included in the contract, and generates an analysis report as needed. Based on this risk analysis, it becomes easier for users to take countermeasures.

[0042] The server also monitors the set renewal deadline and sends reminder notifications to the user via their device as the deadline approaches. This ensures that the user does not miss important contract renewal deadlines.

[0043] Users can request the automatic generation of contract documents via their terminal as needed. The server combines contract templates based on specified parameters and automatically generates a draft. The user can then review the draft on their terminal and input any necessary revisions or additions.

[0044] In this way, the AI ​​contract management system achieves efficient and accurate contract management through a series of automated processes, significantly reducing the workload of users.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The server receives PDF and image files of contracts uploaded by users. These files contain important contract information.

[0048] Step 2:

[0049] The server performs visual character recognition (OCR) processing on the received file, extracting text information from the document using OCR technology. This process accurately reads not only printed text but also handwritten characters.

[0050] Step 3:

[0051] The server analyzes the information extracted by OCR to identify important contract data such as contract title, signatory, and signing date, and stores the results in a database.

[0052] Step 4:

[0053] The server automatically associates information stored in the database and generates a document structure by combining similar contract information. This improves the overall traceability of the document.

[0054] Step 5:

[0055] The terminal sends the natural language search query received from the user to the server, and the server retrieves relevant information corresponding to the query from the database.

[0056] Step 6:

[0057] The server aggregates the search results, identifies the most relevant contract documents, and returns them to the terminal. The terminal then clearly displays these search results to the user.

[0058] Step 7:

[0059] The server analyzes the contract details and generates an analysis report if it identifies specific risk factors. This report is saved so that the user can review it later.

[0060] Step 8:

[0061] The server monitors the renewal deadlines for each contract and sets reminders as the deadline approaches. The terminal then notifies the user of important contract renewals.

[0062] Step 9:

[0063] The user requests the creation of a new contract document from the server via their terminal. Based on the provided conditions, the server automatically generates a draft contract using a generation AI.

[0064] Step 10:

[0065] The user reviews the contract generated on their device and makes corrections as needed. The server then securely stores the final, corrected version.

[0066] (Example 1)

[0067] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0068] Traditional contract document management systems often required manual, complex searches and risk assessments, leading to inefficiencies. Furthermore, managing contract renewal deadlines was cumbersome, increasing the risk of missing renewal deadlines. These challenges need to be addressed to streamline contract management and improve accuracy.

[0069] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0070] This invention includes a server that receives and stores uploaded contracts from users in an organized manner, a server that extracts information from printed text and handwritten characters within electronic documents using OCR technology, and a server that analyzes the extracted information to generate key data such as contract name, signatories, and date and time as a document structure. This enables automated contract management, efficient searching, risk assessment, and deadline management.

[0071] A "user" is someone who uses a terminal to upload contracts or enter search queries into the system.

[0072] A "server" is a central processing unit that receives information from users, analyzes the data using OCR technology, and performs tasks such as storing, searching, displaying, and risk assessment of contract documents.

[0073] "OCR technology" refers to optical character recognition technology, which converts printed text and handwritten characters in electronic documents and images into digital data.

[0074] A "contract document" is an official document that records the details of a transaction or agreement, and is subject to renewal and risk assessment.

[0075] "Risk assessment" is the process of analyzing the contents of a contract document to identify and evaluate factors that could potentially cause problems in the future.

[0076] "Deadline management" is a function that monitors contract renewal deadlines and notifies users to ensure they don't miss the renewal deadline.

[0077] A "draft" is a template for a contract document, a draft automatically generated based on user requests.

[0078] This invention provides an AI-powered contract management system that enables efficient contract management through automated processes. This system operates through the collaboration of servers, terminals, and users.

[0079] The server first receives PDF or image files of contracts uploaded by the user via their device. At this point, the server uses open-source optical character recognition tools such as Tesseract OCR to convert printed and handwritten text within the document into digital data. This technology is used to accurately extract important information.

[0080] From the extracted information, the server extracts key data such as the contract title, signatory, and signing date, and stores this as structured data. This improves the efficiency of association and searching. The information is also stored in a database, and an index is generated for easy searching.

[0081] The terminal receives natural language search queries from the user and sends them to the server. The server searches the database based on the received queries and identifies relevant contract documents. The results are sent to the terminal and made available for the user to view. For example, a prompt such as "Show me the transaction agreements concluded in September 2023" can be used.

[0082] Furthermore, the server analyzes the contract content in real time and assesses potential risk factors. This analysis uses a generative AI model, and based on the evaluation results, it generates and provides an analysis report to the user.

[0083] The contract renewal deadline is monitored by the server, and users are notified via their devices as the deadline approaches. Based on this notification, users can renew their contracts in a timely manner.

[0084] Furthermore, if a user wishes to create a new contract, they can request automatic contract generation through their terminal. The server generates a draft according to pre-prepared templates and user-specified parameters, and the user can review and modify the draft on their terminal.

[0085] These features enable the AI ​​contract management system to improve the efficiency and accuracy of contract management, significantly reducing the workload for users.

[0086] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0087] Step 1:

[0088] The user uploads a PDF or image file of the contract to the system using a terminal. This becomes the input data. The terminal sends the uploaded data to the server, and the data reception process begins. The server temporarily stores this uploaded document in its storage.

[0089] Step 2:

[0090] The server performs OCR processing on the received contract. Here, OCR software is used to recognize text within the image. The input is an image of the contract, and the output is recognized text information. This process analyzes both printed and handwritten characters, extracting text data from each.

[0091] Step 3:

[0092] The server analyzes the text information extracted by OCR to identify key contract data. Important items such as the contract title, signatories, and signing date are extracted. The input is the OCR output text, and the output is a list of this key data. Based on this, the server generates the document structure and stores it in a database.

[0093] Step 4:

[0094] When a user wants to search for specific contract information, they enter a query in natural language format via a terminal. The terminal sends this query to the server. The server parses the query, searches the database, and identifies the relevant contract documents. In this case, the input is the user query, and the output is a list of relevant contract documents.

[0095] Step 5:

[0096] The server sends the search results to the terminal. The terminal displays the results to the user in an easy-to-understand format. The user can view the displayed results and check the details as needed. A possible specific prompt would be, "Show me the transaction agreements concluded in September 2023."

[0097] Step 6:

[0098] The server analyzes the contract content in real time and evaluates risk factors. It uses a generative AI model to perform risk assessments and generates risk analysis reports as needed. The input is the contract content, and the output is the risk assessment results and report. This allows users to take appropriate measures against risks.

[0099] Step 7:

[0100] The server manages the contract renewal deadline. As the contract renewal deadline approaches, the server sends a notification to the user via the terminal. The input is the renewal deadline information, and the output is a notification message. The user can receive this notification and proceed with the appropriate renewal procedure.

[0101] Step 8:

[0102] Users can request automated contract generation from their terminal to create new contracts. The server generates a draft by combining a template with user-provided parameters. The input is user parameters, and the output is the drafted contract. This draft is sent to the terminal for the user to review and make any necessary revisions.

[0103] (Application Example 1)

[0104] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0105] In modern manufacturing, efficiently managing equipment maintenance contracts and various production line contracts is a crucial factor in reducing operational costs and improving production efficiency. However, when these contracts are managed manually, it is difficult to retrieve the necessary information at the appropriate time from a vast amount of contract data. Furthermore, mistakes such as missing contract renewal deadlines are likely to occur, potentially impacting the manufacturing process. Therefore, new methods are needed to solve these problems.

[0106] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0107] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by visual data recognition processing based on the received electronic data, and means for coordinating with a control device incorporated in the equipment to manage and notify maintenance schedules. This enables efficient management of equipment maintenance contracts and other contract information, allows maintenance to be performed at the appropriate time, and enables smooth operation of the manufacturing line.

[0108] An "information processing device" is a system that includes devices for receiving, processing, and managing electronic data, such as computers and servers.

[0109] "Visual data recognition processing" is the process of extracting textual information from image data, and is performed using OCR technology.

[0110] A "natural language query" is an interface in which a user uses ordinary language to request information from a system.

[0111] "Risk factors" refer to potential dangers and problems hidden within contract information, and evaluating these is a necessary element.

[0112] An "analysis document" is a document that summarizes the results of the risk factor assessment and presents them to users in an easy-to-understand manner.

[0113] A "control device" is a device used to manage and operate specific equipment or systems.

[0114] A "maintenance schedule" is a plan that outlines the timing and content of maintenance and inspections of equipment and machinery.

[0115] A "notification" is the act of informing a user of a specific event or piece of information, and often takes the form of an alert or reminder.

[0116] In the system that realizes this invention, the server, terminal, and user work together in coordination.

[0117] The server first receives various electronic data related to the equipment from cameras and scanning devices attached to robotic arms within the factory. The received data is then processed using OCR technology to extract textual information. Existing software such as Tesseract OCR is used for this process. The extracted information is stored in a database and attributed to contract information.

[0118] The terminal accepts natural language queries from users. This interface is designed for users to search for contract information and check maintenance schedules. The server searches the database based on these queries, efficiently extracts the relevant information, and returns it to the terminal.

[0119] Furthermore, the control unit built into the device and the server work together to perform risk assessments and automatically create maintenance schedules based on the extracted data. The assessment algorithm executed on the server identifies risk factors based on contract information and generates analytical reports. In addition, when important renewal deadlines or maintenance schedules approach, reminders are automatically sent to the terminal in the form of emails or alerts.

[0120] As a concrete example, a certain factory requires a monthly maintenance contract for a specific machine. This system uses OCR technology to read the contract related to that machine, automatically calculates the next maintenance date within the system, and adds it to Google Calendar. At the same time, it sends an email notification to the person in charge five days in advance.

[0121] Examples of prompt messages include, "When is the next scheduled maintenance? Please extract the information from the contract." This system design enables more efficient contract management and supports maintenance operations.

[0122] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0123] Step 1:

[0124] The server receives electronic data about the equipment from a camera attached to a robotic arm in the factory. This process involves inputting image data captured by the camera. The server records this image data and prepares it for the next step, which involves visual data recognition processing.

[0125] Step 2:

[0126] The server performs visual data recognition (OCR) processing on the received image data to extract text information. The input is the received image data, and the output is the extracted text information. This process uses software such as Tesseract OCR to recognize both printed and handwritten characters.

[0127] Step 3:

[0128] The server assigns attributes to the contracts based on the extracted text information. In this step, the text information obtained by OCR is used as input, and the output is contract information with attributes such as contract date and maintenance details, which is stored in the database. The server then manages the necessary contract information based on this.

[0129] Step 4:

[0130] The user makes a query using natural language through the terminal. In this case, the input is a prompt such as, "When is the next maintenance scheduled?" The terminal receives this input and sends it to the server.

[0131] Step 5:

[0132] The server searches the database based on the user's query and extracts relevant contract information. The input is a natural language query from the user, and the output is the corresponding contract information. The server selects the most relevant information and sends it to the terminal.

[0133] Step 6:

[0134] The server manages maintenance schedules and generates reminders through coordination with the control devices built into the equipment. Contract information and schedule information are used as input, and reminder information is generated as output. Specifically, it calculates the next maintenance date and sets notifications for email and calendar applications.

[0135] Step 7:

[0136] The terminal displays search results and reminder information received from the server to the user. This output includes the next scheduled maintenance date and related contract information. The user can take appropriate action based on the displayed information.

[0137] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0138] The AI ​​contract management system of the present invention utilizes servers, terminals, users, and an emotion engine to automate the management of contract documents and improve the user experience.

[0139] First, the server receives electronic files of contracts sent by the user. These include PDF and image files. The server performs visual character recognition on these files to extract important information from the contract. This information includes the contract title, signatories, and signing date.

[0140] Next, the server associates the extracted information and generates a document structure. In this process, the sentiment engine considers the user's past preferences and current emotional state to help with optimal association. As a result, document management becomes faster and more accurate.

[0141] The terminal receives search queries from the user in natural language and sends them to the server. The server searches the database based on these queries to identify the most suitable contract documents. Simultaneously, an emotion engine analyzes the user's emotional state and adjusts how search results are displayed and the feedback messages. For example, if the user is stressed, the system helps the user quickly understand the information by summarizing the search results concisely.

[0142] The server also takes emotion engine data into account when performing a risk assessment of the contract and generating an analysis report based on the identified risk factors. When the user is relaxed, a detailed analysis report is provided to allow the user to carefully consider its contents.

[0143] Furthermore, the server monitors contract renewal deadlines and sends reminder notifications via the device as the deadline approaches. The content of the notifications is also adjusted based on insights from the emotion engine, taking care to ensure that users can manage their contracts while maintaining positive emotions.

[0144] Finally, based on the request to generate a contract document, the server generates a document template. At this time, the emotion engine checks the user's current state and helps ensure that the generated document matches the user's requirements.

[0145] In this way, the AI ​​contract management system, by combining an emotion engine, provides an optimized contract management experience for users, aiming to improve operational efficiency and satisfaction.

[0146] The following describes the processing flow.

[0147] Step 1:

[0148] Users use their devices to upload electronic files of contracts to the system. These files include PDF and image formats.

[0149] Step 2:

[0150] The server receives the uploaded contract and uses OCR technology to extract text from the document. This process recognizes both handwritten and printed characters with high accuracy.

[0151] Step 3:

[0152] The server identifies key information such as contract title, signatory, and signing date based on the extracted text data, and stores this information in a database.

[0153] Step 4:

[0154] The server automatically associates information stored in the database with other similar contract documents and generates a document structure. An emotion engine also operates, taking user sentiment into consideration to optimize the associations.

[0155] Step 5:

[0156] The terminal receives a natural language search query from the user, sends it to the server, and starts the search.

[0157] Step 6:

[0158] The server searches the database based on the received query to find the most relevant contract documents. During this process, the sentiment engine evaluates the user's emotional state and adjusts how the search results are presented.

[0159] Step 7:

[0160] The server analyzes the contract details and, if risk factors are identified, generates an analysis report. Using information from the sentiment engine, the report is presented in a user-friendly format.

[0161] Step 8:

[0162] The server monitors the contract renewal deadline and sets a reminder when the deadline approaches. The terminal sends emotionally adaptive reminder notifications to the user.

[0163] Step 9:

[0164] The user requests the automatic generation of a contract document through their terminal. The server uses a generation AI based on the conditions specified by the user to create a contract template.

[0165] Step 10:

[0166] The server uses an emotion engine to analyze the user's state and configures document generation and display to enhance user satisfaction. Users can review the generated contract on their device and make any necessary modifications.

[0167] (Example 2)

[0168] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0169] In modern information management systems, managing electronic data has become increasingly complex, and efficient and effective processing is required, especially for critical data such as contract documents. Furthermore, flexible functions that can adaptively manage data according to the user's emotional state are also necessary. However, conventional systems have struggled to integrate such flexibility with advanced information extraction and management functions.

[0170] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0171] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by optical character recognition processing, and means for creating a data structure by associating data. This enables efficient management of contract documents while providing nuanced responses based on the user's emotions.

[0172] A "data processing device" is a system that combines hardware and software for efficiently managing and processing electronic data.

[0173] "Electronic data" refers to all information that is expressed in digital format and can be processed on a computer.

[0174] "Optical character recognition processing" is a technology that identifies characters and numbers from electronic data in image format and extracts them as text data.

[0175] "Relationship building" refers to the process of analyzing the relationships between extracted information and organizing them in an integrated manner.

[0176] A "data structure" is a method of organizing information in a specific format to enable efficient processing and storage of that information.

[0177] "Emotional state" refers to the user's current emotions and psychological state, and is a factor that influences the system's operation and response.

[0178] "Reporting" refers to the action of providing users with important information or notifications regarding deadlines.

[0179] A "data template" is a standardized format that serves as the basis for generating electronic data such as contract documents.

[0180] The data processing device of the present invention automates the management of contract documents using a server, terminals, users, and an engine that performs sentiment analysis. Its embodiments are described in detail below.

[0181] First, the server receives contract documents sent electronically by the user. These documents are typically in PDF or image format. The server then uses optical character recognition (OCR) software, such as Tesseract, to extract text data from these documents. This digitizes paper-based documents, making them electronically processable.

[0182] Next, the server analyzes the extracted text data to identify key information such as the contract title, the names of the parties, and the signing date. Based on this information, the server generates the data structure for the contract document. At this stage, the sentiment analysis engine takes into account the user's past preferences and current emotional state to improve the accuracy of the associations.

[0183] The terminal receives natural language prompts from the user and sends them to the server. An example of a prompt might be, "Show me the latest contract." The server uses this to search its database and identify the relevant contract document. The server analyzes the user's emotional state through sentiment analysis and adjusts how the results are displayed. For example, if the user is feeling stressed, the information may be visually organized to aid understanding.

[0184] Furthermore, the server evaluates the risk factors of the contract and, if a detailed analysis is required, generates an analytical report that reflects the results of sentiment detection. Depending on the user's emotional state, they can choose to receive a detailed or summary report.

[0185] Finally, the server monitors the contract document's renewal deadline, and when the deadline approaches, a notification is sent to the user via their device. The notification's wording is adjusted according to the user's emotional state to ensure a positive user experience. Furthermore, if the user wishes to create a new contract document, the server generates a template based on the sentiment analysis results, providing the document that best suits the user's needs.

[0186] In this way, the data processing device of the present invention highly automates contract document management and enables the provision of individually adapted information by taking into account the user's emotions.

[0187] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0188] Step 1:

[0189] The server receives contract documents from the user as electronic data. This input includes PDFs and image files. The received electronic data is processed by OCR software (e.g., Tesseract) to extract text data from the visual data. The output of this process is used as text data in subsequent processing.

[0190] Step 2:

[0191] The server analyzes the text data extracted by OCR. It identifies important metadata such as contract title, signatory, and signing date from the input text data and records it in the database. In this step, an information identification algorithm is applied to extract the important information, which is then output as structured data.

[0192] Step 3:

[0193] The server generates a document structure based on the identified metadata. In this process, metadata, the user's past preferences, and current sentiment state are used as input data, and the sentiment analysis engine improves the accuracy of the associations. The output of this process is a well-organized data structure, which is used for subsequent searching and management.

[0194] Step 4:

[0195] The terminal receives natural language prompts from the user and sends them to the server. An example of an input prompt is "Show me the latest contract." The server parses this prompt and searches its database based on the prompt it received as input. It identifies the most relevant contract document and formats it in the optimal way for the user to see it.

[0196] Step 5:

[0197] The server analyzes contract information and assesses potential risk factors from the input data. It then generates an analytical report that reflects the output of the sentiment analysis engine. The report generated through this process is provided in either a detailed or concise format, flexibly adapting to the user's needs.

[0198] Step 6:

[0199] The server monitors the renewal deadlines for contract documents and sends notifications to users via their terminals as the deadline approaches. Based on the input monitoring data, it creates notifications and outputs messages that reflect the results of sentiment analysis. This improves user responsiveness.

[0200] Step 7:

[0201] When a user requests the generation of a new contract document, the server creates the document using a template. This input includes the user's requests and emotional state, and the server generates the most suitable document based on this information. The resulting document is tailored to the user's needs, thereby improving operational efficiency.

[0202] (Application Example 2)

[0203] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0204] Traditional contract management systems have struggled to efficiently manage the content of contract documents and provide appropriate feedback while considering the user's emotional state. Furthermore, there has been a lack of sufficient methods to automate the risk assessment of contract content and reduce the burden on users. Therefore, there is a need to improve the efficiency of contract processing and optimize the user experience.

[0205] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0206] In this invention, the server includes means for performing processing to receive electronic data, means for acquiring information by visual character recognition based on the received data, means for generating related information based on the acquired information, means for receiving instructions in natural language and searching for related electronic data, means for evaluating risks and generating analysis reports based on the analyzed information, means for monitoring data update deadlines and providing notifications when the deadline approaches, means for generating data templates based on document generation instructions, and means for adjusting feedback based on user status. This streamlines the entire contract management process and enables the provision of appropriate feedback that meets user needs.

[0207] "Electronic data" refers to information recorded in digital format, and examples include contract documents and document files.

[0208] "Visual character recognition" is a technology that reads characters from images, PDF files, and other sources and extracts them as text information.

[0209] "Generating related information" means organizing and integrating necessary information based on acquired data to form new, meaningful information.

[0210] "Natural language instructions" refer to instructions and questions based on the language that humans use in everyday life, with the aim of information processing systems understanding and responding to them.

[0211] "Assessing risks and generating an analysis report" is the process of identifying potential problems and risks based on the information obtained and creating a report based on that information.

[0212] "Monitoring and notifying data expiration dates" refers to a function that tracks the expiration dates of data and information held and notifies users when the expiration date is approaching.

[0213] "Generating data templates based on document generation instructions" refers to a function that creates the basic structure and format of necessary documents according to user instructions.

[0214] "Adjusting feedback based on user status" is the process of optimizing the content of information and advice provided by taking into account the user's emotions and circumstances.

[0215] The system for realizing this invention consists mainly of a central server and user terminals. The server receives electronic data transmitted from the user and extracts the necessary information using visual character recognition technology. Software such as OpenCV and Pytesseract is used for this technology. The extracted information is structured in combination with other related information and stored in a database.

[0216] Instructions in natural language are sent to the server via the terminal. The server understands them, searches for the appropriate electronic data, and returns the results. Generative AI models are used in this process to analyze the user's emotional state. For example, if the user is in a hurry, it provides concise feedback; if they are relaxed, it presents a detailed analytical report.

[0217] Furthermore, the server assesses the risks in contract data and generates an analysis report. This helps users make informed decisions by identifying potential risks in the contract and communicating them to them. It also monitors data update deadlines and notifies users via their devices when the deadline approaches. This notification is also delivered in an optimal way, taking into account the user's emotional state.

[0218] When a document is requested to be generated, the server creates a template based on the specified format. In this process, it can reflect the user's emotional state and generate the document in the most appropriate form.

[0219] For example, when handling electronic contracts with construction companies, the server extracts contract details using visual character recognition and performs a risk assessment. Depending on the user's situation, it may provide a concise notification such as, "There is a high risk to your budget," or a more detailed notification such as, "There is a high possibility of exceeding your budget, so please consider additional cost management measures."

[0220] In this way, the system dynamically adjusts feedback based on the user's emotional state, not only increasing the efficiency of contract management but also improving the user experience. For example, a prompt might say, "Assess the risks of electronic contracts with construction companies and generate a concise report if the user is in a hurry, or a detailed report otherwise."

[0221] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0222] Step 1:

[0223] The user sends electronic data to the server using their device. Input can include contract data in PDF or image format. Once this data reaches the server, data reception is complete.

[0224] Step 2:

[0225] The server performs visual character recognition processing on the received electronic data. Specifically, it uses OpenCV and Pytesseract to extract text information from the image data. The input is image data, and the output is the extracted text information.

[0226] Step 3:

[0227] The server generates contract-related information based on the extracted text information. This process involves organizing the information and shaping its related structure. The input is the acquired text information, and the output is the associated contract information.

[0228] Step 4:

[0229] The user sends instructions in natural language to the server using a terminal. The server then analyzes the user's instructions and searches for relevant electronic data. The input is the user's instructions, and the output is the search results data.

[0230] Step 5:

[0231] The server uses a generative AI model to perform a risk assessment based on the search results data and generates an analysis report. At this stage, the user's emotional state is analyzed, and the output report is adjusted accordingly. The input is the search results data, and the output is the adjusted analysis report.

[0232] Step 6:

[0233] The server monitors the data update deadline and sends a notification to the user's terminal when the deadline approaches. The prompt message used is "Assess the risks of the electronic contract with the construction company...". The input is the current date and time and data update information, and the output is a notification based on the conditions.

[0234] Step 7:

[0235] The server generates a data template based on the user's instructions for document creation. In this process, the user's emotional state is reflected, and the document is created in the most optimal format. The input is the user's instructions and emotional state, and the output is the generated document template.

[0236] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0237] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0238] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0239] [Second Embodiment]

[0240] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0241] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0242] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0243] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0244] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0245] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0246] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0247] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0248] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0249] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0250] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0251] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0252] The AI ​​contract management system of the present invention involves a server, terminals, and users working together to perform a series of processes for automated contract document management.

[0253] The server first receives PDF or image files of contracts uploaded by users. The server then performs optical character recognition (OCR) processing on the received files to extract text information from the documents. This process utilizes OCR technology to accurately read both printed and handwritten text.

[0254] The extracted information consists of key data such as contract title, signatory, and signing date. The server uses this data to create associations and generate a document structure. The server also stores the information in a database to prepare for efficient search processing later.

[0255] The terminal receives search queries from the user in natural language format through its interface and sends them to the server. The server searches the database based on the query, identifies the most relevant contract documents, and returns the results to the terminal. The terminal displays the search results in an easy-to-understand format for the user.

[0256] Furthermore, the server analyzes the contract details in real time, evaluates the risk factors included in the contract, and generates an analysis report as needed. Based on this risk analysis, it becomes easier for users to take countermeasures.

[0257] The server also monitors the set renewal deadline and sends reminder notifications to the user via their device as the deadline approaches. This ensures that the user does not miss important contract renewal deadlines.

[0258] Users can request the automatic generation of contract documents via their terminal as needed. The server combines contract templates based on specified parameters and automatically generates a draft. The user can then review the draft on their terminal and input any necessary revisions or additions.

[0259] In this way, the AI ​​contract management system achieves efficient and accurate contract management through a series of automated processes, significantly reducing the workload of users.

[0260] The following describes the processing flow.

[0261] Step 1:

[0262] The server receives PDF and image files of contracts uploaded by users. These files contain important contract information.

[0263] Step 2:

[0264] The server performs visual character recognition (OCR) processing on the received file, extracting text information from the document using OCR technology. This process accurately reads not only printed text but also handwritten characters.

[0265] Step 3:

[0266] The server analyzes the information extracted by OCR to identify important contract data such as contract title, signatory, and signing date, and stores the results in a database.

[0267] Step 4:

[0268] The server automatically associates information stored in the database and generates a document structure by combining similar contract information. This improves the overall traceability of the document.

[0269] Step 5:

[0270] The terminal sends the natural language search query received from the user to the server, and the server retrieves relevant information corresponding to the query from the database.

[0271] Step 6:

[0272] The server aggregates the search results, identifies the most relevant contract documents, and returns them to the terminal. The terminal then clearly displays these search results to the user.

[0273] Step 7:

[0274] The server analyzes the contract details and generates an analysis report if it identifies specific risk factors. This report is saved so that the user can review it later.

[0275] Step 8:

[0276] The server monitors the renewal deadlines of each contract and sets a reminder when the deadline approaches. Then, the terminal notifies the user of important contract renewals.

[0277] Step 9:

[0278] The user requests the server through the terminal to create a new contract document. The server automatically generates a draft of the contract using the generation AI based on the provided conditions.

[0279] Step 10:

[0280] The user checks the contract document generated on the terminal and makes corrections if necessary. Also, the server securely stores the final version after the corrections.

[0281] (Example 1)

[0282] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0283] In a conventional contract document management system, it is necessary to manually perform complex searches and risk evaluations, and the efficiency tends to decrease. Also, managing the renewal deadlines of contract documents is time-consuming, and there is a risk of missing the timing of renewal. It is necessary to solve these problems, improve contract management efficiency, and enhance accuracy.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0285] In this invention, the server includes means for receiving a contract document uploaded by a user and storing it in an organized form, means for extracting information from the printed text and handwritten characters in an electronic document using OCR technology, and means for analyzing the extracted information and generating key data such as contract names, signatories, dates, etc. as a document structure. Thereby, automatic management of contract documents, efficient search, risk assessment, and deadline management become possible.

[0286] "User" refers to a person who uses the system to upload a contract or enter a search query through a terminal.

[0287] "Server" refers to the central processing device that receives information from users, analyzes data using OCR technology, and performs storage, search, display, and risk assessment of contract documents.

[0288] "OCR technology" refers to optical character recognition technology, which is a technology for converting printed text and handwritten characters in electronic documents and images into digital data.

[0289] "Contract document" refers to an official document recording the content of a transaction or agreement, which is the object of update and risk assessment.

[0290] "Risk assessment" refers to the process of analyzing the content in a contract document, detecting and evaluating factors that may cause problems in the future.

[0291] "Deadline management" refers to the function of monitoring the renewal deadline of a contract and notifying the user so that the user does not miss the renewal timing.

[0292] "Draft" refers to a prototype of a contract document, which is a draft automatically created based on requests from users.

[0293] The present invention constitutes an AI contract management system and realizes efficient contract management through an automated process. This system operates with the cooperation of a server, a terminal, and a user.

[0294] First, the server receives PDF or image files of contract documents uploaded by users through terminals. At this time, the server uses an optical character recognition tool such as open-source Tesseract OCR to convert printed text and handwritten characters in the document into digital data. This technology is used to accurately extract important information.

[0295] From the extracted information, the server extracts key data such as the contract title, signatory, and signing date, and stores this as structured data. This improves the efficiency of association and searching. The information is also stored in a database, and an index is generated for easy searching.

[0296] The terminal receives natural language search queries from the user and sends them to the server. The server searches the database based on the received queries and identifies relevant contract documents. The results are sent to the terminal and made available for the user to view. For example, a prompt such as "Show me the transaction agreements concluded in September 2023" can be used.

[0297] Furthermore, the server analyzes the contract content in real time and assesses potential risk factors. This analysis uses a generative AI model, and based on the evaluation results, it generates and provides an analysis report to the user.

[0298] The contract renewal deadline is monitored by the server, and users are notified via their devices as the deadline approaches. Based on this notification, users can renew their contracts in a timely manner.

[0299] Furthermore, if a user wishes to create a new contract, they can request automatic contract generation through their terminal. The server generates a draft according to pre-prepared templates and user-specified parameters, and the user can review and modify the draft on their terminal.

[0300] These features enable the AI ​​contract management system to improve the efficiency and accuracy of contract management, significantly reducing the workload for users.

[0301] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0302] Step 1:

[0303] The user uses the terminal to upload the PDF or image file of the contract to the system. This becomes the input data. The terminal sends the uploaded data to the server, and the data reception process is started. The server temporarily stores this uploaded document in storage.

[0304] Step 2:

[0305] The server performs OCR processing on the received contract. Here, OCR software is utilized to recognize the text in the image. The input is the image of the contract, and the recognized string information is obtained as the output. In this process, both printed characters and handwritten characters are analyzed, and text data is extracted from each.

[0306] Step 3:

[0307] The server analyzes the text information extracted by OCR and identifies the key data of the contract. At this time, important items such as the title of the contract, the parties to the contract, and the conclusion date are extracted. The input is the output text of OCR, and the output is a list of these key data. The server generates the structure of the document based on this and saves it in the database.

[0308] Step 4:

[0309] When the user wants to search for specific contract information, a query in natural language form is input via the terminal. The terminal sends this query to the server. The server analyzes the query, searches the database, and identifies relevant contract documents. At this time, the input is the user query, and the output is a list of relevant contract documents.

[0310] Step 5:

[0311] The server sends the search results to the terminal. The terminal displays the results to the user in an easy-to-understand format. The user can view the displayed results and check the details as needed. A possible specific prompt would be, "Show me the transaction agreements concluded in September 2023."

[0312] Step 6:

[0313] The server analyzes the contract content in real time and evaluates risk factors. It uses a generative AI model to perform risk assessments and generates risk analysis reports as needed. The input is the contract content, and the output is the risk assessment results and report. This allows users to take appropriate measures against risks.

[0314] Step 7:

[0315] The server manages the contract renewal deadline. As the contract renewal deadline approaches, the server sends a notification to the user via the terminal. The input is the renewal deadline information, and the output is a notification message. The user can receive this notification and proceed with the appropriate renewal procedure.

[0316] Step 8:

[0317] Users can request automated contract generation from their terminal to create new contracts. The server generates a draft by combining a template with user-provided parameters. The input is user parameters, and the output is the drafted contract. This draft is sent to the terminal for the user to review and make any necessary revisions.

[0318] (Application Example 1)

[0319] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0320] In modern manufacturing, efficiently managing equipment maintenance contracts and various production line contracts is a crucial factor in reducing operational costs and improving production efficiency. However, when these contracts are managed manually, it is difficult to retrieve the necessary information at the appropriate time from a vast amount of contract data. Furthermore, mistakes such as missing contract renewal deadlines are likely to occur, potentially impacting the manufacturing process. Therefore, new methods are needed to solve these problems.

[0321] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0322] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by visual data recognition processing based on the received electronic data, and means for coordinating with a control device incorporated in the equipment to manage and notify maintenance schedules. This enables efficient management of equipment maintenance contracts and other contract information, allows maintenance to be performed at the appropriate time, and enables smooth operation of the manufacturing line.

[0323] An "information processing device" is a system that includes devices for receiving, processing, and managing electronic data, such as computers and servers.

[0324] "Visual data recognition processing" is the process of extracting textual information from image data, and is performed using OCR technology.

[0325] A "natural language query" is an interface in which a user uses ordinary language to request information from a system.

[0326] "Risk factors" refer to potential dangers and problems hidden within contract information, and evaluating these is a necessary element.

[0327] An "analysis document" is a document that summarizes the results of the risk factor assessment and presents them to users in an easy-to-understand manner.

[0328] A "control device" is a device used to manage and operate specific equipment or systems.

[0329] A "maintenance schedule" is a plan that outlines the timing and content of maintenance and inspections of equipment and machinery.

[0330] A "notification" is the act of informing a user of a specific event or piece of information, and often takes the form of an alert or reminder.

[0331] In the system that realizes this invention, the server, terminal, and user work together in coordination.

[0332] The server first receives various electronic data related to the equipment from cameras and scanning devices attached to robotic arms within the factory. The received data is then processed using OCR technology to extract textual information. Existing software such as Tesseract OCR is used for this process. The extracted information is stored in a database and attributed to contract information.

[0333] The terminal accepts natural language queries from users. This interface is designed for users to search for contract information and check maintenance schedules. The server searches the database based on these queries, efficiently extracts the relevant information, and returns it to the terminal.

[0334] Furthermore, the control unit built into the device and the server work together to perform risk assessments and automatically create maintenance schedules based on the extracted data. The assessment algorithm executed on the server identifies risk factors based on contract information and generates analytical reports. In addition, when important renewal deadlines or maintenance schedules approach, reminders are automatically sent to the terminal in the form of emails or alerts.

[0335] As a concrete example, in one factory, a maintenance contract for a specific machine is required every month. This system uses OCR technology to read the contract related to that machine, automatically calculates the next maintenance date within the system, and adds it to Google Calendar. At the same time, it sends an email notification to the person in charge five days in advance.

[0336] Examples of prompt messages include, "When is the next scheduled maintenance? Please extract the information from the contract." This system design enables more efficient contract management and supports maintenance operations.

[0337] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0338] Step 1:

[0339] The server receives electronic data about the equipment from a camera attached to a robotic arm in the factory. This process involves inputting image data captured by the camera. The server records this image data and prepares it for the next step, which involves visual data recognition processing.

[0340] Step 2:

[0341] The server performs visual data recognition (OCR) processing on the received image data to extract text information. The input is the received image data, and the output is the extracted text information. This process uses software such as Tesseract OCR to recognize both printed and handwritten characters.

[0342] Step 3:

[0343] The server assigns attributes to the contracts based on the extracted text information. In this step, the text information obtained by OCR is used as input, and the output is contract information with attributes such as contract date and maintenance details, which is stored in the database. The server then manages the necessary contract information based on this.

[0344] Step 4:

[0345] The user makes a query using natural language through the terminal. In this case, the input is a prompt such as, "When is the next maintenance scheduled?" The terminal receives this input and sends it to the server.

[0346] Step 5:

[0347] The server searches the database based on the user's query and extracts relevant contract information. The input is a natural language query from the user, and the output is the corresponding contract information. The server selects the most relevant information and sends it to the terminal.

[0348] Step 6:

[0349] The server manages maintenance schedules and generates reminders through coordination with the control devices built into the equipment. Contract information and schedule information are used as input, and reminder information is generated as output. Specifically, it calculates the next maintenance date and sets notifications for email and calendar applications.

[0350] Step 7:

[0351] The terminal displays search results and reminder information received from the server to the user. This output includes the next scheduled maintenance date and related contract information. The user can take appropriate action based on the displayed information.

[0352] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0353] The AI ​​contract management system of the present invention utilizes servers, terminals, users, and an emotion engine to automate the management of contract documents and improve the user experience.

[0354] First, the server receives electronic files of contracts sent by the user. These include PDF and image files. The server performs visual character recognition on these files to extract important information from the contract. This information includes the contract title, signatories, and signing date.

[0355] Next, the server associates the extracted information and generates a document structure. In this process, the sentiment engine considers the user's past preferences and current emotional state to help with optimal association. As a result, document management becomes faster and more accurate.

[0356] The terminal receives search queries from the user in natural language and sends them to the server. The server searches the database based on these queries to identify the most suitable contract documents. Simultaneously, an emotion engine analyzes the user's emotional state and adjusts how search results are displayed and the feedback messages. For example, if the user is stressed, the system helps the user quickly understand the information by summarizing the search results concisely.

[0357] The server also takes emotion engine data into account when performing a risk assessment of the contract and generating an analysis report based on the identified risk factors. When the user is relaxed, a detailed analysis report is provided to allow the user to carefully consider its contents.

[0358] Furthermore, the server monitors contract renewal deadlines and sends reminder notifications via the device as the deadline approaches. The content of the notifications is also adjusted based on insights from the emotion engine, taking care to ensure that users can manage their contracts while maintaining positive emotions.

[0359] Finally, based on the request to generate a contract document, the server generates a document template. At this time, the emotion engine checks the user's current state and helps ensure that the generated document matches the user's requirements.

[0360] In this way, the AI ​​contract management system, by combining an emotion engine, provides an optimized contract management experience for users, aiming to improve operational efficiency and satisfaction.

[0361] The following describes the processing flow.

[0362] Step 1:

[0363] Users use their devices to upload electronic files of contracts to the system. These files include PDF and image formats.

[0364] Step 2:

[0365] The server receives the uploaded contract and uses OCR technology to extract text from the document. This process recognizes both handwritten and printed characters with high accuracy.

[0366] Step 3:

[0367] The server identifies key information such as contract title, signatory, and signing date based on the extracted text data, and stores this information in a database.

[0368] Step 4:

[0369] The server automatically associates information stored in the database with other similar contract documents and generates a document structure. An emotion engine also operates, taking user sentiment into consideration to optimize the associations.

[0370] Step 5:

[0371] The terminal receives a natural language search query from the user, sends it to the server, and starts the search.

[0372] Step 6:

[0373] The server searches the database based on the received query to find the most relevant contract documents. During this process, the sentiment engine evaluates the user's emotional state and adjusts how the search results are presented.

[0374] Step 7:

[0375] The server analyzes the contract details and, if risk factors are identified, generates an analysis report. Using information from the sentiment engine, the report is presented in a user-friendly format.

[0376] Step 8:

[0377] The server monitors the contract renewal deadline and sets a reminder when the deadline approaches. The terminal sends emotionally adaptive reminder notifications to the user.

[0378] Step 9:

[0379] The user requests the automatic generation of a contract document through their terminal. The server uses a generation AI based on the conditions specified by the user to create a contract template.

[0380] Step 10:

[0381] The server uses an emotion engine to analyze the user's state and configures document generation and display to enhance user satisfaction. Users can review the generated contract on their device and make any necessary modifications.

[0382] (Example 2)

[0383] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0384] In modern information management systems, managing electronic data has become increasingly complex, and efficient and effective processing is required, especially for critical data such as contract documents. Furthermore, flexible functions that can adaptively manage data according to the user's emotional state are also necessary. However, conventional systems have struggled to integrate such flexibility with advanced information extraction and management functions.

[0385] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0386] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by optical character recognition processing, and means for creating a data structure by associating data. This enables efficient management of contract documents while providing nuanced responses based on the user's emotions.

[0387] A "data processing device" is a system that combines hardware and software for efficiently managing and processing electronic data.

[0388] "Electronic data" refers to all information that is expressed in digital format and can be processed on a computer.

[0389] "Optical character recognition processing" is a technology that identifies characters and numbers from electronic data in image format and extracts them as text data.

[0390] "Relationship building" refers to the process of analyzing the relationships between extracted information and organizing them in an integrated manner.

[0391] A "data structure" is a method of organizing information in a specific format to enable efficient processing and storage of that information.

[0392] "Emotional state" refers to the user's current emotions and psychological state, and is a factor that influences the system's operation and response.

[0393] "Reporting" refers to the action of providing users with important information or notifications regarding deadlines.

[0394] A "data template" is a standardized format that serves as the basis for generating electronic data such as contract documents.

[0395] The data processing device of the present invention automates the management of contract documents using a server, terminals, users, and an engine that performs sentiment analysis. Its embodiments are described in detail below.

[0396] First, the server receives contract documents sent electronically by the user. These documents are typically in PDF or image format. The server then uses optical character recognition (OCR) software, such as Tesseract, to extract text data from these documents. This digitizes paper-based documents, making them electronically processable.

[0397] Next, the server analyzes the extracted text data to identify key information such as the contract title, the names of the parties, and the signing date. Based on this information, the server generates the data structure for the contract document. At this stage, the sentiment analysis engine takes into account the user's past preferences and current emotional state to improve the accuracy of the associations.

[0398] The terminal receives natural language prompts from the user and sends them to the server. An example of a prompt might be, "Show me the latest contract." The server uses this to search its database and identify the relevant contract document. The server analyzes the user's emotional state through sentiment analysis and adjusts how the results are displayed. For example, if the user is feeling stressed, the information may be visually organized to aid understanding.

[0399] Furthermore, the server evaluates the risk factors of the contract and, if a detailed analysis is required, generates an analytical report that reflects the results of sentiment detection. Depending on the user's emotional state, they can choose to receive a detailed or summary report.

[0400] Finally, the server monitors the contract document's renewal deadline, and when the deadline approaches, a notification is sent to the user via their device. The notification's wording is adjusted according to the user's emotional state to ensure a positive user experience. Furthermore, if the user wishes to create a new contract document, the server generates a template based on the sentiment analysis results, providing the document that best suits the user's needs.

[0401] In this way, the data processing device of the present invention highly automates contract document management and enables the provision of individually adapted information by taking into account the user's emotions.

[0402] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0403] Step 1:

[0404] The server receives contract documents from the user as electronic data. This input includes PDFs and image files. The received electronic data is processed by OCR software (e.g., Tesseract) to extract text data from the visual data. The output of this process is used as text data in subsequent processing.

[0405] Step 2:

[0406] The server analyzes the text data extracted by OCR. It identifies important metadata such as contract title, signatory, and signing date from the input text data and records it in the database. In this step, an information identification algorithm is applied to extract the important information, which is then output as structured data.

[0407] Step 3:

[0408] The server generates a document structure based on the identified metadata. In this process, metadata, the user's past preferences, and current sentiment state are used as input data, and the sentiment analysis engine improves the accuracy of the associations. The output of this process is a well-organized data structure, which is used for subsequent searching and management.

[0409] Step 4:

[0410] The terminal receives natural language prompts from the user and sends them to the server. An example of an input prompt is "Show me the latest contract." The server parses this prompt and searches its database based on the prompt it received as input. It identifies the most relevant contract document and formats it in the optimal way for the user to see it.

[0411] Step 5:

[0412] The server analyzes contract information and assesses potential risk factors from the input data. It then generates an analytical report that reflects the output of the sentiment analysis engine. The report generated through this process is provided in either a detailed or concise format, flexibly adapting to the user's needs.

[0413] Step 6:

[0414] The server monitors the renewal deadlines for contract documents and sends notifications to users via their terminals as the deadline approaches. Based on the input monitoring data, it creates notifications and outputs messages that reflect the results of sentiment analysis. This improves user responsiveness.

[0415] Step 7:

[0416] When a user requests the generation of a new contract document, the server creates the document using a template. This input includes the user's requests and emotional state, and the server generates the most suitable document based on this information. The resulting document is tailored to the user's needs, thereby improving operational efficiency.

[0417] (Application Example 2)

[0418] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0419] Traditional contract management systems have struggled to efficiently manage the content of contract documents and provide appropriate feedback while considering the user's emotional state. Furthermore, there has been a lack of sufficient methods to automate the risk assessment of contract content and reduce the burden on users. Therefore, there is a need to improve the efficiency of contract processing and optimize the user experience.

[0420] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0421] In this invention, the server includes means for performing processing to receive electronic data, means for acquiring information by visual character recognition based on the received data, means for generating related information based on the acquired information, means for receiving instructions in natural language and searching for related electronic data, means for evaluating risks and generating analysis reports based on the analyzed information, means for monitoring data update deadlines and providing notifications when the deadline approaches, means for generating data templates based on document generation instructions, and means for adjusting feedback based on user status. This streamlines the entire contract management process and enables the provision of appropriate feedback that meets user needs.

[0422] "Electronic data" refers to information recorded in digital format, and examples include contract documents and document files.

[0423] "Visual character recognition" is a technology that reads characters from images, PDF files, and other sources and extracts them as text information.

[0424] "Generating related information" means organizing and integrating necessary information based on acquired data to form new, meaningful information.

[0425] "Natural language instructions" refer to instructions and questions based on the language that humans use in everyday life, with the aim of information processing systems understanding and responding to them.

[0426] "Assessing risks and generating an analysis report" is the process of identifying potential problems and risks based on the information obtained and creating a report based on that information.

[0427] "Monitoring and notifying data expiration dates" refers to a function that tracks the expiration dates of data and information held and notifies users when the expiration date is approaching.

[0428] "Generating data templates based on document generation instructions" refers to a function that creates the basic structure and format of necessary documents according to user instructions.

[0429] "Adjusting feedback based on user status" is the process of optimizing the content of information and advice provided by taking into account the user's emotions and circumstances.

[0430] The system for realizing this invention consists mainly of a central server and user terminals. The server receives electronic data transmitted from the user and extracts the necessary information using visual character recognition technology. Software such as OpenCV and Pytesseract is used for this technology. The extracted information is structured in combination with other related information and stored in a database.

[0431] Instructions in natural language are sent to the server via the terminal. The server understands them, searches for the appropriate electronic data, and returns the results. Generative AI models are used in this process to analyze the user's emotional state. For example, if the user is in a hurry, it provides concise feedback; if they are relaxed, it presents a detailed analytical report.

[0432] Furthermore, the server assesses the risks in contract data and generates an analysis report. This helps users make informed decisions by identifying potential risks in the contract and communicating them to them. It also monitors data update deadlines and notifies users via their devices when the deadline approaches. This notification is also delivered in an optimal way, taking into account the user's emotional state.

[0433] When a document is requested to be generated, the server creates a template based on the specified format. In this process, it can reflect the user's emotional state and generate the document in the most appropriate form.

[0434] For example, when handling electronic contracts with construction companies, the server extracts contract details using visual character recognition and performs a risk assessment. Depending on the user's situation, it may provide a concise notification such as, "There is a high risk to your budget," or a more detailed notification such as, "There is a high possibility of exceeding your budget, so please consider additional cost management measures."

[0435] In this way, the system dynamically adjusts feedback based on the user's emotional state, not only increasing the efficiency of contract management but also improving the user experience. For example, a prompt might say, "Assess the risks of electronic contracts with construction companies and generate a concise report if the user is in a hurry, or a detailed report otherwise."

[0436] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0437] Step 1:

[0438] The user sends electronic data to the server using their device. Input can include contract data in PDF or image format. Once this data reaches the server, data reception is complete.

[0439] Step 2:

[0440] The server performs visual character recognition processing on the received electronic data. Specifically, it uses OpenCV and Pytesseract to extract text information from the image data. The input is image data, and the output is the extracted text information.

[0441] Step 3:

[0442] The server generates contract-related information based on the extracted text information. This process involves organizing the information and shaping its related structure. The input is the acquired text information, and the output is the associated contract information.

[0443] Step 4:

[0444] The user sends instructions in natural language to the server using a terminal. The server then analyzes the user's instructions and searches for relevant electronic data. The input is the user's instructions, and the output is the search results data.

[0445] Step 5:

[0446] The server uses a generative AI model to perform a risk assessment based on the search results data and generates an analysis report. At this stage, the user's emotional state is analyzed, and the output report is adjusted accordingly. The input is the search results data, and the output is the adjusted analysis report.

[0447] Step 6:

[0448] The server monitors the data update deadline and sends a notification to the user's terminal when the deadline approaches. The prompt message used is "Assess the risks of the electronic contract with the construction company...". The input is the current date and time and data update information, and the output is a notification based on the conditions.

[0449] Step 7:

[0450] The server generates a data template based on the user's instructions for document creation. In this process, the user's emotional state is reflected, and the document is created in the most optimal format. The input is the user's instructions and emotional state, and the output is the generated document template.

[0451] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0452] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0453] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0454] [Third Embodiment]

[0455] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0456] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0457] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0458] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0459] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0460] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0461] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0462] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0463] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0464] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0465] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0466] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0467] The AI ​​contract management system of the present invention involves a server, terminals, and users working together to perform a series of processes for automated contract document management.

[0468] The server first receives PDF or image files of contracts uploaded by users. The server then performs optical character recognition (OCR) processing on the received files to extract text information from the documents. This process utilizes OCR technology to accurately read both printed and handwritten text.

[0469] The extracted information consists of key data such as contract title, signatory, and signing date. The server uses this data to create associations and generate a document structure. The server also stores the information in a database to prepare for efficient search processing later.

[0470] The terminal receives search queries from the user in natural language format through its interface and sends them to the server. The server searches the database based on the query, identifies the most relevant contract documents, and returns the results to the terminal. The terminal displays the search results in an easy-to-understand format for the user.

[0471] Furthermore, the server analyzes the contract details in real time, evaluates the risk factors included in the contract, and generates an analysis report as needed. Based on this risk analysis, it becomes easier for users to take countermeasures.

[0472] The server also monitors the set renewal deadline and sends reminder notifications to the user via their device as the deadline approaches. This ensures that the user does not miss important contract renewal deadlines.

[0473] Users can request the automatic generation of contract documents via their terminal as needed. The server combines contract templates based on specified parameters and automatically generates a draft. The user can then review the draft on their terminal and input any necessary revisions or additions.

[0474] In this way, the AI ​​contract management system achieves efficient and accurate contract management through a series of automated processes, significantly reducing the workload of users.

[0475] The following describes the processing flow.

[0476] Step 1:

[0477] The server receives PDF and image files of contracts uploaded by users. These files contain important contract information.

[0478] Step 2:

[0479] The server performs visual character recognition (OCR) processing on the received file, extracting text information from the document using OCR technology. This process accurately reads not only printed text but also handwritten characters.

[0480] Step 3:

[0481] The server analyzes the information extracted by OCR to identify important contract data such as contract title, signatory, and signing date, and stores the results in a database.

[0482] Step 4:

[0483] The server automatically associates information stored in the database and generates a document structure by combining similar contract information. This improves the overall traceability of the document.

[0484] Step 5:

[0485] The terminal sends the natural language search query received from the user to the server, and the server retrieves relevant information corresponding to the query from the database.

[0486] Step 6:

[0487] The server aggregates the search results, identifies the most relevant contract documents, and returns them to the terminal. The terminal then clearly displays these search results to the user.

[0488] Step 7:

[0489] The server analyzes the contract details and generates an analysis report if it identifies specific risk factors. This report is saved so that the user can review it later.

[0490] Step 8:

[0491] The server monitors the renewal deadlines for each contract and sets reminders as the deadline approaches. The terminal then notifies the user of important contract renewals.

[0492] Step 9:

[0493] The user requests the creation of a new contract document from the server via their terminal. Based on the provided conditions, the server automatically generates a draft contract using a generation AI.

[0494] Step 10:

[0495] The user reviews the contract generated on their device and makes corrections as needed. The server then securely stores the final, corrected version.

[0496] (Example 1)

[0497] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0498] Traditional contract document management systems often required manual, complex searches and risk assessments, leading to inefficiencies. Furthermore, managing contract renewal deadlines was cumbersome, increasing the risk of missing renewal deadlines. These challenges need to be addressed to streamline contract management and improve accuracy.

[0499] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0500] This invention includes a server that receives and stores uploaded contracts from users in an organized manner, a server that extracts information from printed text and handwritten characters within electronic documents using OCR technology, and a server that analyzes the extracted information to generate key data such as contract name, signatories, and date and time as a document structure. This enables automated contract management, efficient searching, risk assessment, and deadline management.

[0501] A "user" is someone who uses a terminal to upload contracts or enter search queries into the system.

[0502] A "server" is a central processing unit that receives information from users, analyzes the data using OCR technology, and performs tasks such as storing, searching, displaying, and risk assessment of contract documents.

[0503] "OCR technology" refers to optical character recognition technology, which converts printed text and handwritten characters in electronic documents and images into digital data.

[0504] A "contract document" is an official document that records the details of a transaction or agreement, and is subject to renewal and risk assessment.

[0505] "Risk assessment" is the process of analyzing the contents of a contract document to identify and evaluate factors that could potentially cause problems in the future.

[0506] "Deadline management" is a function that monitors contract renewal deadlines and notifies users to ensure they don't miss the renewal deadline.

[0507] A "draft" is a template for a contract document, a draft automatically generated based on user requests.

[0508] This invention provides an AI-powered contract management system that enables efficient contract management through automated processes. This system operates through the collaboration of servers, terminals, and users.

[0509] The server first receives PDF or image files of contracts uploaded by the user via their device. At this point, the server uses open-source optical character recognition tools such as Tesseract OCR to convert printed and handwritten text within the document into digital data. This technology is used to accurately extract important information.

[0510] From the extracted information, the server extracts key data such as the contract title, signatory, and signing date, and stores this as structured data. This improves the efficiency of association and searching. The information is also stored in a database, and an index is generated for easy searching.

[0511] The terminal receives natural language search queries from the user and sends them to the server. The server searches the database based on the received queries and identifies relevant contract documents. The results are sent to the terminal and made available for the user to view. For example, a prompt such as "Show me the transaction agreements concluded in September 2023" can be used.

[0512] Furthermore, the server analyzes the contract content in real time and assesses potential risk factors. This analysis uses a generative AI model, and based on the evaluation results, it generates and provides an analysis report to the user.

[0513] The contract renewal deadline is monitored by the server, and users are notified via their devices as the deadline approaches. Based on this notification, users can renew their contracts in a timely manner.

[0514] Furthermore, if a user wishes to create a new contract, they can request automatic contract generation through their terminal. The server generates a draft according to pre-prepared templates and user-specified parameters, and the user can review and modify the draft on their terminal.

[0515] These features enable the AI ​​contract management system to improve the efficiency and accuracy of contract management, significantly reducing the workload for users.

[0516] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0517] Step 1:

[0518] The user uploads a PDF or image file of the contract to the system using a terminal. This becomes the input data. The terminal sends the uploaded data to the server, and the data reception process begins. The server temporarily stores this uploaded document in its storage.

[0519] Step 2:

[0520] The server performs OCR processing on the received contract. Here, OCR software is used to recognize text within the image. The input is an image of the contract, and the output is recognized text information. This process analyzes both printed and handwritten characters, extracting text data from each.

[0521] Step 3:

[0522] The server analyzes the text information extracted by OCR to identify key contract data. Important items such as the contract title, signatories, and signing date are extracted. The input is the OCR output text, and the output is a list of this key data. Based on this, the server generates the document structure and stores it in a database.

[0523] Step 4:

[0524] When a user wants to search for specific contract information, they enter a query in natural language format via a terminal. The terminal sends this query to the server. The server parses the query, searches the database, and identifies the relevant contract documents. In this case, the input is the user query, and the output is a list of relevant contract documents.

[0525] Step 5:

[0526] The server sends the search results to the terminal. The terminal displays the results to the user in an easy-to-understand format. The user can view the displayed results and check the details as needed. A possible specific prompt would be, "Show me the transaction agreements concluded in September 2023."

[0527] Step 6:

[0528] The server analyzes the contract content in real time and evaluates risk factors. It uses a generative AI model to perform risk assessments and generates risk analysis reports as needed. The input is the contract content, and the output is the risk assessment results and report. This allows users to take appropriate measures against risks.

[0529] Step 7:

[0530] The server manages the contract renewal deadline. As the contract renewal deadline approaches, the server sends a notification to the user via the terminal. The input is the renewal deadline information, and the output is a notification message. The user can receive this notification and proceed with the appropriate renewal procedure.

[0531] Step 8:

[0532] Users can request automated contract generation from their terminal to create new contracts. The server generates a draft by combining a template with user-provided parameters. The input is user parameters, and the output is the drafted contract. This draft is sent to the terminal for the user to review and make any necessary revisions.

[0533] (Application Example 1)

[0534] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0535] In modern manufacturing, efficiently managing equipment maintenance contracts and various production line contracts is a crucial factor in reducing operational costs and improving production efficiency. However, when these contracts are managed manually, it is difficult to retrieve the necessary information at the appropriate time from a vast amount of contract data. Furthermore, mistakes such as missing contract renewal deadlines are likely to occur, potentially impacting the manufacturing process. Therefore, new methods are needed to solve these problems.

[0536] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0537] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by visual data recognition processing based on the received electronic data, and means for coordinating with a control device incorporated in the equipment to manage and notify maintenance schedules. This enables efficient management of equipment maintenance contracts and other contract information, allows maintenance to be performed at the appropriate time, and enables smooth operation of the manufacturing line.

[0538] An "information processing device" is a system that includes devices for receiving, processing, and managing electronic data, such as computers and servers.

[0539] "Visual data recognition processing" is the process of extracting textual information from image data, and is performed using OCR technology.

[0540] A "natural language query" is an interface in which a user uses ordinary language to request information from a system.

[0541] "Risk factors" refer to potential dangers and problems hidden within contract information, and evaluating these is a necessary element.

[0542] An "analysis document" is a document that summarizes the results of the risk factor assessment and presents them to users in an easy-to-understand manner.

[0543] A "control device" is a device used to manage and operate specific equipment or systems.

[0544] A "maintenance schedule" is a plan that outlines the timing and content of maintenance and inspections of equipment and machinery.

[0545] A "notification" is the act of informing a user of a specific event or piece of information, and often takes the form of an alert or reminder.

[0546] In the system that realizes this invention, the server, terminal, and user work together in coordination.

[0547] The server first receives various electronic data related to the equipment from cameras and scanning devices attached to robotic arms within the factory. The received data is then processed using OCR technology to extract textual information. Existing software such as Tesseract OCR is used for this process. The extracted information is stored in a database and attributed to contract information.

[0548] The terminal accepts natural language queries from users. This interface is designed for users to search for contract information and check maintenance schedules. The server searches the database based on these queries, efficiently extracts the relevant information, and returns it to the terminal.

[0549] Furthermore, the control unit built into the device and the server work together to perform risk assessments and automatically create maintenance schedules based on the extracted data. The assessment algorithm executed on the server identifies risk factors based on contract information and generates analytical reports. In addition, when important renewal deadlines or maintenance schedules approach, reminders are automatically sent to the terminal in the form of emails or alerts.

[0550] As a concrete example, in one factory, a maintenance contract for a specific machine is required every month. This system uses OCR technology to read the contract related to that machine, automatically calculates the next maintenance date within the system, and adds it to Google Calendar. At the same time, it sends an email notification to the person in charge five days in advance.

[0551] Examples of prompt messages include, "When is the next scheduled maintenance? Please extract the information from the contract." This system design enables more efficient contract management and supports maintenance operations.

[0552] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0553] Step 1:

[0554] The server receives electronic data about the equipment from a camera attached to a robotic arm in the factory. This process involves inputting image data captured by the camera. The server records this image data and prepares it for the next step, which involves visual data recognition processing.

[0555] Step 2:

[0556] The server performs visual data recognition (OCR) processing on the received image data to extract text information. The input is the received image data, and the output is the extracted text information. This process uses software such as Tesseract OCR to recognize both printed and handwritten characters.

[0557] Step 3:

[0558] The server assigns attributes to the contracts based on the extracted text information. In this step, the text information obtained by OCR is used as input, and the output is contract information with attributes such as contract date and maintenance details, which is stored in the database. The server then manages the necessary contract information based on this.

[0559] Step 4:

[0560] The user makes a query using natural language through the terminal. In this case, the input is a prompt such as, "When is the next maintenance scheduled?" The terminal receives this input and sends it to the server.

[0561] Step 5:

[0562] The server searches the database based on the user's query and extracts relevant contract information. The input is a natural language query from the user, and the output is the corresponding contract information. The server selects the most relevant information and sends it to the terminal.

[0563] Step 6:

[0564] The server manages maintenance schedules and generates reminders through coordination with the control devices built into the equipment. Contract information and schedule information are used as input, and reminder information is generated as output. Specifically, it calculates the next maintenance date and sets notifications for email and calendar applications.

[0565] Step 7:

[0566] The terminal displays search results and reminder information received from the server to the user. This output includes the next scheduled maintenance date and related contract information. The user can take appropriate action based on the displayed information.

[0567] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0568] The AI ​​contract management system of the present invention utilizes servers, terminals, users, and an emotion engine to automate the management of contract documents and improve the user experience.

[0569] First, the server receives electronic files of contracts sent by the user. These include PDF and image files. The server performs visual character recognition on these files to extract important information from the contract. This information includes the contract title, signatories, and signing date.

[0570] Next, the server associates the extracted information and generates a document structure. In this process, the sentiment engine considers the user's past preferences and current emotional state to help with optimal association. As a result, document management becomes faster and more accurate.

[0571] The terminal receives search queries from the user in natural language and sends them to the server. The server searches the database based on these queries to identify the most suitable contract documents. Simultaneously, an emotion engine analyzes the user's emotional state and adjusts how search results are displayed and the feedback messages. For example, if the user is stressed, the system helps the user quickly understand the information by summarizing the search results concisely.

[0572] The server also takes emotion engine data into account when performing a risk assessment of the contract and generating an analysis report based on the identified risk factors. When the user is relaxed, a detailed analysis report is provided to allow the user to carefully consider its contents.

[0573] Furthermore, the server monitors contract renewal deadlines and sends reminder notifications via the device as the deadline approaches. The content of the notifications is also adjusted based on insights from the emotion engine, taking care to ensure that users can manage their contracts while maintaining positive emotions.

[0574] Finally, based on the request to generate a contract document, the server generates a document template. At this time, the emotion engine checks the user's current state and helps ensure that the generated document matches the user's requirements.

[0575] In this way, the AI ​​contract management system, by combining an emotion engine, provides an optimized contract management experience for users, aiming to improve operational efficiency and satisfaction.

[0576] The following describes the processing flow.

[0577] Step 1:

[0578] Users use their devices to upload electronic files of contracts to the system. These files include PDF and image formats.

[0579] Step 2:

[0580] The server receives the uploaded contract and uses OCR technology to extract text from the document. This process recognizes both handwritten and printed characters with high accuracy.

[0581] Step 3:

[0582] The server identifies key information such as contract title, signatory, and signing date based on the extracted text data, and stores this information in a database.

[0583] Step 4:

[0584] The server automatically associates information stored in the database with other similar contract documents and generates a document structure. An emotion engine also operates, taking user sentiment into consideration to optimize the associations.

[0585] Step 5:

[0586] The terminal receives a natural language search query from the user, sends it to the server, and starts the search.

[0587] Step 6:

[0588] The server searches the database based on the received query to find the most relevant contract documents. During this process, the sentiment engine evaluates the user's emotional state and adjusts how the search results are presented.

[0589] Step 7:

[0590] The server analyzes the contract details and, if risk factors are identified, generates an analysis report. Using information from the sentiment engine, the report is presented in a user-friendly format.

[0591] Step 8:

[0592] The server monitors the contract renewal deadline and sets a reminder when the deadline approaches. The terminal sends emotionally adaptive reminder notifications to the user.

[0593] Step 9:

[0594] The user requests the automatic generation of a contract document through their terminal. The server uses a generation AI based on the conditions specified by the user to create a contract template.

[0595] Step 10:

[0596] The server uses an emotion engine to analyze the user's state and configures document generation and display to enhance user satisfaction. Users can review the generated contract on their device and make any necessary modifications.

[0597] (Example 2)

[0598] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0599] In modern information management systems, managing electronic data has become increasingly complex, and efficient and effective processing is required, especially for critical data such as contract documents. Furthermore, flexible functions that can adaptively manage data according to the user's emotional state are also necessary. However, conventional systems have struggled to integrate such flexibility with advanced information extraction and management functions.

[0600] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0601] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by optical character recognition processing, and means for creating a data structure by associating data. This enables efficient management of contract documents while providing nuanced responses based on the user's emotions.

[0602] A "data processing device" is a system that combines hardware and software for efficiently managing and processing electronic data.

[0603] "Electronic data" refers to all information that is expressed in digital format and can be processed on a computer.

[0604] "Optical character recognition processing" is a technology that identifies characters and numbers from electronic data in image format and extracts them as text data.

[0605] "Relationship building" refers to the process of analyzing the relationships between extracted information and organizing them in an integrated manner.

[0606] A "data structure" is a method of organizing information in a specific format to enable efficient processing and storage of that information.

[0607] "Emotional state" refers to the user's current emotions and psychological state, and is a factor that influences the system's operation and response.

[0608] "Reporting" refers to the action of providing users with important information or notifications regarding deadlines.

[0609] A "data template" is a standardized format that serves as the basis for generating electronic data such as contract documents.

[0610] The data processing device of the present invention automates the management of contract documents using a server, terminals, users, and an engine that performs sentiment analysis. Its embodiments are described in detail below.

[0611] First, the server receives contract documents sent electronically by the user. These documents are typically in PDF or image format. The server then uses optical character recognition (OCR) software, such as Tesseract, to extract text data from these documents. This digitizes paper-based documents, making them electronically processable.

[0612] Next, the server analyzes the extracted text data to identify key information such as the contract title, the names of the parties, and the signing date. Based on this information, the server generates the data structure for the contract document. At this stage, the sentiment analysis engine takes into account the user's past preferences and current emotional state to improve the accuracy of the associations.

[0613] The terminal receives natural language prompts from the user and sends them to the server. An example of a prompt might be, "Show me the latest contract." The server uses this to search its database and identify the relevant contract document. The server analyzes the user's emotional state through sentiment analysis and adjusts how the results are displayed. For example, if the user is feeling stressed, the information may be visually organized to aid understanding.

[0614] Furthermore, the server evaluates the risk factors of the contract and, if a detailed analysis is required, generates an analytical report that reflects the results of sentiment detection. Depending on the user's emotional state, they can choose to receive a detailed or summary report.

[0615] Finally, the server monitors the contract document's renewal deadline, and when the deadline approaches, a notification is sent to the user via their device. The notification's wording is adjusted according to the user's emotional state to ensure a positive user experience. Furthermore, if the user wishes to create a new contract document, the server generates a template based on the sentiment analysis results, providing the document that best suits the user's needs.

[0616] In this way, the data processing device of the present invention highly automates contract document management and enables the provision of individually adapted information by taking into account the user's emotions.

[0617] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0618] Step 1:

[0619] The server receives contract documents from the user as electronic data. This input includes PDFs and image files. The received electronic data is processed by OCR software (e.g., Tesseract) to extract text data from the visual data. The output of this process is used as text data in subsequent processing.

[0620] Step 2:

[0621] The server analyzes the text data extracted by OCR. It identifies important metadata such as contract title, signatory, and signing date from the input text data and records it in the database. In this step, an information identification algorithm is applied to extract the important information, which is then output as structured data.

[0622] Step 3:

[0623] The server generates a document structure based on the identified metadata. In this process, metadata, the user's past preferences, and current sentiment state are used as input data, and the sentiment analysis engine improves the accuracy of the associations. The output of this process is a well-organized data structure, which is used for subsequent searching and management.

[0624] Step 4:

[0625] The terminal receives natural language prompts from the user and sends them to the server. An example of an input prompt is "Show me the latest contract." The server parses this prompt and searches its database based on the prompt it received as input. It identifies the most relevant contract document and formats it in the optimal way for the user to see it.

[0626] Step 5:

[0627] The server analyzes contract information and assesses potential risk factors from the input data. It then generates an analytical report that reflects the output of the sentiment analysis engine. The report generated through this process is provided in either a detailed or concise format, flexibly adapting to the user's needs.

[0628] Step 6:

[0629] The server monitors the renewal deadlines for contract documents and sends notifications to users via their terminals as the deadline approaches. Based on the input monitoring data, it creates notifications and outputs messages that reflect the results of sentiment analysis. This improves user responsiveness.

[0630] Step 7:

[0631] When a user requests the generation of a new contract document, the server creates the document using a template. This input includes the user's requests and emotional state, and the server generates the most suitable document based on this information. The resulting document is tailored to the user's needs, thereby improving operational efficiency.

[0632] (Application Example 2)

[0633] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0634] Traditional contract management systems have struggled to efficiently manage the content of contract documents and provide appropriate feedback while considering the user's emotional state. Furthermore, there has been a lack of sufficient methods to automate the risk assessment of contract content and reduce the burden on users. Therefore, there is a need to improve the efficiency of contract processing and optimize the user experience.

[0635] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0636] In this invention, the server includes means for performing processing to receive electronic data, means for acquiring information by visual character recognition based on the received data, means for generating related information based on the acquired information, means for receiving instructions in natural language and searching for related electronic data, means for evaluating risks and generating analysis reports based on the analyzed information, means for monitoring data update deadlines and providing notifications when the deadline approaches, means for generating data templates based on document generation instructions, and means for adjusting feedback based on user status. This streamlines the entire contract management process and enables the provision of appropriate feedback that meets user needs.

[0637] "Electronic data" refers to information recorded in digital format, and examples include contract documents and document files.

[0638] "Visual character recognition" is a technology that reads characters from images, PDF files, and other sources and extracts them as text information.

[0639] "Generating related information" means organizing and integrating necessary information based on acquired data to form new, meaningful information.

[0640] "Natural language instructions" refer to instructions and questions based on the language that humans use in everyday life, with the aim of information processing systems understanding and responding to them.

[0641] "Assessing risks and generating an analysis report" is the process of identifying potential problems and risks based on the information obtained and creating a report based on that information.

[0642] "Monitoring and notifying data expiration dates" refers to a function that tracks the expiration dates of data and information held and notifies users when the expiration date is approaching.

[0643] "Generating data templates based on document generation instructions" refers to a function that creates the basic structure and format of necessary documents according to user instructions.

[0644] "Adjusting feedback based on user status" is the process of optimizing the content of information and advice provided by taking into account the user's emotions and circumstances.

[0645] The system for realizing this invention consists mainly of a central server and user terminals. The server receives electronic data transmitted from the user and extracts the necessary information using visual character recognition technology. Software such as OpenCV and Pytesseract is used for this technology. The extracted information is structured in combination with other related information and stored in a database.

[0646] Instructions in natural language are sent to the server via the terminal. The server understands them, searches for the appropriate electronic data, and returns the results. Generative AI models are used in this process to analyze the user's emotional state. For example, if the user is in a hurry, it provides concise feedback; if they are relaxed, it presents a detailed analytical report.

[0647] Furthermore, the server assesses the risks in contract data and generates an analysis report. This helps users make informed decisions by identifying potential risks in the contract and communicating them to them. It also monitors data update deadlines and notifies users via their devices when the deadline approaches. This notification is also delivered in an optimal way, taking into account the user's emotional state.

[0648] When a document is requested to be generated, the server creates a template based on the specified format. In this process, it can reflect the user's emotional state and generate the document in the most appropriate form.

[0649] For example, when handling electronic contracts with construction companies, the server extracts contract details using visual character recognition and performs a risk assessment. Depending on the user's situation, it may provide a concise notification such as, "There is a high risk to your budget," or a more detailed notification such as, "There is a high possibility of exceeding your budget, so please consider additional cost management measures."

[0650] In this way, the system dynamically adjusts feedback based on the user's emotional state, not only increasing the efficiency of contract management but also improving the user experience. For example, a prompt might say, "Assess the risks of electronic contracts with construction companies and generate a concise report if the user is in a hurry, or a detailed report otherwise."

[0651] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0652] Step 1:

[0653] The user sends electronic data to the server using their device. Input can include contract data in PDF or image format. Once this data reaches the server, data reception is complete.

[0654] Step 2:

[0655] The server performs visual character recognition processing on the received electronic data. Specifically, it uses OpenCV and Pytesseract to extract text information from the image data. The input is image data, and the output is the extracted text information.

[0656] Step 3:

[0657] The server generates contract-related information based on the extracted text information. This process involves organizing the information and shaping its related structure. The input is the acquired text information, and the output is the associated contract information.

[0658] Step 4:

[0659] The user sends instructions in natural language to the server using a terminal. The server then analyzes the user's instructions and searches for relevant electronic data. The input is the user's instructions, and the output is the search results data.

[0660] Step 5:

[0661] The server uses a generative AI model to perform a risk assessment based on the search results data and generates an analysis report. At this stage, the user's emotional state is analyzed, and the output report is adjusted accordingly. The input is the search results data, and the output is the adjusted analysis report.

[0662] Step 6:

[0663] The server monitors the data update deadline and sends a notification to the user's terminal when the deadline approaches. The prompt message used is "Assess the risks of the electronic contract with the construction company...". The input is the current date and time and data update information, and the output is a notification based on the conditions.

[0664] Step 7:

[0665] The server generates a data template based on the user's instructions for document creation. In this process, the user's emotional state is reflected, and the document is created in the most optimal format. The input is the user's instructions and emotional state, and the output is the generated document template.

[0666] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0667] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0668] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0669] [Fourth Embodiment]

[0670] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0671] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0672] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0673] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0674] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0675] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0676] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0677] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0678] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0679] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0680] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0681] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0682] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0683] The AI ​​contract management system of the present invention involves a server, terminals, and users working together to perform a series of processes for automated contract document management.

[0684] The server first receives PDF or image files of contracts uploaded by users. The server then performs optical character recognition (OCR) processing on the received files to extract text information from the documents. This process utilizes OCR technology to accurately read both printed and handwritten text.

[0685] The extracted information consists of key data such as contract title, signatory, and signing date. The server uses this data to create associations and generate a document structure. The server also stores the information in a database to prepare for efficient search processing later.

[0686] The terminal receives search queries from the user in natural language format through its interface and sends them to the server. The server searches the database based on the query, identifies the most relevant contract documents, and returns the results to the terminal. The terminal displays the search results in an easy-to-understand format for the user.

[0687] Furthermore, the server analyzes the contract details in real time, evaluates the risk factors included in the contract, and generates an analysis report as needed. Based on this risk analysis, it becomes easier for users to take countermeasures.

[0688] The server also monitors the set renewal deadline and sends reminder notifications to the user via their device as the deadline approaches. This ensures that the user does not miss important contract renewal deadlines.

[0689] Users can request the automatic generation of contract documents via their terminal as needed. The server combines contract templates based on specified parameters and automatically generates a draft. The user can then review the draft on their terminal and input any necessary revisions or additions.

[0690] In this way, the AI ​​contract management system achieves efficient and accurate contract management through a series of automated processes, significantly reducing the workload of users.

[0691] The following describes the processing flow.

[0692] Step 1:

[0693] The server receives PDF and image files of contracts uploaded by users. These files contain important contract information.

[0694] Step 2:

[0695] The server performs visual character recognition (OCR) processing on the received file, extracting text information from the document using OCR technology. This process accurately reads not only printed text but also handwritten characters.

[0696] Step 3:

[0697] The server analyzes the information extracted by OCR to identify important contract data such as contract title, signatory, and signing date, and stores the results in a database.

[0698] Step 4:

[0699] The server automatically associates information stored in the database and generates a document structure by combining similar contract information. This improves the overall traceability of the document.

[0700] Step 5:

[0701] The terminal sends the natural language search query received from the user to the server, and the server retrieves relevant information corresponding to the query from the database.

[0702] Step 6:

[0703] The server aggregates the search results, identifies the most relevant contract documents, and returns them to the terminal. The terminal then clearly displays these search results to the user.

[0704] Step 7:

[0705] The server analyzes the contract details and generates an analysis report if it identifies specific risk factors. This report is saved so that the user can review it later.

[0706] Step 8:

[0707] The server monitors the renewal deadlines for each contract and sets reminders as the deadline approaches. The terminal then notifies the user of important contract renewals.

[0708] Step 9:

[0709] The user requests the creation of a new contract document from the server via their terminal. Based on the provided conditions, the server automatically generates a draft contract using a generation AI.

[0710] Step 10:

[0711] The user reviews the contract generated on their device and makes corrections as needed. The server then securely stores the final, corrected version.

[0712] (Example 1)

[0713] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0714] Traditional contract document management systems often required manual, complex searches and risk assessments, leading to inefficiencies. Furthermore, managing contract renewal deadlines was cumbersome, increasing the risk of missing renewal deadlines. These challenges need to be addressed to streamline contract management and improve accuracy.

[0715] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0716] This invention includes a server that receives and stores uploaded contracts from users in an organized manner, a server that extracts information from printed text and handwritten characters within electronic documents using OCR technology, and a server that analyzes the extracted information to generate key data such as contract name, signatories, and date and time as a document structure. This enables automated contract management, efficient searching, risk assessment, and deadline management.

[0717] A "user" is someone who uses a terminal to upload contracts or enter search queries into the system.

[0718] A "server" is a central processing unit that receives information from users, analyzes the data using OCR technology, and performs tasks such as storing, searching, displaying, and risk assessment of contract documents.

[0719] "OCR technology" refers to optical character recognition technology, which converts printed text and handwritten characters in electronic documents and images into digital data.

[0720] A "contract document" is an official document that records the details of a transaction or agreement, and is subject to renewal and risk assessment.

[0721] "Risk assessment" is the process of analyzing the contents of a contract document to identify and evaluate factors that could potentially cause problems in the future.

[0722] "Deadline management" is a function that monitors contract renewal deadlines and notifies users to ensure they don't miss the renewal deadline.

[0723] A "draft" is a template for a contract document, a draft automatically generated based on user requests.

[0724] This invention provides an AI-powered contract management system that enables efficient contract management through automated processes. This system operates through the collaboration of servers, terminals, and users.

[0725] The server first receives PDF or image files of contracts uploaded by the user via their device. At this point, the server uses open-source optical character recognition tools such as Tesseract OCR to convert printed and handwritten text within the document into digital data. This technology is used to accurately extract important information.

[0726] From the extracted information, the server extracts key data such as the contract title, signatory, and signing date, and stores this as structured data. This improves the efficiency of association and searching. The information is also stored in a database, and an index is generated for easy searching.

[0727] The terminal receives natural language search queries from the user and sends them to the server. The server searches the database based on the received queries and identifies relevant contract documents. The results are sent to the terminal and made available for the user to view. For example, a prompt such as "Show me the transaction agreements concluded in September 2023" can be used.

[0728] Furthermore, the server analyzes the contract content in real time and assesses potential risk factors. This analysis uses a generative AI model, and based on the evaluation results, it generates and provides an analysis report to the user.

[0729] The contract renewal deadline is monitored by the server, and users are notified via their devices as the deadline approaches. Based on this notification, users can renew their contracts in a timely manner.

[0730] Furthermore, if a user wishes to create a new contract, they can request automatic contract generation through their terminal. The server generates a draft according to pre-prepared templates and user-specified parameters, and the user can review and modify the draft on their terminal.

[0731] These features enable the AI ​​contract management system to improve the efficiency and accuracy of contract management, significantly reducing the workload for users.

[0732] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0733] Step 1:

[0734] The user uploads a PDF or image file of the contract to the system using a terminal. This becomes the input data. The terminal sends the uploaded data to the server, and the data reception process begins. The server temporarily stores this uploaded document in its storage.

[0735] Step 2:

[0736] The server performs OCR processing on the received contract. Here, OCR software is used to recognize text within the image. The input is an image of the contract, and the output is recognized text information. This process analyzes both printed and handwritten characters, extracting text data from each.

[0737] Step 3:

[0738] The server analyzes the text information extracted by OCR to identify key contract data. Important items such as the contract title, signatories, and signing date are extracted. The input is the OCR output text, and the output is a list of this key data. Based on this, the server generates the document structure and stores it in a database.

[0739] Step 4:

[0740] When a user wants to search for specific contract information, they enter a query in natural language format via a terminal. The terminal sends this query to the server. The server parses the query, searches the database, and identifies the relevant contract documents. In this case, the input is the user query, and the output is a list of relevant contract documents.

[0741] Step 5:

[0742] The server sends the search results to the terminal. The terminal displays the results to the user in an easy-to-understand format. The user can view the displayed results and check the details as needed. A possible specific prompt would be, "Show me the transaction agreements concluded in September 2023."

[0743] Step 6:

[0744] The server analyzes the contract content in real time and evaluates risk factors. It uses a generative AI model to perform risk assessments and generates risk analysis reports as needed. The input is the contract content, and the output is the risk assessment results and report. This allows users to take appropriate measures against risks.

[0745] Step 7:

[0746] The server manages the contract renewal deadline. As the contract renewal deadline approaches, the server sends a notification to the user via the terminal. The input is the renewal deadline information, and the output is a notification message. The user can receive this notification and proceed with the appropriate renewal procedure.

[0747] Step 8:

[0748] Users can request automated contract generation from their terminal to create new contracts. The server generates a draft by combining a template with user-provided parameters. The input is user parameters, and the output is the drafted contract. This draft is sent to the terminal for the user to review and make any necessary revisions.

[0749] (Application Example 1)

[0750] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0751] In modern manufacturing, efficiently managing equipment maintenance contracts and various production line contracts is a crucial factor in reducing operational costs and improving production efficiency. However, when these contracts are managed manually, it is difficult to retrieve the necessary information at the appropriate time from a vast amount of contract data. Furthermore, mistakes such as missing contract renewal deadlines are likely to occur, potentially impacting the manufacturing process. Therefore, new methods are needed to solve these problems.

[0752] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0753] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by visual data recognition processing based on the received electronic data, and means for coordinating with a control device incorporated in the equipment to manage and notify maintenance schedules. This enables efficient management of equipment maintenance contracts and other contract information, allows maintenance to be performed at the appropriate time, and enables smooth operation of the manufacturing line.

[0754] An "information processing device" is a system that includes devices for receiving, processing, and managing electronic data, such as computers and servers.

[0755] "Visual data recognition processing" is the process of extracting textual information from image data, and is performed using OCR technology.

[0756] A "natural language query" is an interface in which a user uses ordinary language to request information from a system.

[0757] "Risk factors" refer to potential dangers and problems hidden within contract information, and evaluating these is a necessary element.

[0758] An "analysis document" is a document that summarizes the results of the risk factor assessment and presents them to users in an easy-to-understand manner.

[0759] A "control device" is a device used to manage and operate specific equipment or systems.

[0760] A "maintenance schedule" is a plan that outlines the timing and content of maintenance and inspections of equipment and machinery.

[0761] A "notification" is the act of informing a user of a specific event or piece of information, and often takes the form of an alert or reminder.

[0762] In the system that realizes this invention, the server, terminal, and user work together in coordination.

[0763] The server first receives various electronic data related to the equipment from cameras and scanning devices attached to robotic arms within the factory. The received data is then processed using OCR technology to extract textual information. Existing software such as Tesseract OCR is used for this process. The extracted information is stored in a database and attributed to contract information.

[0764] The terminal accepts natural language queries from users. This interface is designed for users to search for contract information and check maintenance schedules. The server searches the database based on these queries, efficiently extracts the relevant information, and returns it to the terminal.

[0765] Furthermore, the control unit built into the device and the server work together to perform risk assessments and automatically create maintenance schedules based on the extracted data. The assessment algorithm executed on the server identifies risk factors based on contract information and generates analytical reports. In addition, when important renewal deadlines or maintenance schedules approach, reminders are automatically sent to the terminal in the form of emails or alerts.

[0766] As a concrete example, in one factory, a maintenance contract for a specific machine is required every month. This system uses OCR technology to read the contract related to that machine, automatically calculates the next maintenance date within the system, and adds it to Google Calendar. At the same time, it sends an email notification to the person in charge five days in advance.

[0767] Examples of prompt messages include, "When is the next scheduled maintenance? Please extract the information from the contract." This system design enables more efficient contract management and supports maintenance operations.

[0768] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0769] Step 1:

[0770] The server receives electronic data about the equipment from a camera attached to a robotic arm in the factory. This process involves inputting image data captured by the camera. The server records this image data and prepares it for the next step, which involves visual data recognition processing.

[0771] Step 2:

[0772] The server performs visual data recognition (OCR) processing on the received image data to extract text information. The input is the received image data, and the output is the extracted text information. This process uses software such as Tesseract OCR to recognize both printed and handwritten characters.

[0773] Step 3:

[0774] The server assigns attributes to the contracts based on the extracted text information. In this step, the text information obtained by OCR is used as input, and the output is contract information with attributes such as contract date and maintenance details, which is stored in the database. The server then manages the necessary contract information based on this.

[0775] Step 4:

[0776] The user makes a query using natural language through the terminal. In this case, the input is a prompt such as, "When is the next maintenance scheduled?" The terminal receives this input and sends it to the server.

[0777] Step 5:

[0778] The server searches the database based on the user's query and extracts relevant contract information. The input is a natural language query from the user, and the output is the corresponding contract information. The server selects the most relevant information and sends it to the terminal.

[0779] Step 6:

[0780] The server manages maintenance schedules and generates reminders through coordination with the control devices built into the equipment. Contract information and schedule information are used as input, and reminder information is generated as output. Specifically, it calculates the next maintenance date and sets notifications for email and calendar applications.

[0781] Step 7:

[0782] The terminal displays search results and reminder information received from the server to the user. This output includes the next scheduled maintenance date and related contract information. The user can take appropriate action based on the displayed information.

[0783] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0784] The AI ​​contract management system of the present invention utilizes servers, terminals, users, and an emotion engine to automate the management of contract documents and improve the user experience.

[0785] First, the server receives electronic files of contracts sent by the user. These include PDF and image files. The server performs visual character recognition on these files to extract important information from the contract. This information includes the contract title, signatories, and signing date.

[0786] Next, the server associates the extracted information and generates a document structure. In this process, the sentiment engine considers the user's past preferences and current emotional state to help with optimal association. As a result, document management becomes faster and more accurate.

[0787] The terminal receives search queries from the user in natural language and sends them to the server. The server searches the database based on these queries to identify the most suitable contract documents. Simultaneously, an emotion engine analyzes the user's emotional state and adjusts how search results are displayed and the feedback messages. For example, if the user is stressed, the system helps the user quickly understand the information by summarizing the search results concisely.

[0788] The server also takes emotion engine data into account when performing a risk assessment of the contract and generating an analysis report based on the identified risk factors. When the user is relaxed, a detailed analysis report is provided to allow the user to carefully consider its contents.

[0789] Furthermore, the server monitors contract renewal deadlines and sends reminder notifications via the device as the deadline approaches. The content of the notifications is also adjusted based on insights from the emotion engine, taking care to ensure that users can manage their contracts while maintaining positive emotions.

[0790] Finally, based on the request to generate a contract document, the server generates a document template. At this time, the emotion engine checks the user's current state and helps ensure that the generated document matches the user's requirements.

[0791] In this way, the AI ​​contract management system, by combining an emotion engine, provides an optimized contract management experience for users, aiming to improve operational efficiency and satisfaction.

[0792] The following describes the processing flow.

[0793] Step 1:

[0794] Users use their devices to upload electronic files of contracts to the system. These files include PDF and image formats.

[0795] Step 2:

[0796] The server receives the uploaded contract and uses OCR technology to extract text from the document. This process recognizes both handwritten and printed characters with high accuracy.

[0797] Step 3:

[0798] The server identifies key information such as contract title, signatory, and signing date based on the extracted text data, and stores this information in a database.

[0799] Step 4:

[0800] The server automatically associates information stored in the database with other similar contract documents and generates a document structure. An emotion engine also operates, taking user sentiment into consideration to optimize the associations.

[0801] Step 5:

[0802] The terminal receives a natural language search query from the user, sends it to the server, and starts the search.

[0803] Step 6:

[0804] The server searches the database based on the received query to find the most relevant contract documents. During this process, the sentiment engine evaluates the user's emotional state and adjusts how the search results are presented.

[0805] Step 7:

[0806] The server analyzes the contract details and, if risk factors are identified, generates an analysis report. Using information from the sentiment engine, the report is presented in a user-friendly format.

[0807] Step 8:

[0808] The server monitors the contract renewal deadline and sets a reminder when the deadline approaches. The terminal sends emotionally adaptive reminder notifications to the user.

[0809] Step 9:

[0810] The user requests the automatic generation of a contract document through their terminal. The server uses a generation AI based on the conditions specified by the user to create a contract template.

[0811] Step 10:

[0812] The server uses an emotion engine to analyze the user's state and configures document generation and display to enhance user satisfaction. Users can review the generated contract on their device and make any necessary modifications.

[0813] (Example 2)

[0814] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0815] In modern information management systems, managing electronic data has become increasingly complex, and efficient and effective processing is required, especially for critical data such as contract documents. Furthermore, flexible functions that can adaptively manage data according to the user's emotional state are also necessary. However, conventional systems have struggled to integrate such flexibility with advanced information extraction and management functions.

[0816] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0817] In this invention, the server includes means for performing electronic data reception processing, means for extracting information by optical character recognition processing, and means for creating a data structure by associating data. This enables efficient management of contract documents while providing nuanced responses based on the user's emotions.

[0818] A "data processing device" is a system that combines hardware and software for efficiently managing and processing electronic data.

[0819] "Electronic data" refers to all information that is expressed in digital format and can be processed on a computer.

[0820] "Optical character recognition processing" is a technology that identifies characters and numbers from electronic data in image format and extracts them as text data.

[0821] "Relationship building" refers to the process of analyzing the relationships between extracted information and organizing them in an integrated manner.

[0822] A "data structure" is a method of organizing information in a specific format to enable efficient processing and storage of that information.

[0823] "Emotional state" refers to the user's current emotions and psychological state, and is a factor that influences the system's operation and response.

[0824] "Reporting" refers to the action of providing users with important information or notifications regarding deadlines.

[0825] A "data template" is a standardized format that serves as the basis for generating electronic data such as contract documents.

[0826] The data processing device of the present invention automates the management of contract documents using a server, terminals, users, and an engine that performs sentiment analysis. Its embodiments are described in detail below.

[0827] First, the server receives contract documents sent electronically by the user. These documents are typically in PDF or image format. The server then uses optical character recognition (OCR) software, such as Tesseract, to extract text data from these documents. This digitizes paper-based documents, making them electronically processable.

[0828] Next, the server analyzes the extracted text data to identify key information such as the contract title, the names of the parties, and the signing date. Based on this information, the server generates the data structure for the contract document. At this stage, the sentiment analysis engine takes into account the user's past preferences and current emotional state to improve the accuracy of the associations.

[0829] The terminal receives natural language prompts from the user and sends them to the server. An example of a prompt might be, "Show me the latest contract." The server uses this to search its database and identify the relevant contract document. The server analyzes the user's emotional state through sentiment analysis and adjusts how the results are displayed. For example, if the user is feeling stressed, the information may be visually organized to aid understanding.

[0830] Furthermore, the server evaluates the risk factors of the contract and, if a detailed analysis is required, generates an analytical report that reflects the results of sentiment detection. Depending on the user's emotional state, they can choose to receive a detailed or summary report.

[0831] Finally, the server monitors the contract document's renewal deadline, and when the deadline approaches, a notification is sent to the user via their device. The notification's wording is adjusted according to the user's emotional state to ensure a positive user experience. Furthermore, if the user wishes to create a new contract document, the server generates a template based on the sentiment analysis results, providing the document that best suits the user's needs.

[0832] In this way, the data processing device of the present invention highly automates contract document management and enables the provision of individually adapted information by taking into account the user's emotions.

[0833] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0834] Step 1:

[0835] The server receives contract documents from the user as electronic data. This input includes PDFs and image files. The received electronic data is processed by OCR software (e.g., Tesseract) to extract text data from the visual data. The output of this process is used as text data in subsequent processing.

[0836] Step 2:

[0837] The server analyzes the text data extracted by OCR. It identifies important metadata such as contract title, signatory, and signing date from the input text data and records it in the database. In this step, an information identification algorithm is applied to extract the important information, which is then output as structured data.

[0838] Step 3:

[0839] The server generates a document structure based on the identified metadata. In this process, metadata, the user's past preferences, and current sentiment state are used as input data, and the sentiment analysis engine improves the accuracy of the associations. The output of this process is a well-organized data structure, which is used for subsequent searching and management.

[0840] Step 4:

[0841] The terminal receives natural language prompts from the user and sends them to the server. An example of an input prompt is "Show me the latest contract." The server parses this prompt and searches its database based on the prompt it received as input. It identifies the most relevant contract document and formats it in the optimal way for the user to see it.

[0842] Step 5:

[0843] The server analyzes contract information and assesses potential risk factors from the input data. It then generates an analytical report that reflects the output of the sentiment analysis engine. The report generated through this process is provided in either a detailed or concise format, flexibly adapting to the user's needs.

[0844] Step 6:

[0845] The server monitors the renewal deadlines for contract documents and sends notifications to users via their terminals as the deadline approaches. Based on the input monitoring data, it creates notifications and outputs messages that reflect the results of sentiment analysis. This improves user responsiveness.

[0846] Step 7:

[0847] When a user requests the generation of a new contract document, the server creates the document using a template. This input includes the user's requests and emotional state, and the server generates the most suitable document based on this information. The resulting document is tailored to the user's needs, thereby improving operational efficiency.

[0848] (Application Example 2)

[0849] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0850] Traditional contract management systems have struggled to efficiently manage the content of contract documents and provide appropriate feedback while considering the user's emotional state. Furthermore, there has been a lack of sufficient methods to automate the risk assessment of contract content and reduce the burden on users. Therefore, there is a need to improve the efficiency of contract processing and optimize the user experience.

[0851] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0852] In this invention, the server includes means for performing processing to receive electronic data, means for acquiring information by visual character recognition based on the received data, means for generating related information based on the acquired information, means for receiving instructions in natural language and searching for related electronic data, means for evaluating risks and generating analysis reports based on the analyzed information, means for monitoring data update deadlines and providing notifications when the deadline approaches, means for generating data templates based on document generation instructions, and means for adjusting feedback based on user status. This streamlines the entire contract management process and enables the provision of appropriate feedback that meets user needs.

[0853] "Electronic data" refers to information recorded in digital format, and examples include contract documents and document files.

[0854] "Visual character recognition" is a technology that reads characters from images, PDF files, and other sources and extracts them as text information.

[0855] "Generating related information" means organizing and integrating necessary information based on acquired data to form new, meaningful information.

[0856] "Natural language instructions" refer to instructions and questions based on the language that humans use in everyday life, with the aim of information processing systems understanding and responding to them.

[0857] "Assessing risks and generating an analysis report" is the process of identifying potential problems and risks based on the information obtained and creating a report based on that information.

[0858] "Monitoring and notifying data expiration dates" refers to a function that tracks the expiration dates of data and information held and notifies users when the expiration date is approaching.

[0859] "Generating data templates based on document generation instructions" refers to a function that creates the basic structure and format of necessary documents according to user instructions.

[0860] "Adjusting feedback based on user status" is the process of optimizing the content of information and advice provided by taking into account the user's emotions and circumstances.

[0861] The system for realizing this invention consists mainly of a central server and user terminals. The server receives electronic data transmitted from the user and extracts the necessary information using visual character recognition technology. Software such as OpenCV and Pytesseract is used for this technology. The extracted information is structured in combination with other related information and stored in a database.

[0862] Instructions in natural language are sent to the server via the terminal. The server understands them, searches for the appropriate electronic data, and returns the results. Generative AI models are used in this process to analyze the user's emotional state. For example, if the user is in a hurry, it provides concise feedback; if they are relaxed, it presents a detailed analytical report.

[0863] Furthermore, the server assesses the risks in contract data and generates an analysis report. This helps users make informed decisions by identifying potential risks in the contract and communicating them to them. It also monitors data update deadlines and notifies users via their devices when the deadline approaches. This notification is also delivered in an optimal way, taking into account the user's emotional state.

[0864] When a document is requested to be generated, the server creates a template based on the specified format. In this process, it can reflect the user's emotional state and generate the document in the most appropriate form.

[0865] For example, when handling electronic contracts with construction companies, the server extracts contract details using visual character recognition and performs a risk assessment. Depending on the user's situation, it may provide a concise notification such as, "There is a high risk to your budget," or a more detailed notification such as, "There is a high possibility of exceeding your budget, so please consider additional cost management measures."

[0866] In this way, the system dynamically adjusts feedback based on the user's emotional state, not only increasing the efficiency of contract management but also improving the user experience. For example, a prompt might say, "Assess the risks of electronic contracts with construction companies and generate a concise report if the user is in a hurry, or a detailed report otherwise."

[0867] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0868] Step 1:

[0869] The user sends electronic data to the server using their device. Input can include contract data in PDF or image format. Once this data reaches the server, data reception is complete.

[0870] Step 2:

[0871] The server performs visual character recognition processing on the received electronic data. Specifically, it uses OpenCV and Pytesseract to extract text information from the image data. The input is image data, and the output is the extracted text information.

[0872] Step 3:

[0873] The server generates contract-related information based on the extracted text information. This process involves organizing the information and shaping its related structure. The input is the acquired text information, and the output is the associated contract information.

[0874] Step 4:

[0875] The user sends instructions in natural language to the server using a terminal. The server then analyzes the user's instructions and searches for relevant electronic data. The input is the user's instructions, and the output is the search results data.

[0876] Step 5:

[0877] The server uses a generative AI model to perform a risk assessment based on the search results data and generates an analysis report. At this stage, the user's emotional state is analyzed, and the output report is adjusted accordingly. The input is the search results data, and the output is the adjusted analysis report.

[0878] Step 6:

[0879] The server monitors the data update deadline and sends a notification to the user's terminal when the deadline approaches. The prompt message used is "Assess the risks of the electronic contract with the construction company...". The input is the current date and time and data update information, and the output is a notification based on the conditions.

[0880] Step 7:

[0881] The server generates a data template based on the user's instructions for document creation. In this process, the user's emotional state is reflected, and the document is created in the most optimal format. The input is the user's instructions and emotional state, and the output is the generated document template.

[0882] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0883] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0884] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0885] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0886] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0887] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0888] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0889] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0890] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0891] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0892] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0893] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0894] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0895] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0896] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0897] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0898] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0899] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0900] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0901] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0902] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0903] The following is further disclosed regarding the embodiments described above.

[0904] (Claim 1)

[0905] An information processing device that performs automated document management,

[0906] A means for performing the receiving process of electronic documents,

[0907] A means for extracting information from a received electronic document using visual character recognition processing,

[0908] A means of generating a document structure by associating information based on the extracted information,

[0909] A means of accepting queries in natural language and searching for related electronic documents,

[0910] A means of evaluating risk factors based on the analyzed information and generating an analysis report,

[0911] A means of monitoring document update deadlines and notifying when the deadline approaches,

[0912] A means of generating a document template based on a request for contract document generation,

[0913] A system that includes this.

[0914] (Claim 2)

[0915] The system according to claim 1, which identifies similar documents based on association criteria in the generation of the document structure.

[0916] (Claim 3)

[0917] The system according to claim 1, wherein the visual character recognition process extracts information from an image that includes handwritten characters.

[0918] "Example 1"

[0919] (Claim 1)

[0920] A means of receiving and organizing contracts uploaded by users,

[0921] A means for extracting information from printed text and handwritten characters in electronic documents using OCR technology,

[0922] A means for analyzing the extracted information and generating key data such as contract name, signatory, and date / time as a document structure,

[0923] A means of receiving search instructions from users in natural language format, identifying and displaying the relevant contract documents,

[0924] A means of analyzing contract details, assessing risks, and generating reports based on the analysis results,

[0925] A means of monitoring the contract renewal deadline and notifying when the deadline is approaching,

[0926] A means of generating a draft contract document based on parameters and prompting the user for review,

[0927] A system that includes this.

[0928] (Claim 2)

[0929] The system according to claim 1, which, in generating the document structure, identifies related documents based on association criteria.

[0930] (Claim 3)

[0931] The system according to claim 1, wherein in the OCR processing described above, information is extracted from an electronic document containing handwritten characters.

[0932] "Application Example 1"

[0933] (Claim 1)

[0934] A device for automated information processing,

[0935] Means for performing the receiving process of electronic data,

[0936] A means for extracting information by visual data recognition processing based on received electronic data,

[0937] A means of assigning attributes based on extracted information and generating a document format,

[0938] A means of accepting queries in natural language and retrieving related information,

[0939] A means of evaluating risk factors based on the analyzed information and generating analytical materials,

[0940] A means of monitoring the deadline for updating information and notifying when the deadline approaches,

[0941] A means for generating a document template based on a data generation request,

[0942] A means of managing and notifying maintenance schedules in cooperation with a control device built into the equipment,

[0943] A system that includes this.

[0944] (Claim 2)

[0945] The system according to claim 1, which identifies similar information based on attribute criteria in generating the aforementioned document format.

[0946] (Claim 3)

[0947] The system according to claim 1, wherein the visual data recognition process extracts information from an image that includes handwritten data.

[0948] "Example 2 of combining an emotion engine"

[0949] (Claim 1)

[0950] A data processing device that performs automated information management,

[0951] Means for performing the receiving process of electronic data,

[0952] A means for extracting information by optical character recognition processing based on received electronic data,

[0953] A means of generating a data structure by creating relationships based on extracted information,

[0954] A means to analyze users' past preferences and current emotional states and optimize their associations,

[0955] A means of receiving requests in natural language and searching for related electronic data,

[0956] A means for evaluating potential factors based on the analyzed information and generating an analysis report,

[0957] A means of monitoring data update deadlines and notifying when the deadline approaches,

[0958] A means of adjusting the content of the report according to the emotional state,

[0959] A means for generating a data template based on a document generation request,

[0960] Means for adjusting the generated data to suit the user's requirements,

[0961] A system that includes this.

[0962] (Claim 2)

[0963] The system according to claim 1, wherein in generating the aforementioned data structure, similar data is identified based on a correlation criterion.

[0964] (Claim 3)

[0965] The system according to claim 1, wherein in the optical character recognition process, information is extracted from an image including handwritten data.

[0966] "Application example 2 of combining emotional engines"

[0967] (Claim 1)

[0968] An automated information processing system,

[0969] A means for performing the process of receiving electronic data,

[0970] A means of acquiring information by visual character recognition based on received data,

[0971] A means of generating related information based on acquired information,

[0972] A means of accepting instructions in natural language and searching for related electronic data,

[0973] A means of evaluating risk and generating an analysis report based on the analyzed information,

[0974] A means of monitoring data update deadlines and notifying when the deadline approaches,

[0975] A means of generating a data template based on instructions for document generation,

[0976] A means of adjusting feedback based on user status,

[0977] A system that includes this.

[0978] (Claim 2)

[0979] The system according to claim 1, wherein in generating the aforementioned related information, the feedback is adjusted based on the emotional state.

[0980] (Claim 3)

[0981] The system according to claim 1, wherein the information acquisition process takes into account the user's emotional state when acquiring information from an image. [Explanation of Symbols]

[0982] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An information processing device that performs automated document management, A means for performing the receiving process of electronic documents, A means for extracting information from a received electronic document using visual character recognition processing, A means of generating a document structure by associating information based on the extracted information, A means of accepting queries in natural language and searching for related electronic documents, A means of evaluating risk factors based on the analyzed information and generating an analysis report, A means of monitoring document update deadlines and notifying when the deadline approaches, A means of generating a document template based on a request for contract document generation, A system that includes this.

2. The system according to claim 1, which, in generating the document structure, identifies similar documents based on association criteria.

3. The system according to claim 1, wherein the visual character recognition process extracts information from an image that includes handwritten characters.

Images