system

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to input a URL of a web page to be corrected or a captured image of a paper medium to the system; means for a terminal to transmit the input URL or image data to a server; means for the server to recognize characters from the input data using OCR technology; means for the server to apply a designated correction rule based on the recognized character string; means for the server to analyze context and generate an appropriate correction instruction; and means for the terminal to receive the generated correction instruction from the server.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When revising web pages or paper documents, identifying the necessary revisions and creating revision instructions for changes to service names or conditions is time-consuming, and manual work can easily result in oversights. In particular, variations in text or notation within images cannot be detected by search, making it easy for revision errors and omissions to occur. This places a heavy burden on human resources, and a method for efficiently creating revision instructions is needed. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following means. First, a means is provided for a user to input the URL of a web page to be corrected or a captured image of a paper medium into the system, and a means is provided for the terminal to send the input URL and image data to a server. The server has means for recognizing characters from the input data using OCR technology and applying specified correction rules based on the recognized character string. The server further has means for analyzing the context and generating appropriate correction instructions, and means for marking the corrections and creating a correction instruction document. Finally, a system is provided in which the terminal receives the created correction instruction document from the server. In this way, the present invention prevents omissions of corrections and inconsistencies in spelling, and realizes efficient creation of correction instruction documents.

[0006] A "user" is a person or entity that inputs data to be modified into the system.

[0007] A "terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, and is a device that transmits data to a server and receives data from the server.

[0008] A "server" is a computer system installed in a cloud or data center, and is a device that processes received data and performs central processing such as generating correction instructions.

[0009] "OCR technology" is an abbreviation for optical character recognition technology, which is a technology that reads character data from images or scanned documents and converts them into digital text.

[0010] "Means for recognizing characters" refers to software and algorithms for reading characters from data using OCR technology.

[0011] "Modification rules" are predefined rules for replacing, adding, or deleting specific parts of text with other text.

[0012] "Means for analyzing the context" refers to a method that uses natural language processing technology to understand the context of the recognized string and make appropriate corrections.

[0013] A "correction instruction sheet" is a document that describes the identified correction areas and the specific correction contents, and is generated in a format such as PDF or Word.

[0014] "Marking" refers to marks or colors that are added to visually emphasize the corrections.

[0015] "Means for inputting" refers to an interface through which a user inputs data into the system.

[0016] "Means for sending" refers to the communication functions and protocols that allow a terminal to send data to a server.

[0017] "Means for receiving" refers to the communication functions and protocols that allow the terminal to receive the generated correction instructions from the server. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented in the following manner.

[0040] System Embodiments

[0041] 1. User data entry

[0042] Through the system interface, the user selects the URL of the web page or a captured image of the paper document to be edited and inputs it into the system. For example, the user can upload the URL of the web page "https: / / example.com / page1" or a captured image of the paper document.

[0043] 2. Data transmission by the terminal

[0044] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[0045] 3. OCR processing by the server

[0046] The server analyzes the received data and uses OCR (Optical Character Recognition) technology to recognize the characters within the data. In the case of a URL, it retrieves the HTML of the web page, and in the case of an image, it analyzes the characters within the image. This process is typically performed using an OCR engine such as Tesseract.

[0047] 4. Server applies modification rules

[0048] The server applies pre-defined modification rules based on the recognized strings. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[0049] 5. Context Analysis by the Server

[0050] The server analyzes the context surrounding the recognized string and applies appropriate corrections, using natural language processing techniques to adjust the context before and after the correction to ensure consistency.

[0051] 6. Server creates correction instructions

[0052] The server marks the corrections and automatically generates a correction instruction sheet that reflects the specific correction instructions. The correction instruction sheet contains details of the corrections and the specific correction content. The correction instruction sheet is usually generated in a format such as PDF or Word.

[0053] 7. Receiving correction instructions via terminal

[0054] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[0055] Specific examples

[0056] Rename a web page

[0057] User Action:

[0058] The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[0059] Terminal behavior:

[0060] The device sends the URL "https: / / example.com / page1" to the server.

[0061] Server Action:

[0062] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[0063] Terminal behavior:

[0064] The terminal receives the correction instructions and displays them to the user.

[0065] Changes in paper media conditions

[0066] User Action:

[0067] The user uploads a captured image of the paper media to the system.

[0068] Terminal behavior:

[0069] The terminal transmits the image data to the server.

[0070] Server Action:

[0071] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[0072] Terminal behavior:

[0073] The terminal receives the correction instructions and displays them to the user.

[0074] As described above, the system of the present invention prevents omissions of corrections and inconsistencies in notation, and realizes efficient creation of correction instructions.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] A user accesses the system interface and inputs the URL of the web page to be modified or a captured image of the paper medium. For example, the user inputs the URL of the web page "https: / / example.com / page1" or uploads an image of the paper medium.

[0078] Step 2:

[0079] The device sends the entered URL or captured image to the server via an HTTP request, with the data usually sent in JSON or binary format.

[0080] Step 3:

[0081] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[0082] Step 4:

[0083] The server uses OCR technology to recognize characters from HTML pages and captured images. In the case of images, an OCR engine such as Tesseract is used to extract character data.

[0084] Step 5:

[0085] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[0086] Step 6:

[0087] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[0088] Step 7:

[0089] The server marks the areas to be corrected and automatically generates a correction instruction sheet, which clearly describes the identified areas to be corrected and the specific corrections to be made.

[0090] Step 8:

[0091] The terminal receives the generated correction instructions from the server as an HTTP response.

[0092] Step 9:

[0093] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[0094] Through the above steps, this system efficiently creates correction instructions and prevents omissions of corrections and inconsistencies in notation.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] Conventional correction work is often done manually, which can lead to problems such as missing corrections, inconsistencies in spelling, and inconsistencies in context. Furthermore, there is a lack of systems that can efficiently correct both paper and digital content. Therefore, there is a need to improve the efficiency and accuracy of correction work.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for recognizing characters from input data using OCR technology, means for applying specified correction rules based on the recognized character string, and means for analyzing the context using natural language processing technology, which makes it possible to prevent omissions and inconsistencies in spelling and automatically make corrections that are consistent with the context.

[0100] "User" refers to the person or end user who uses the system to input data to be corrected and checks the correction results.

[0101] "Content identification information" refers to information that uniquely identifies the digital data to be modified, such as the URL of a web page or the ID of a digital document.

[0102] "Digital data" refers to data expressed in a format that can be processed by a computer, and includes, for example, text, images, audio, and video.

[0103] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that has the ability to communicate with the system and send and receive data.

[0104] "Server" refers to a computer system that receives requests from terminals, processes the input information, and generates correction instructions.

[0105] "OCR technology" is an abbreviation for Optical Character Recognition, and refers to the technology that analyzes characters in an image and converts them into text data.

[0106] The "modification rule" refers to a rule for replacing a specific character string or information with another character string or information, or a rule for adding necessary information or deleting unnecessary information.

[0107] "Natural language processing technology" refers to computer technology for understanding, analyzing, and generating human language, and includes, for example, algorithms for context analysis and semantic understanding.

[0108] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and is usually generated in a digital format such as PDF or Word.

[0109] A "prompt" is text used to input instructions or questions to a model, and refers to a document that specifically instructs an AI model on what to do.

[0110] An "AI model" refers to an algorithm or neural network that learns from data and automatically performs specific tasks.

[0111] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented as follows.

[0112] The system consists of multiple components: users, terminals, and servers. Users access the system using their terminals and enter the data to be corrected. The data to be corrected can be a web page URL or a captured image of a paper document.

[0113] The device (PC, smartphone, etc.) provides a means to send the URL or captured image entered by the user to the server. Data is sent in JSON or binary format using an HTTP request.

[0114] The server performs various processes using the following hardware and software:

[0115] Hardware: A server with a powerful processor, sufficient memory and storage

[0116] software:

[0117] OCR Technology: Tesseract OCR Engine

[0118] Natural language processing technology: NLP libraries (e.g., SpaCy, NLTK)

[0119] Data processing: Programs written in Python, Java, etc.

[0120] The system processing flow is as follows:

[0121] The server receives the data sent from the terminal and first applies OCR technology to recognize characters from the input data. In this step, character recognition is performed from an image using the Tesseract OCR engine. For example, character data is extracted from a captured image of a paper document uploaded by the user.

[0122] The server then applies pre-defined modification rules based on the recognized strings, which may include replacing specific strings, removing unnecessary information, or adding necessary information, such as replacing "old service name" with "new service name."

[0123] The server then uses natural language processing technology to analyze the context surrounding the recognized string. This allows for appropriate corrections to be made that are consistent with the context. In this step, a generative AI model is used to perform the appropriate context analysis. Specifically, the generative AI model is prompted with the following instruction: "Please make appropriate corrections so that the entire paragraph flows properly."

[0124] The server then marks the corrections and automatically generates a correction instruction sheet, which includes details of the corrections and specific correction content, and is usually generated in PDF or Word format, allowing the user to receive clear correction instructions.

[0125] Finally, the terminal receives the generated correction instructions from the server via an HTTP response and provides them to the user, who can then check the correction instructions and perform the actual correction work.

[0126] Specific examples

[0127] Rename a web page

[0128] User action: The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[0129] What the device does: Sends the URL in JSON format to the server.

[0130] Server process: Obtain HTML and analyze the text with Tesseract. Replace "old service name" with "new service name" and adjust using context analysis. Generate a PDF of the correction instructions.

[0131] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[0132] Changes in paper media conditions

[0133] User Action: Upload a captcha image.

[0134] Device operation: Sends image data to the server.

[0135] Server processing: Recognize text in the image using Tesseract. Replace "old conditions" with "new conditions" and adjust using context analysis. Generate a PDF of correction instructions.

[0136] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[0137] Through the above process, this system is able to carry out correction work efficiently and with high accuracy.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1:

[0140] The user enters the data to be modified. The user selects the URL of a web page or a captured image of a paper document in the system interface and clicks the submit button. For example, the user can enter the URL "https: / / example.com / page1" or upload a captured image file. This operation inputs the web page URL or image data into the system.

[0141] input:

[0142] Webpage URL

[0143] Captured image data

[0144] output:

[0145] Send button pressed event

[0146] Step 2:

[0147] The terminal sends the entered data to the server. The terminal converts the entered URL or captured image into JSON or binary format and sends it to the server via an HTTP request. The event of pressing the send button triggers the sending of JSON format data to the server.

[0148] input:

[0149] Web page URL or captured image data

[0150] Send button pressed event

[0151] output:

[0152] HTTP request (sent by server)

[0153] Step 3:

[0154] The server analyzes the data using OCR technology. If the received URL is a webpage, the server retrieves the HTML and applies OCR technology to extract the text data. If the captured image is a text file, the server uses the Tesseract engine to recognize characters from the image. This process generates the text data.

[0155] input:

[0156] HTTP request (web page URL and captured image data)

[0157] output:

[0158] Text data

[0159] Step 4:

[0160] The server applies pre-defined correction rules. Based on the recognized text data, the server replaces specific strings with other strings, deletes unnecessary information, and adds necessary information. At this stage, the server refers to correction rules stored in a specific memory or database. For example, it replaces "old service name" with "new service name."

[0161] input:

[0162] Text data

[0163] output:

[0164] Corrected text data

[0165] Step 5:

[0166] The server analyzes the context using natural language processing technology. The server uses a generative AI model to analyze the context of the corrected text data and make appropriate adjustments to maintain consistency. The prompt used is "Please make appropriate corrections so that the entire paragraph flows properly."

[0167] input:

[0168] Corrected text data

[0169] output:

[0170] Contextually aligned text data

[0171] Step 6:

[0172] The server creates a correction instruction document. The server marks all the corrections and generates a correction instruction document that summarizes the specific corrections. This instruction document is usually created in PDF or Word format. A PDF generation library is used to generate the instruction document.

[0173] input:

[0174] Contextually aligned text data

[0175] output:

[0176] Correction Instructions (PDF or Word)

[0177] Step 7:

[0178] The terminal receives the correction instructions and provides them to the user. The terminal receives the HTTP response from the server and displays the correction instructions to the user. The user uses these as a reference to perform the actual correction work.

[0179] input:

[0180] Correction Instructions (PDF or Word)

[0181] output:

[0182] Display of correction instructions

[0183] By following the above steps, the system can efficiently carry out correction work.

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] Corrections to work instructions and manuals in factories are often done manually, which can lead to problems such as missing corrections and inconsistencies in notation. Furthermore, the significant time and effort required for corrections reduces work efficiency. There is a need for a system that can solve these problems and streamline the process of correcting work instructions in factories.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes: a means for a user to input the URL of a web page to be corrected or a captured image of a paper medium into the system; a means for a terminal to send the input URL or image data to the server; a means for the server to recognize characters from the input data using OCR technology; a means for the server to apply specified correction rules based on the recognized character strings; a means for the server to analyze the context and generate appropriate correction instructions; a means for the server to mark the corrections and create a correction instruction; a means for the terminal to receive the created correction instruction from the server and provide it to the user as a factory work instruction; and a means for using a camera of a factory robot to take a capture image of the paper medium and input it into the system. This prevents omissions and inconsistencies in work instructions and enables efficient correction work.

[0189] A "user" is a person who uses the system to input the URL of a web page to be modified or a captured image of a paper medium.

[0190] A "terminal" is a device that sends URLs and image data entered by a user to a server and receives correction instructions from the server.

[0191] The "server" is a system that uses OCR technology to recognize characters from input data, applies specified correction rules to analyze the context, and generates correction instructions and creates a correction instruction document.

[0192] "OCR technology" is a technology that optically recognizes characters in an image and converts them into digital text information.

[0193] "Correction rules" are rules that are applied based on recognized character strings and define operations for replacing specific character strings with other character strings, or for adding or deleting necessary information.

[0194] "Context analysis" is a process in which the context surrounding the recognized character string is analyzed using natural language processing technology and appropriate corrections are made.

[0195] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and marks the areas that require correction.

[0196] The "factory robot camera" is a camera built into a robot used in a factory, and is a device that takes captured images of paper media.

[0197] A "factory work instruction" is a document that describes work instructions within a factory, and provides workers with the latest work content based on the correction instruction.

[0198] This invention is a system for streamlining the revision of work instructions and manuals in factories. This system automates a series of processes, from the stage where a user inputs the URL of a web page to be revised or a captured image of a paper medium, to the stage where a revision instruction is generated and the updated work instructions are provided within the factory.

[0199] The system includes a user, a terminal, a server, and a factory robot's camera. The user enters the URL to be corrected or a captured image of the paper media into the system's interface. The input data is sent by the terminal to the server. The server uses OCR technology to recognize characters from the input data and then applies specified correction rules based on the recognized characters. The server then analyzes the context and generates appropriate correction instructions. Finally, it creates a correction instruction sheet containing the marked-up corrections and sends it to the terminal. The terminal then provides the created correction instruction sheet to the user as a factory work instruction sheet.

[0200] The server uses the following hardware and software: The Tesseract OCR engine is used for OCR processing, and PIL (Python Imaging Library) is used as the image processing library. The requests library is used to send and receive HTTP requests. A general NLP library is used for context analysis using natural language processing (NLP).

[0201] The terminal has a function to send data entered by the user to the server in JSON or binary format, receive correction instructions from the server, and provide them to the user as factory work instructions.The factory robot's camera is also used to take capture images of paper media and input the image data into the system.

[0202] For example, if a manual used in a factory contains a description of "old process A" and needs to be changed to a new "new process B," a captured image can be uploaded to the system. The system then automatically corrects "old process A" to "new process B," and a robot presents the new work instructions to the worker. Examples of input prompts for the generative AI model include the following:

[0203] "Please modify the following text. Original text: For this task, please use old process A. Old part X is also required. Modification instructions: Please modify old process A to new process B and old part X to new part Y."

[0204] As described above, the system of the present invention can improve the efficiency of work instruction correction work in factories and prevent oversight of corrections and inconsistencies in notation.

[0205] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0206] Step 1:

[0207] The user inputs the URL of the web page to be edited or a captured image of the paper document into the system. The input can be the URL of the web page (e.g., "https: / / example.com / page1") or a captured image of the paper document. The terminal receives this input data.

[0208] Step 2:

[0209] The device sends the entered URL and image data to the server. The URL and image data are sent to the server in JSON or binary format via an HTTP request. The server receives them and begins processing.

[0210] Step 3:

[0211] The server uses OCR technology to recognize characters from the input data. For example, the Tesseract OCR engine is used to extract text information from captured images of paper media. The input is image data, and the output is a string of characters.

[0212] Step 4:

[0213] The server applies the specified modification rules based on the recognized string. The modification rules include, for example, replacing "old process A" with "new process B." The input is the recognized string, and the output is the modified string.

[0214] Step 5:

[0215] The server analyzes the context and generates appropriate correction instructions. It uses natural language processing technology to analyze the context and make corrections appropriate to the context. The input is the corrected string, and the output is the appropriate correction string after context analysis.

[0216] Step 6:

[0217] The server marks the corrections and creates a correction instruction sheet. The correction instruction sheet contains details of the corrections and specific correction content, and is generated in PDF or Word format, for example. The input is the correction string after context analysis, and the output is the correction instruction sheet.

[0218] Step 7:

[0219] The terminal receives the created correction instructions from the server and provides them to the user as factory work instructions. The user refers to these to perform the actual correction work. The input is the correction instructions, and the output is the factory work instructions that the user refers to.

[0220] Step 8:

[0221] A factory robot's camera is used to capture an image of the paper medium and input it into the system. The captured image data is then subjected to OCR processing as a target for correction. The input is the image of the paper medium, and the output is the image data input into the system.

[0222] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0223] The present invention combines a system for efficiently editing web pages and paper media with an emotion engine that recognizes the user's emotions and responds accordingly. The system is implemented as follows:

[0224] System Embodiments

[0225] 1. User data entry

[0226] The user accesses the system interface and enters the URL of the web page or a captured image of the paper to be edited. For example, the user can enter the URL of the web page "https: / / example.com / page1" or upload an image of the paper.

[0227] 2. Data transmission by the terminal

[0228] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[0229] 3. Emotion engine for recognizing user emotions

[0230] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, and the emotion engine analyzes these to recognize the user's emotions. The emotion engine uses facial recognition and voice analysis technologies to identify the user's emotional state, such as joy, anger, sadness, or happiness.

[0231] 4. OCR processing by the server

[0232] The server receives the input data, retrieves the HTML of the corresponding web page in the case of a URL, or prepares the image for analysis in the case of a paper image. The server then uses OCR technology to recognize characters within the data. For example, an OCR engine such as Tesseract can be used to extract the character data.

[0233] 5. Server applies modification rules

[0234] The server applies the configured modification rules based on the recognized string. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[0235] 6. Context Analysis by the Server

[0236] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[0237] 7. Server creates correction instructions

[0238] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[0239] 8. Adjusting correction instructions with the emotion engine

[0240] The emotion engine adjusts the content and format of the correction instructions based on the user's emotions, for example, simplifying the instructions or providing additional support information if the user is feeling stressed.

[0241] 9. Receiving correction instructions via terminal

[0242] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[0243] Specific examples

[0244] Rename a web page

[0245] User Action:

[0246] The user enters "https: / / example.com / page1" into the system interface and presses the send button. The video and audio recorded during the input are captured and analyzed by the emotion engine.

[0247] Terminal behavior:

[0248] The device sends the URL "https: / / example.com / page1" to the server.

[0249] Server Action:

[0250] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[0251] Emotion Engine Processing:

[0252] The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations and support if the user is feeling anxious.

[0253] Terminal behavior:

[0254] The terminal receives the correction instructions and displays them to the user.

[0255] Changes in paper media conditions

[0256] User Action:

[0257] Users upload captured images of paper to the system, and the emotion engine analyzes the user's emotions as they are input.

[0258] Terminal behavior:

[0259] The terminal transmits the image data to the server.

[0260] Server Action:

[0261] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[0262] Emotion Engine Processing:

[0263] An emotion engine adjusts the format and content of correction instructions based on the user's emotional state, for example generating simple instructions if the user is frustrated.

[0264] Terminal behavior:

[0265] The terminal receives the correction instructions and displays them to the user.

[0266] As described above, by combining an emotion engine, it becomes possible to provide correction instructions that take into account the user's emotional state, thereby realizing a more user-friendly system.

[0267] The processing flow will be explained below.

[0268] Step 1:

[0269] A user accesses the system interface and enters the URL of the web page to be edited or a captured image of the paper. For example, the user enters the URL of the web page "https: / / example.com / page1" or uploads an image of the paper.

[0270] Step 2:

[0271] The device sends the entered URL or captured image to the server via an HTTP request, and the data is usually sent in JSON or binary format.

[0272] Step 3:

[0273] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, thereby obtaining the user's emotional data.

[0274] Step 4:

[0275] The device transmits the acquired emotion data to the emotion engine, which analyzes it and identifies the user's emotion.

[0276] Step 5:

[0277] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[0278] Step 6:

[0279] The server uses OCR technology to recognize characters from the HTML of the web page or the captured image, for example, by using an OCR engine such as Tesseract to extract the character data.

[0280] Step 7:

[0281] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[0282] Step 8:

[0283] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[0284] Step 9:

[0285] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[0286] Step 10:

[0287] The emotion engine adjusts the content and format of the correction instructions based on the user's emotional data. For example, if the user is feeling stressed, the instructions may be simplified or additional support information may be provided.

[0288] Step 11:

[0289] The terminal receives the generated correction instructions from the server as an HTTP response.

[0290] Step 12:

[0291] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[0292] Through these steps, the system efficiently creates correction instructions and takes into account the user's emotional state to prevent omissions and inconsistencies in spelling. Furthermore, by responding to emotions, the system becomes more user-friendly.

[0293] Example 2

[0294] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0295] Conventional editing work on web pages and paper documents is time-consuming and inefficient. Furthermore, no system exists that takes into account the user's emotions, and it is not possible to appropriately address the stress and anxiety users feel while editing. Therefore, there is a need for a user-friendly editing system.

[0296] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0297] In this invention, the server

[0298] means for recognizing characters from the input data using optical character recognition techniques;

[0299] means for applying specified modification rules based on the recognized characters;

[0300] a means for analyzing the context of the recognized character using natural language processing technology;

[0301] and means for recognizing the user's emotional state using emotion recognition technology and adjusting the content and format of the correction instructions.

[0302] This makes it possible to carry out efficient and appropriate correction work that takes into account the user's feelings.

[0303] A "user" is an entity that operates the system and inputs digital information to be modified.

[0304] A "terminal" is a device operated by a user, such as a PC or smartphone, and is responsible for sending input data to a server.

[0305] A "server" is a central processing unit that receives data sent from a terminal and performs various processes.

[0306] Optical character recognition (OCR) is a technology that recognizes characters from image data and converts them into text data.

[0307] "Characters" are elements such as letters and symbols recognized by OCR technology.

[0308] A "correction rule" is an operation rule such as substitution, addition, or deletion that is applied to a recognized character string.

[0309] "Context" refers to the text that exists before and after a particular string of characters, and is information that helps determine its meaning and appropriateness.

[0310] "Natural language processing technology" is a set of technologies that enable computers to understand, analyze, and generate human language.

[0311] "Emotion recognition technology" is a technology for identifying and classifying a user's emotional state from facial expressions, voice, etc.

[0312] A "correction instruction sheet" is a document that marks the corrections and clearly states the corrections.

[0313] The present invention is an efficient system for modifying digital information that takes into account the user's feelings, and is implemented in the following manner.

[0314] First, the user accesses the system interface and enters the digital information to be modified (this can be a web page URL, e.g., "https: / / example.com / page1," or a captured image of a paper document). The user enters this information and presses the submit button.

[0315] Next, the device (user's PC or smartphone) sends the entered URL and image data to the server. The data is usually sent in JSON or binary format. This allows the data to be passed to the server.

[0316] The server analyzes the received URL or image data. In the case of a URL, it retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares the image for analysis. This analysis uses optical character recognition (OCR) technology such as Tesseract to recognize characters from the input data. It then applies correction rules based on the recognized characters. Correction rules can include replacing specific strings with other strings, or adding or deleting necessary information. This is done using Python scripts or regular expressions (RegEx).

[0317] The server then analyzes the surrounding context using natural language processing techniques (such as SpaCy or NLTK) to determine whether the revised text is contextually appropriate. If not, it makes further adjustments.

[0318] Additionally, the present invention incorporates emotion recognition technology. The device captures the user's facial expressions and voice through a camera and microphone, which are then analyzed by the emotion engine. Here, facial expression recognition technology (OpenCV) and voice analysis technology (Google Cloud Speech-to-Text) are used to recognize the user's emotions. The emotion engine then adjusts the content and format of the correction instructions based on the user's emotions. For example, if the device detects that the user is anxious, it can simplify the instructions or provide additional explanations.

[0319] Once all the processes are completed, the correction instructions are generated in a format such as PDF or Word and sent to the terminal. The user can refer to this and carry out the necessary corrections.

[0320] Specific examples

[0321] Rename a web page

[0322] User operation: The user enters "https: / / example.com / page1" into the system interface and presses the submit button. The video and audio recorded during the input are captured and analyzed by the emotion engine.

[0323] Device behavior: The device sends the URL "https: / / example.com / page1" to the server.

[0324] Server processing: The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[0325] Emotion engine processing: The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations or support if the user is feeling anxious.

[0326] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[0327] Example prompt sentence:

[0328] "Enter a webpage URL to generate correction instructions. For example, enter "https: / / example.com / page1". Please keep your facial expressions and voice natural while capturing. Emotional data will be used to refine your correction instructions."

[0329] Changes in paper media conditions

[0330] User operation: The user uploads a captured image of a paper document to the system, and the emotion engine analyzes the user's emotions as they are input.

[0331] Terminal operation: The terminal sends image data to the server.

[0332] Server processing: The server uses OCR technology to recognize the characters in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[0333] Emotion engine processing: The emotion engine adjusts the format and content of the correction instructions based on the user's emotional state. For example, if the user is frustrated, it will generate simple instructions.

[0334] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[0335] Example prompt sentence:

[0336] "Upload a paper capture image and replace the old conditions with the new ones. Please make sure the image is clear and you remain calm while typing. Emotional data will affect the conciseness of the correction instructions."

[0337] As described above, the system of the present invention makes it possible to carry out correction work efficiently while taking into consideration the user's feelings.

[0338] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0339] Step 1:

[0340] The user accesses the system interface and enters the digital information to be modified (a web page URL or a captured image of a paper document).

[0341] Input: For web pages, a URL such as "https: / / example.com / page1". For paper documents, a capture image file.

[0342] Specific operation: The user opens a browser and enters a URL into the input form, or takes a photo of a paper document with the smartphone camera and uploads the image file.

[0343] Output: The input URL or image data.

[0344] Step 2:

[0345] The device sends the entered URL and image data to the server.

[0346] Input: URLs and captcha images entered by the user.

[0347] Specific operation: The terminal creates data in JSON or binary format and sends it to the server address using a POST request.

[0348] Output: URL or image data sent to the server.

[0349] Step 3:

[0350] The server analyzes the URL or captcha image received.

[0351] Input: URL or image data received by the server.

[0352] Specific operation: In the case of a URL, the server retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares for OCR processing.

[0353] Output: HTML content of a web page or image data to be OCR processed.

[0354] Step 4:

[0355] The emotion engine captures the user's facial expressions and voice through the device and recognizes their emotional state.

[0356] Input: Captured facial and voice data of the user.

[0357] Specific operation: Use OpenCV to obtain facial expression data. Use Google Cloud Speech-to-Text to analyze audio data.

[0358] Output: The user's emotional state (e.g., happiness, anger, sadness, anxiety, irritation).

[0359] Step 5:

[0360] The server uses optical character recognition technology (OCR) to recognize characters within the image data.

[0361] Input: Image data to be OCR processed.

[0362] What it does: The server calls the Tesseract library to extract text from the image, using functions like pytesseract.image_to_string(image) .

[0363] Output: OCR recognized text data.

[0364] Step 6:

[0365] The server applies the specified modification rules based on the recognized string.

[0366] Input: Text data recognized by OCR.

[0367] Specific operation: Using Python scripts and regular expressions (RegEx), for example, replace "old service name" with "new service name". Implemented as follows: re.sub(r'\bold service name\b', 'new service name', text).

[0368] Output: Text data with correction rules applied.

[0369] Step 7:

[0370] The server uses natural language processing technology to analyze the context and make appropriate adjustments.

[0371] Input: Text data with correction rules applied.

[0372] Specific operation: Use SpaCy or NLTK to analyze the context and identify areas that need fine-tuning. Execute nlp = spacy.load('en_core_web_sm'); doc = nlp(text) etc.

[0373] Output: Contextually adjusted text data.

[0374] Step 8:

[0375] The server marks the corrections and generates a correction instruction.

[0376] Input: Contextually adjusted text data.

[0377] What it does: Generate a PDF or Word document with the revisions highlighted. For example, generate a PDF using ReportLab. Use canvas.drawString(x, y, revised_text) or similar.

[0378] Output: Generated correction instructions (PDF and Word formats).

[0379] Step 9:

[0380] An emotional engine adjusts the content and format of revision instructions based on the user's emotional state.

[0381] Input: Generated correction instructions and the user's emotional state.

[0382] Specific behavior: If the user is feeling anxious, simplify the instructions or provide additional support information. Implemented as follows: if user_emotion == 'anxiety': add_support_info(document) .

[0383] Output: Adjusted correction instructions.

[0384] Step 10:

[0385] The terminal receives the generated correction instructions from the server and displays them to the user.

[0386] Input: Correction instructions sent from the server.

[0387] Specific operation: Receive the correction instructions in the HTTP response and display them to the user in a browser or dedicated app. For example, use the download_document(response) function.

[0388] Output: Correction instructions displayed to the user.

[0389] The above is the specific processing flow of this system.

[0390] (Application example 2)

[0391] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0392] In factories, there is a demand for accurate and fast correction work on web pages and paper documents. However, conventional systems do not take user emotions into consideration, which can result in poor user-friendliness and reduced efficiency. The present invention aims to provide a technology that recognizes user emotions and responds accordingly, thereby improving the efficiency of correction work and user satisfaction.

[0393] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input a URL or digital image of an information source to be corrected to the system; means for a terminal to transmit the input URL or image data to the server; means for the server to recognize characters from the input data using character recognition technology; means for the server to apply specified correction rules based on the recognized character string; means for the server to analyze the context and generate appropriate correction instructions; means for the server to mark the corrections and create a correction instruction; means for the terminal to receive the created correction instruction from the server; means for the system to recognize the user's emotions; and means for the server to adjust the content and format of the correction instruction based on the emotion data. This makes it possible to provide correction instructions that take the user's emotions into consideration, thereby improving the efficiency of correction work on the factory floor and user satisfaction.

[0394] "User" means the person or operator who uses the system to enter information to be corrected and receives correction instructions.

[0395] A "URL" is an address that indicates the location of a resource on the Internet, such as a web page.

[0396] A "digital image" is an image that captures a paper document or other visual information and stores it as an electronic file.

[0397] "System" refers to a set of hardware and software that performs correction work based on information entered by the user and generates and provides correction instructions.

[0398] "Terminal" means an electronic device used by a User to input and transmit information and receive correction instructions.

[0399] A "server" is a central device that receives and analyzes data sent from a terminal and generates correction instructions.

[0400] "Character recognition technology" is a technology that identifies character information in an image and converts it into text data.

[0401] "Modification rules" are a set of rules for string substitution or modification based on recognized strings.

[0402] "Context" refers to the relationships between the sentences and words surrounding the recognized string.

[0403] "Emotion" refers to the psychological state or feedback that a user shows when entering information.

[0404] "Emotional Data" refers to data collected and analyzed to express a user's emotions.

[0405] A "correction instruction sheet" is a document generated by the system that lists the corrections to be made and the specific corrections to be made.

[0406] An "emotion engine" is a software component for analyzing a user's emotions.

[0407] An embodiment of the present invention will now be described. The system starts when the user inputs the URL or digital image of the information source to be corrected. The user accesses the system interface and inputs and uploads the URL of the web page to be corrected or a captured image of paper media. The terminal is responsible for sending the input URL or image data to the server.

[0408] The server receives the transmitted data and uses the character recognition technology "pytesseract" to recognize characters from the input data. This makes it possible to extract text information from captured images of paper media. The server also applies specified correction rules based on the recognized strings. These correction rules include replacing specific strings with other strings.

[0409] Additionally, the server uses a pipeline of "transformers" as natural language processing technology to analyze the context surrounding the recognized string, determining whether the corrected text is contextually appropriate and making further adjustments if necessary.

[0410] Meanwhile, the system incorporates an "emotion engine," which uses the device's camera and microphone to recognize the user's emotions during input. This emotion engine uses the "DeepFace" library to analyze the user's facial expressions and generate emotion data, which is then sent to the server to indicate the user's emotions.

[0411] The server adjusts the content and format of the correction instructions based on the emotion data. For example, if the user is frustrated, the server can simplify the correction instructions to reduce stress. If the user is anxious, the server can add detailed explanations to provide reassurance.

[0412] Finally, the server marks the corrections and automatically generates a correction instruction sheet. The generated correction instruction sheet is sent to the terminal as an HTTP response and provided to the user. The user can refer to this correction instruction sheet and perform the actual correction work.

[0413] As a concrete example, the following prompt sentence is shown.

[0414] Example prompt sentence:

[0415] "I'd like you to fix this instruction manual. I'm very busy and frustrated."

[0416] Thus, the present invention takes into account the user's feelings and provides an efficient and user-friendly correction system.

[0417] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0418] Step 1:

[0419] The user inputs the URL or digital image of the source to be modified into the system, and the input URL or captured image is sent to the server through the terminal interface.

[0420] Step 2:

[0421] The device sends the input URL or image data to the server. The sent data is transferred to the server in JSON or binary format. The server receives the input data and starts the next process.

[0422] Step 3:

[0423] The server analyzes the received data using optical character recognition (OCR). Specifically, it uses the "pytesseract" library to extract text from digital images. If the input data is a URL, it retrieves the relevant text from the HTML of the web page.

[0424] Step 4:

[0425] The server applies modification rules based on the recognized strings. The server performs operations to replace certain strings with other strings. For example, it replaces "old service name" with "new service name".

[0426] Step 5:

[0427] The server uses natural language processing techniques to analyze the context of the string, using a pipeline of "transformers" to determine whether the revised text is contextually appropriate and make further adjustments if necessary.

[0428] Step 6:

[0429] The system's terminal uses a camera and microphone to recognize the user's emotions. The emotion engine uses the "DeepFace" library to analyze emotions from the user's facial expressions and voice. The analyzed emotion data is sent to the server.

[0430] Step 7:

[0431] The server adjusts the content and format of the correction instructions based on the emotion data, for example, if the user is frustrated, the server adjusts the correction instructions to be briefer and less stressful.

[0432] Step 8:

[0433] The server marks the corrections and automatically generates a correction instruction sheet, which is created in PDF, Word, or other formats.

[0434] Step 9:

[0435] The terminal receives the generated correction instructions from the server and displays them to the user, who can then refer to the correction instructions to carry out the actual correction work.

[0436] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0437] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0438] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0439] [Second embodiment]

[0440] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0441] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0442] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0443] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0444] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0446] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0447] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0448] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0449] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0450] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0451] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0452] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented in the following manner.

[0453] System Embodiments

[0454] 1. User data entry

[0455] Through the system interface, the user selects the URL of the web page or a captured image of the paper document to be edited and inputs it into the system. For example, the user can upload the URL of the web page "https: / / example.com / page1" or a captured image of the paper document.

[0456] 2. Data transmission by the terminal

[0457] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[0458] 3. OCR processing by the server

[0459] The server analyzes the received data and uses OCR (Optical Character Recognition) technology to recognize the characters within the data. In the case of a URL, it retrieves the HTML of the web page, and in the case of an image, it analyzes the characters within the image. This process is typically performed using an OCR engine such as Tesseract.

[0460] 4. Server applies modification rules

[0461] The server applies pre-defined modification rules based on the recognized strings. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[0462] 5. Context Analysis by the Server

[0463] The server analyzes the context surrounding the recognized string and applies appropriate corrections, using natural language processing techniques to adjust the context before and after the correction to ensure consistency.

[0464] 6. Server creates correction instructions

[0465] The server marks the corrections and automatically generates a correction instruction sheet that reflects the specific correction instructions. The correction instruction sheet contains details of the corrections and the specific correction content. The correction instruction sheet is usually generated in a format such as PDF or Word.

[0466] 7. Receiving correction instructions via terminal

[0467] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[0468] Specific examples

[0469] Rename a web page

[0470] User Action:

[0471] The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[0472] Terminal behavior:

[0473] The device sends the URL "https: / / example.com / page1" to the server.

[0474] Server Action:

[0475] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[0476] Terminal behavior:

[0477] The terminal receives the correction instructions and displays them to the user.

[0478] Changes in paper media conditions

[0479] User Action:

[0480] The user uploads a captured image of the paper media to the system.

[0481] Terminal behavior:

[0482] The terminal transmits the image data to the server.

[0483] Server Action:

[0484] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[0485] Terminal behavior:

[0486] The terminal receives the correction instructions and displays them to the user.

[0487] As described above, the system of the present invention prevents omissions of corrections and inconsistencies in notation, and realizes efficient creation of correction instructions.

[0488] The processing flow will be explained below.

[0489] Step 1:

[0490] A user accesses the system interface and inputs the URL of the web page to be modified or a captured image of the paper medium. For example, the user inputs the URL of the web page "https: / / example.com / page1" or uploads an image of the paper medium.

[0491] Step 2:

[0492] The device sends the entered URL or captured image to the server via an HTTP request, with the data usually sent in JSON or binary format.

[0493] Step 3:

[0494] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[0495] Step 4:

[0496] The server uses OCR technology to recognize characters from HTML pages and captured images. In the case of images, an OCR engine such as Tesseract is used to extract character data.

[0497] Step 5:

[0498] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[0499] Step 6:

[0500] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[0501] Step 7:

[0502] The server marks the areas to be corrected and automatically generates a correction instruction sheet, which clearly describes the identified areas to be corrected and the specific corrections to be made.

[0503] Step 8:

[0504] The terminal receives the generated correction instructions from the server as an HTTP response.

[0505] Step 9:

[0506] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[0507] Through the above steps, this system efficiently creates correction instructions and prevents omissions of corrections and inconsistencies in notation.

[0508] Example 1

[0509] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0510] Conventional correction work is often done manually, which can lead to problems such as missing corrections, inconsistencies in spelling, and inconsistencies in context. Furthermore, there is a lack of systems that can efficiently correct both paper and digital content. Therefore, there is a need to improve the efficiency and accuracy of correction work.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0512] In this invention, the server includes means for recognizing characters from input data using OCR technology, means for applying specified correction rules based on the recognized character string, and means for analyzing the context using natural language processing technology, which makes it possible to prevent omissions and inconsistencies in spelling and automatically make corrections that are consistent with the context.

[0513] "User" refers to the person or end user who uses the system to input data to be corrected and checks the correction results.

[0514] "Content identification information" refers to information that uniquely identifies the digital data to be modified, such as the URL of a web page or the ID of a digital document.

[0515] "Digital data" refers to data expressed in a format that can be processed by a computer, and includes, for example, text, images, audio, and video.

[0516] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that has the ability to communicate with the system and send and receive data.

[0517] "Server" refers to a computer system that receives requests from terminals, processes the input information, and generates correction instructions.

[0518] "OCR technology" is an abbreviation for Optical Character Recognition, and refers to the technology that analyzes characters in an image and converts them into text data.

[0519] The "modification rule" refers to a rule for replacing a specific character string or information with another character string or information, or a rule for adding necessary information or deleting unnecessary information.

[0520] "Natural language processing technology" refers to computer technology for understanding, analyzing, and generating human language, and includes, for example, algorithms for context analysis and semantic understanding.

[0521] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and is usually generated in a digital format such as PDF or Word.

[0522] A "prompt" is text used to input instructions or questions to a model, and refers to a document that specifically instructs an AI model on what to do.

[0523] An "AI model" refers to an algorithm or neural network that learns from data and automatically performs specific tasks.

[0524] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented as follows.

[0525] The system consists of multiple components: users, terminals, and servers. Users access the system using their terminals and enter the data to be corrected. The data to be corrected can be a web page URL or a captured image of a paper document.

[0526] The device (PC, smartphone, etc.) provides a means to send the URL or captured image entered by the user to the server. Data is sent in JSON or binary format using an HTTP request.

[0527] The server performs various processes using the following hardware and software:

[0528] Hardware: A server with a powerful processor, sufficient memory and storage

[0529] software:

[0530] OCR Technology: Tesseract OCR Engine

[0531] Natural language processing technology: NLP libraries (e.g., SpaCy, NLTK)

[0532] Data processing: Programs written in Python, Java, etc.

[0533] The system processing flow is as follows:

[0534] The server receives the data sent from the terminal and first applies OCR technology to recognize characters from the input data. In this step, character recognition is performed from an image using the Tesseract OCR engine. For example, character data is extracted from a captured image of a paper document uploaded by the user.

[0535] The server then applies pre-defined modification rules based on the recognized strings, which may include replacing specific strings, removing unnecessary information, or adding necessary information, such as replacing "old service name" with "new service name."

[0536] The server then uses natural language processing technology to analyze the context surrounding the recognized string. This allows for appropriate corrections to be made that are consistent with the context. In this step, a generative AI model is used to perform the appropriate context analysis. Specifically, the generative AI model is prompted with the following instruction: "Please make appropriate corrections so that the entire paragraph flows properly."

[0537] The server then marks the corrections and automatically generates a correction instruction sheet, which includes details of the corrections and specific correction content, and is usually generated in PDF or Word format, allowing the user to receive clear correction instructions.

[0538] Finally, the terminal receives the generated correction instructions from the server via an HTTP response and provides them to the user, who can then check the correction instructions and perform the actual correction work.

[0539] Specific examples

[0540] Rename a web page

[0541] User action: The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[0542] What the device does: Sends the URL in JSON format to the server.

[0543] Server process: Obtain HTML and analyze the text with Tesseract. Replace "old service name" with "new service name" and adjust using context analysis. Generate a PDF of the correction instructions.

[0544] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[0545] Changes in paper media conditions

[0546] User Action: Upload a captcha image.

[0547] Device operation: Sends image data to the server.

[0548] Server processing: Recognize text in the image using Tesseract. Replace "old conditions" with "new conditions" and adjust using context analysis. Generate a PDF of correction instructions.

[0549] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[0550] Through the above process, this system is able to carry out correction work efficiently and with high accuracy.

[0551] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0552] Step 1:

[0553] The user enters the data to be modified. The user selects the URL of a web page or a captured image of a paper document in the system interface and clicks the submit button. For example, the user can enter the URL "https: / / example.com / page1" or upload a captured image file. This operation inputs the web page URL or image data into the system.

[0554] input:

[0555] Webpage URL

[0556] Captured image data

[0557] output:

[0558] Send button pressed event

[0559] Step 2:

[0560] The terminal sends the entered data to the server. The terminal converts the entered URL or captured image into JSON or binary format and sends it to the server via an HTTP request. The event of pressing the send button triggers the sending of JSON format data to the server.

[0561] input:

[0562] Web page URL or captured image data

[0563] Send button pressed event

[0564] output:

[0565] HTTP request (sent by server)

[0566] Step 3:

[0567] The server analyzes the data using OCR technology. If the received URL is a webpage, the server retrieves the HTML and applies OCR technology to extract the text data. If the captured image is a text file, the server uses the Tesseract engine to recognize characters from the image. This process generates the text data.

[0568] input:

[0569] HTTP request (web page URL and captured image data)

[0570] output:

[0571] Text data

[0572] Step 4:

[0573] The server applies pre-defined correction rules. Based on the recognized text data, the server replaces specific strings with other strings, deletes unnecessary information, and adds necessary information. At this stage, the server refers to correction rules stored in a specific memory or database. For example, it replaces "old service name" with "new service name."

[0574] input:

[0575] Text data

[0576] output:

[0577] Corrected text data

[0578] Step 5:

[0579] The server analyzes the context using natural language processing technology. The server uses a generative AI model to analyze the context of the corrected text data and make appropriate adjustments to maintain consistency. The prompt used is "Please make appropriate corrections so that the entire paragraph flows properly."

[0580] input:

[0581] Corrected text data

[0582] output:

[0583] Contextually aligned text data

[0584] Step 6:

[0585] The server creates a correction instruction document. The server marks all the corrections and generates a correction instruction document that summarizes the specific corrections. This instruction document is usually created in PDF or Word format. A PDF generation library is used to generate the instruction document.

[0586] input:

[0587] Contextually aligned text data

[0588] output:

[0589] Correction Instructions (PDF or Word)

[0590] Step 7:

[0591] The terminal receives the correction instructions and provides them to the user. The terminal receives the HTTP response from the server and displays the correction instructions to the user. The user uses these as a reference to perform the actual correction work.

[0592] input:

[0593] Correction Instructions (PDF or Word)

[0594] output:

[0595] Display of correction instructions

[0596] By following the above steps, the system can efficiently carry out correction work.

[0597] (Application example 1)

[0598] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0599] Corrections to work instructions and manuals in factories are often done manually, which can lead to problems such as missing corrections and inconsistencies in notation. Furthermore, the significant time and effort required for corrections reduces work efficiency. There is a need for a system that can solve these problems and streamline the process of correcting work instructions in factories.

[0600] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0601] In this invention, the server includes: a means for a user to input the URL of a web page to be corrected or a captured image of a paper medium into the system; a means for a terminal to send the input URL or image data to the server; a means for the server to recognize characters from the input data using OCR technology; a means for the server to apply specified correction rules based on the recognized character strings; a means for the server to analyze the context and generate appropriate correction instructions; a means for the server to mark the corrections and create a correction instruction; a means for the terminal to receive the created correction instruction from the server and provide it to the user as a factory work instruction; and a means for using a camera of a factory robot to take a capture image of the paper medium and input it into the system. This prevents omissions and inconsistencies in work instructions and enables efficient correction work.

[0602] A "user" is a person who uses the system to input the URL of a web page to be modified or a captured image of a paper medium.

[0603] A "terminal" is a device that sends URLs and image data entered by a user to a server and receives correction instructions from the server.

[0604] The "server" is a system that uses OCR technology to recognize characters from input data, applies specified correction rules to analyze the context, and generates correction instructions and creates a correction instruction document.

[0605] "OCR technology" is a technology that optically recognizes characters in an image and converts them into digital text information.

[0606] "Correction rules" are rules that are applied based on recognized character strings and define operations for replacing specific character strings with other character strings, or for adding or deleting necessary information.

[0607] "Context analysis" is a process in which the context surrounding the recognized character string is analyzed using natural language processing technology and appropriate corrections are made.

[0608] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and marks the areas that require correction.

[0609] The "factory robot camera" is a camera built into a robot used in a factory, and is a device that takes captured images of paper media.

[0610] A "factory work instruction" is a document that describes work instructions within a factory, and provides workers with the latest work content based on the correction instruction.

[0611] This invention is a system for streamlining the work of correcting work instructions and manuals in factories. This system automates a series of processes, from the stage where a user inputs the URL of a web page to be corrected or a captured image of a paper medium, to the stage where correction instructions are generated and the updated work instructions are provided within the factory.

[0612] The system includes a user, a terminal, a server, and a factory robot's camera. The user enters the URL to be corrected or a captured image of the paper media into the system's interface. The input data is sent by the terminal to the server. The server uses OCR technology to recognize characters from the input data and then applies specified correction rules based on the recognized characters. The server then analyzes the context and generates appropriate correction instructions. Finally, it creates a correction instruction sheet containing the marked-up corrections and sends it to the terminal. The terminal then provides the created correction instruction sheet to the user as a factory work instruction sheet.

[0613] The server uses the following hardware and software: The Tesseract OCR engine is used for OCR processing, and PIL (Python Imaging Library) is used as the image processing library. The requests library is used to send and receive HTTP requests. A general NLP library is used for context analysis using natural language processing (NLP).

[0614] The terminal has a function to send data entered by the user to the server in JSON or binary format, receive correction instructions from the server, and provide them to the user as factory work instructions.The factory robot's camera is also used to take capture images of paper media and input the image data into the system.

[0615] For example, if a manual used in a factory contains a description of "old process A" and needs to be changed to a new "new process B," a captured image can be uploaded to the system. The system then automatically corrects "old process A" to "new process B," and a robot presents the new work instructions to the worker. Examples of input prompts for the generative AI model include the following:

[0616] "Please modify the following text. Original text: For this task, please use old process A. Old part X is also required. Modification instructions: Please modify old process A to new process B and old part X to new part Y."

[0617] As described above, the system of the present invention can improve the efficiency of work instruction correction work in factories and prevent oversight of corrections and inconsistencies in notation.

[0618] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0619] Step 1:

[0620] The user inputs the URL of the web page to be edited or a captured image of the paper document into the system. The input can be the URL of the web page (e.g., "https: / / example.com / page1") or a captured image of the paper document. The terminal receives this input data.

[0621] Step 2:

[0622] The device sends the entered URL and image data to the server. The URL and image data are sent to the server in JSON or binary format via an HTTP request. The server receives them and begins processing.

[0623] Step 3:

[0624] The server uses OCR technology to recognize characters from the input data. For example, the Tesseract OCR engine is used to extract text information from captured images of paper media. The input is image data, and the output is a string of characters.

[0625] Step 4:

[0626] The server applies the specified modification rules based on the recognized string. The modification rules include, for example, replacing "old process A" with "new process B." The input is the recognized string, and the output is the modified string.

[0627] Step 5:

[0628] The server analyzes the context and generates appropriate correction instructions. It uses natural language processing technology to analyze the context and make corrections appropriate to the context. The input is the corrected string, and the output is the appropriate correction string after context analysis.

[0629] Step 6:

[0630] The server marks the corrections and creates a correction instruction sheet. The correction instruction sheet contains details of the corrections and specific correction content, and is generated in PDF or Word format, for example. The input is the correction string after context analysis, and the output is the correction instruction sheet.

[0631] Step 7:

[0632] The terminal receives the created correction instructions from the server and provides them to the user as factory work instructions. The user refers to these to perform the actual correction work. The input is the correction instructions, and the output is the factory work instructions that the user refers to.

[0633] Step 8:

[0634] A factory robot's camera is used to capture an image of the paper medium and input it into the system. The captured image data is then subjected to OCR processing as a target for correction. The input is the image of the paper medium, and the output is the image data input into the system.

[0635] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0636] The present invention combines a system for efficiently editing web pages and paper media with an emotion engine that recognizes the user's emotions and responds accordingly. The system is implemented as follows:

[0637] System Embodiments

[0638] 1. User data entry

[0639] The user accesses the system interface and enters the URL of the web page or a captured image of the paper to be edited. For example, the user can enter the URL of the web page "https: / / example.com / page1" or upload an image of the paper.

[0640] 2. Data transmission by the terminal

[0641] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[0642] 3. Emotional engine for recognizing user emotions

[0643] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, and the emotion engine analyzes these to recognize the user's emotions. The emotion engine uses facial recognition and voice analysis technologies to identify the user's emotional state, such as joy, anger, sadness, or happiness.

[0644] 4. OCR processing by the server

[0645] The server receives the input data, retrieves the HTML of the corresponding web page in the case of a URL, or prepares the image for analysis in the case of a paper image. The server then uses OCR technology to recognize characters within the data. For example, an OCR engine such as Tesseract can be used to extract the character data.

[0646] 5. Server applies modification rules

[0647] The server applies the configured modification rules based on the recognized string. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[0648] 6. Context Analysis by the Server

[0649] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[0650] 7. Server creates correction instructions

[0651] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[0652] 8. Adjusting correction instructions with the emotion engine

[0653] The emotion engine adjusts the content and format of the correction instructions based on the user's emotions, for example, simplifying the instructions or providing additional support information if the user is feeling stressed.

[0654] 9. Receiving correction instructions via terminal

[0655] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[0656] Specific examples

[0657] Rename a web page

[0658] User Action:

[0659] The user enters "https: / / example.com / page1" into the system interface and presses the submit button. The video and audio recorded during the input are captured and analyzed by the emotion engine.

[0660] Terminal behavior:

[0661] The device sends the URL "https: / / example.com / page1" to the server.

[0662] Server Action:

[0663] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[0664] Emotion Engine Processing:

[0665] The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations and support if the user is feeling anxious.

[0666] Terminal behavior:

[0667] The terminal receives the correction instructions and displays them to the user.

[0668] Changes in paper media conditions

[0669] User Action:

[0670] Users upload captured images of paper to the system, and the emotion engine analyzes the user's emotions as they are input.

[0671] Terminal behavior:

[0672] The terminal transmits the image data to the server.

[0673] Server Action:

[0674] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[0675] Emotion Engine Processing:

[0676] An emotion engine adjusts the format and content of correction instructions based on the user's emotional state, for example generating simple instructions if the user is frustrated.

[0677] Terminal behavior:

[0678] The terminal receives the correction instructions and displays them to the user.

[0679] As described above, by combining an emotion engine, it becomes possible to provide correction instructions that take into account the user's emotional state, thereby realizing a more user-friendly system.

[0680] The processing flow will be explained below.

[0681] Step 1:

[0682] A user accesses the system interface and enters the URL of the web page to be edited or a captured image of the paper. For example, the user enters the URL of the web page "https: / / example.com / page1" or uploads an image of the paper.

[0683] Step 2:

[0684] The device sends the entered URL or captured image to the server via an HTTP request, and the data is usually sent in JSON or binary format.

[0685] Step 3:

[0686] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, thereby obtaining the user's emotional data.

[0687] Step 4:

[0688] The device transmits the acquired emotion data to the emotion engine, which analyzes it and identifies the user's emotion.

[0689] Step 5:

[0690] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[0691] Step 6:

[0692] The server uses OCR technology to recognize characters from the HTML of the web page or the captured image, for example, by using an OCR engine such as Tesseract to extract the character data.

[0693] Step 7:

[0694] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[0695] Step 8:

[0696] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[0697] Step 9:

[0698] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[0699] Step 10:

[0700] The emotion engine adjusts the content and format of the correction instructions based on the user's emotional data. For example, if the user is feeling stressed, the instructions may be simplified or additional support information may be provided.

[0701] Step 11:

[0702] The terminal receives the generated correction instructions from the server as an HTTP response.

[0703] Step 12:

[0704] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[0705] Through these steps, the system efficiently creates correction instructions and takes into account the user's emotional state to prevent omissions and inconsistencies in spelling. Furthermore, by responding to emotions, the system becomes more user-friendly.

[0706] Example 2

[0707] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0708] Conventional editing work on web pages and paper documents is time-consuming and inefficient. Furthermore, no system exists that takes into account the user's emotions, and it is not possible to appropriately address the stress and anxiety users feel while editing. Therefore, there is a need for a user-friendly editing system.

[0709] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0710] In this invention, the server

[0711] means for recognizing characters from the input data using optical character recognition techniques;

[0712] means for applying specified modification rules based on the recognized characters;

[0713] a means for analyzing the context of the recognized character using natural language processing technology;

[0714] and means for recognizing the user's emotional state using emotion recognition technology and adjusting the content and format of the correction instructions.

[0715] This makes it possible to carry out efficient and appropriate correction work that takes into account the user's feelings.

[0716] A "user" is an entity that operates the system and inputs digital information to be modified.

[0717] A "terminal" is a device operated by a user, such as a PC or smartphone, and is responsible for sending input data to a server.

[0718] A "server" is a central processing unit that receives data sent from a terminal and performs various processes.

[0719] Optical character recognition (OCR) is a technology that recognizes characters from image data and converts them into text data.

[0720] "Characters" are elements such as letters and symbols recognized by OCR technology.

[0721] A "correction rule" is an operation rule such as substitution, addition, or deletion that is applied to a recognized character string.

[0722] "Context" refers to the text that exists before and after a particular string of characters, and is information that helps determine its meaning and appropriateness.

[0723] "Natural language processing technology" is a set of technologies that enable computers to understand, analyze, and generate human language.

[0724] "Emotion recognition technology" is a technology for identifying and classifying a user's emotional state from facial expressions, voice, etc.

[0725] A "correction instruction sheet" is a document that marks the corrections and clearly states the corrections.

[0726] The present invention is an efficient system for modifying digital information that takes into account the user's feelings, and is implemented in the following manner.

[0727] First, the user accesses the system interface and enters the digital information to be modified (this can be a web page URL, e.g., "https: / / example.com / page1," or a captured image of a paper document). The user enters this information and presses the submit button.

[0728] Next, the device (user's PC or smartphone) sends the entered URL and image data to the server. The data is usually sent in JSON or binary format. This allows the data to be passed to the server.

[0729] The server analyzes the received URL or image data. In the case of a URL, it retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares the image for analysis. This analysis uses optical character recognition (OCR) technology such as Tesseract to recognize characters from the input data. It then applies correction rules based on the recognized characters. Correction rules can include replacing specific strings with other strings, or adding or deleting necessary information. This is done using Python scripts or regular expressions (RegEx).

[0730] The server then analyzes the surrounding context using natural language processing techniques (such as SpaCy or NLTK) to determine whether the revised text is contextually appropriate. If not, it makes further adjustments.

[0731] Additionally, the present invention incorporates emotion recognition technology. The device captures the user's facial expressions and voice through a camera and microphone, which are then analyzed by the emotion engine. Here, facial expression recognition technology (OpenCV) and voice analysis technology (Google Cloud Speech-to-Text) are used to recognize the user's emotions. The emotion engine then adjusts the content and format of the correction instructions based on the user's emotions. For example, if the device detects that the user is anxious, it can simplify the instructions or provide additional explanations.

[0732] Once all the processing is complete, the correction instructions are generated in a format such as PDF or Word and sent to the terminal. The user can refer to this and carry out the necessary corrections.

[0733] Specific examples

[0734] Rename a web page

[0735] User operation: The user enters "https: / / example.com / page1" into the system interface and presses the submit button. The video and audio input are captured and analyzed by the emotion engine.

[0736] Device behavior: The device sends the URL "https: / / example.com / page1" to the server.

[0737] Server processing: The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[0738] Emotion engine processing: The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations or support if the user is feeling anxious.

[0739] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[0740] Example prompt sentence:

[0741] "Enter a webpage URL to generate correction instructions. For example, enter "https: / / example.com / page1". Please keep your facial expressions and voice natural while capturing. Emotional data will be used to refine your correction instructions."

[0742] Changes in paper media conditions

[0743] User operation: The user uploads a captured image of a paper document to the system, and the emotion engine analyzes the user's emotions as they are input.

[0744] Terminal operation: The terminal sends image data to the server.

[0745] Server processing: The server uses OCR technology to recognize the characters in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[0746] Emotion engine processing: The emotion engine adjusts the format and content of the correction instructions based on the user's emotional state. For example, if the user is frustrated, it will generate simple instructions.

[0747] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[0748] Example prompt sentence:

[0749] "Upload a paper capture image and replace the old conditions with the new ones. Please make sure the image is clear and you remain calm while typing. Emotional data will affect the conciseness of the correction instructions."

[0750] As described above, the system of the present invention makes it possible to carry out correction work efficiently while taking into consideration the user's feelings.

[0751] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0752] Step 1:

[0753] The user accesses the system interface and enters the digital information to be modified (a web page URL or a captured image of a paper document).

[0754] Input: For web pages, a URL such as "https: / / example.com / page1". For paper documents, a capture image file.

[0755] Specific operation: The user opens a browser and enters a URL into the input form, or takes a photo of a paper document with the smartphone camera and uploads the image file.

[0756] Output: The input URL or image data.

[0757] Step 2:

[0758] The device sends the entered URL and image data to the server.

[0759] Input: URLs and captcha images entered by the user.

[0760] Specific operation: The terminal creates data in JSON or binary format and sends it to the server address using a POST request.

[0761] Output: URL or image data sent to the server.

[0762] Step 3:

[0763] The server analyzes the URL or captcha image received.

[0764] Input: URL or image data received by the server.

[0765] Specific operation: In the case of a URL, the server retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares for OCR processing.

[0766] Output: HTML content of a web page or image data to be OCR processed.

[0767] Step 4:

[0768] The emotion engine captures the user's facial expressions and voice through the device and recognizes their emotional state.

[0769] Input: Captured facial and voice data of the user.

[0770] Specific operation: Use OpenCV to obtain facial expression data. Use Google Cloud Speech-to-Text to analyze audio data.

[0771] Output: The user's emotional state (e.g., happiness, anger, sadness, anxiety, irritation).

[0772] Step 5:

[0773] The server uses optical character recognition technology (OCR) to recognize characters within the image data.

[0774] Input: Image data to be OCR processed.

[0775] What it does: The server calls the Tesseract library to extract text from the image, using functions like pytesseract.image_to_string(image) .

[0776] Output: OCR recognized text data.

[0777] Step 6:

[0778] The server applies the specified modification rules based on the recognized string.

[0779] Input: Text data recognized by OCR.

[0780] Specific operation: Using Python scripts and regular expressions (RegEx), for example, replace "old service name" with "new service name". Implemented as follows: re.sub(r'\bold service name\b', 'new service name', text).

[0781] Output: Text data with correction rules applied.

[0782] Step 7:

[0783] The server uses natural language processing technology to analyze the context and make appropriate adjustments.

[0784] Input: Text data with correction rules applied.

[0785] Specific operation: Use SpaCy or NLTK to analyze the context and identify areas that need fine-tuning. Execute nlp = spacy.load('en_core_web_sm'); doc = nlp(text) etc.

[0786] Output: Contextually adjusted text data.

[0787] Step 8:

[0788] The server marks the corrections and generates a correction instruction.

[0789] Input: Contextually adjusted text data.

[0790] What it does: Generate a PDF or Word document with the revisions highlighted. For example, generate a PDF using ReportLab. Use canvas.drawString(x, y, revised_text) or similar.

[0791] Output: Generated correction instructions (PDF and Word formats).

[0792] Step 9:

[0793] An emotional engine adjusts the content and format of revision instructions based on the user's emotional state.

[0794] Input: Generated correction instructions and the user's emotional state.

[0795] Specific behavior: If the user is feeling anxious, simplify the instructions or provide additional support information. Implemented as follows: if user_emotion == 'anxiety': add_support_info(document) .

[0796] Output: Adjusted correction instructions.

[0797] Step 10:

[0798] The terminal receives the generated correction instructions from the server and displays them to the user.

[0799] Input: Correction instructions sent from the server.

[0800] Specific operation: Receive the correction instructions in the HTTP response and display them to the user in a browser or dedicated app. For example, use the download_document(response) function.

[0801] Output: Correction instructions displayed to the user.

[0802] The above is the specific processing flow of this system.

[0803] (Application example 2)

[0804] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0805] In factories, there is a demand for accurate and fast correction work on web pages and paper documents. However, conventional systems do not take user emotions into consideration, which can result in poor user-friendliness and reduced efficiency. The present invention aims to provide a technology that recognizes user emotions and responds accordingly, thereby improving the efficiency of correction work and user satisfaction.

[0806] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input a URL or digital image of an information source to be corrected to the system; means for a terminal to transmit the input URL or image data to the server; means for the server to recognize characters from the input data using character recognition technology; means for the server to apply specified correction rules based on the recognized character string; means for the server to analyze the context and generate appropriate correction instructions; means for the server to mark the corrections and create a correction instruction; means for the terminal to receive the created correction instruction from the server; means for the system to recognize the user's emotions; and means for the server to adjust the content and format of the correction instruction based on the emotion data. This makes it possible to provide correction instructions that take the user's emotions into consideration, thereby improving the efficiency of correction work on the factory floor and user satisfaction.

[0807] "User" means the person or operator who uses the system to enter information to be corrected and receives correction instructions.

[0808] A "URL" is an address that indicates the location of a resource on the Internet, such as a web page.

[0809] A "digital image" is an image that captures a paper document or other visual information and stores it as an electronic file.

[0810] "System" refers to a set of hardware and software that performs correction work based on information entered by the user and generates and provides correction instructions.

[0811] "Terminal" means an electronic device used by a User to input and transmit information and receive correction instructions.

[0812] A "server" is a central device that receives and analyzes data sent from a terminal and generates correction instructions.

[0813] "Character recognition technology" is a technology that identifies character information in an image and converts it into text data.

[0814] "Modification rules" are a set of rules for string substitution or modification based on recognized strings.

[0815] "Context" refers to the relationships between the sentences and words surrounding the recognized string.

[0816] "Emotion" refers to the psychological state or feedback that a user shows when entering information.

[0817] "Emotional Data" refers to data collected and analyzed to express a user's emotions.

[0818] A "correction instruction sheet" is a document generated by the system that lists the corrections to be made and the specific corrections to be made.

[0819] An "emotion engine" is a software component for analyzing a user's emotions.

[0820] An embodiment of the present invention will now be described. The system starts when the user inputs the URL or digital image of the information source to be corrected. The user accesses the system interface and inputs and uploads the URL of the web page to be corrected or a captured image of paper media. The terminal is responsible for sending the input URL or image data to the server.

[0821] The server receives the transmitted data and uses the character recognition technology "pytesseract" to recognize characters from the input data. This makes it possible to extract text information from captured images of paper media. The server also applies specified correction rules based on the recognized strings. These correction rules include replacing specific strings with other strings.

[0822] Additionally, the server uses a pipeline of "transformers" as natural language processing technology to analyze the context surrounding the recognized string, determining whether the corrected text is contextually appropriate and making further adjustments if necessary.

[0823] Meanwhile, the system incorporates an "emotion engine," which uses the device's camera and microphone to recognize the user's emotions during input. This emotion engine uses the "DeepFace" library to analyze the user's facial expressions and generate emotion data, which is then sent to the server to indicate the user's emotions.

[0824] The server adjusts the content and format of the correction instructions based on the emotion data. For example, if the user is frustrated, the server can simplify the correction instructions to reduce stress. If the user is anxious, the server can add detailed explanations to provide reassurance.

[0825] Finally, the server marks the corrections and automatically generates a correction instruction sheet. The generated correction instruction sheet is sent to the terminal as an HTTP response and provided to the user. The user can refer to this correction instruction sheet and perform the actual correction work.

[0826] As a concrete example, the following prompt sentence is shown.

[0827] Example prompt sentence:

[0828] "I'd like you to fix this instruction manual. I'm very busy and frustrated."

[0829] Thus, the present invention takes into account the user's feelings and provides an efficient and user-friendly correction system.

[0830] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0831] Step 1:

[0832] The user inputs the URL or digital image of the source to be modified into the system, and the input URL or captured image is sent to the server through the terminal interface.

[0833] Step 2:

[0834] The device sends the input URL or image data to the server. The sent data is transferred to the server in JSON or binary format. The server receives the input data and starts the next process.

[0835] Step 3:

[0836] The server analyzes the received data using optical character recognition (OCR). Specifically, it uses the "pytesseract" library to extract text from digital images. If the input data is a URL, it retrieves the relevant text from the HTML of the web page.

[0837] Step 4:

[0838] The server applies modification rules based on the recognized strings. The server performs operations to replace certain strings with other strings. For example, it replaces "old service name" with "new service name".

[0839] Step 5:

[0840] The server uses natural language processing techniques to analyze the context of the string, using a pipeline of "transformers" to determine whether the revised text is contextually appropriate and make further adjustments if necessary.

[0841] Step 6:

[0842] The system's terminal uses a camera and microphone to recognize the user's emotions. The emotion engine uses the "DeepFace" library to analyze emotions from the user's facial expressions and voice. The analyzed emotion data is sent to the server.

[0843] Step 7:

[0844] The server adjusts the content and format of the correction instructions based on the emotion data, for example, if the user is frustrated, the server adjusts the correction instructions to be briefer and less stressful.

[0845] Step 8:

[0846] The server marks the corrections and automatically generates a correction instruction sheet, which is created in PDF, Word, or other formats.

[0847] Step 9:

[0848] The terminal receives the generated correction instructions from the server and displays them to the user, who can then refer to the correction instructions to carry out the actual correction work.

[0849] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0850] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0851] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0852] [Third embodiment]

[0853] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0854] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0855] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0856] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0857] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0858] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0859] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0860] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0861] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0862] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0863] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0864] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0865] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented in the following manner.

[0866] System Embodiments

[0867] 1. User data entry

[0868] Through the system interface, the user selects the URL of the web page or a captured image of the paper document to be edited and inputs it into the system. For example, the user can upload the URL of the web page "https: / / example.com / page1" or a captured image of the paper document.

[0869] 2. Data transmission by the terminal

[0870] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[0871] 3. OCR processing by the server

[0872] The server analyzes the received data and uses OCR (Optical Character Recognition) technology to recognize the characters within the data. In the case of a URL, it retrieves the HTML of the web page, and in the case of an image, it analyzes the characters within the image. This process is typically performed using an OCR engine such as Tesseract.

[0873] 4. Server applies modification rules

[0874] The server applies pre-defined modification rules based on the recognized strings. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[0875] 5. Context Analysis by the Server

[0876] The server analyzes the context surrounding the recognized string and applies appropriate corrections, using natural language processing techniques to adjust the context before and after the correction to ensure consistency.

[0877] 6. Server creates correction instructions

[0878] The server marks the corrections and automatically generates a correction instruction sheet that reflects the specific correction instructions. The correction instruction sheet contains details of the corrections and the specific correction content. The correction instruction sheet is usually generated in a format such as PDF or Word.

[0879] 7. Receiving correction instructions via terminal

[0880] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[0881] Specific examples

[0882] Rename a web page

[0883] User Action:

[0884] The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[0885] Terminal behavior:

[0886] The device sends the URL "https: / / example.com / page1" to the server.

[0887] Server Action:

[0888] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[0889] Terminal behavior:

[0890] The terminal receives the correction instructions and displays them to the user.

[0891] Changes in paper media conditions

[0892] User Action:

[0893] The user uploads a captured image of the paper media to the system.

[0894] Terminal behavior:

[0895] The terminal transmits the image data to the server.

[0896] Server Action:

[0897] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[0898] Terminal behavior:

[0899] The terminal receives the correction instructions and displays them to the user.

[0900] As described above, the system of the present invention prevents omissions of corrections and inconsistencies in notation, and realizes efficient creation of correction instructions.

[0901] The processing flow will be explained below.

[0902] Step 1:

[0903] A user accesses the system interface and inputs the URL of the web page to be modified or a captured image of the paper medium. For example, the user inputs the URL of the web page "https: / / example.com / page1" or uploads an image of the paper medium.

[0904] Step 2:

[0905] The device sends the entered URL or captured image to the server via an HTTP request, with the data usually sent in JSON or binary format.

[0906] Step 3:

[0907] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[0908] Step 4:

[0909] The server uses OCR technology to recognize characters from HTML pages and captured images. In the case of images, an OCR engine such as Tesseract is used to extract character data.

[0910] Step 5:

[0911] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[0912] Step 6:

[0913] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[0914] Step 7:

[0915] The server marks the areas to be corrected and automatically generates a correction instruction sheet, which clearly describes the identified areas to be corrected and the specific corrections to be made.

[0916] Step 8:

[0917] The terminal receives the generated correction instructions from the server as an HTTP response.

[0918] Step 9:

[0919] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[0920] Through the above steps, this system efficiently creates correction instructions and prevents omissions of corrections and inconsistencies in notation.

[0921] Example 1

[0922] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0923] Conventional correction work is often done manually, which can lead to problems such as missing corrections, inconsistencies in spelling, and inconsistencies in context. Furthermore, there is a lack of systems that can efficiently correct both paper and digital content. Therefore, there is a need to improve the efficiency and accuracy of correction work.

[0924] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0925] In this invention, the server includes means for recognizing characters from input data using OCR technology, means for applying specified correction rules based on the recognized character string, and means for analyzing the context using natural language processing technology, which makes it possible to prevent omissions and inconsistencies in spelling and automatically make corrections that are consistent with the context.

[0926] "User" refers to the person or end user who uses the system to input data to be corrected and checks the correction results.

[0927] "Content identification information" refers to information that uniquely identifies the digital data to be modified, such as the URL of a web page or the ID of a digital document.

[0928] "Digital data" refers to data expressed in a format that can be processed by a computer, and includes, for example, text, images, audio, and video.

[0929] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that has the ability to communicate with the system and send and receive data.

[0930] "Server" refers to a computer system that receives requests from terminals, processes the input information, and generates correction instructions.

[0931] "OCR technology" is an abbreviation for Optical Character Recognition, and refers to the technology that analyzes characters in an image and converts them into text data.

[0932] The "modification rule" refers to a rule for replacing a specific character string or information with another character string or information, or a rule for adding necessary information or deleting unnecessary information.

[0933] "Natural language processing technology" refers to computer technology for understanding, analyzing, and generating human language, and includes, for example, algorithms for context analysis and semantic understanding.

[0934] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and is usually generated in a digital format such as PDF or Word.

[0935] A "prompt" is text used to input instructions or questions to a model, and refers to a document that specifically instructs an AI model on what to do.

[0936] An "AI model" refers to an algorithm or neural network that learns from data and automatically performs specific tasks.

[0937] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented as follows.

[0938] The system consists of multiple components: users, terminals, and servers. Users access the system using their terminals and enter the data to be corrected. The data to be corrected can be a web page URL or a captured image of a paper document.

[0939] The device (PC, smartphone, etc.) provides a means to send the URL or captured image entered by the user to the server. Data is sent in JSON or binary format using an HTTP request.

[0940] The server performs various processes using the following hardware and software:

[0941] Hardware: A server with a powerful processor, sufficient memory and storage

[0942] software:

[0943] OCR Technology: Tesseract OCR Engine

[0944] Natural language processing technology: NLP libraries (e.g., SpaCy, NLTK)

[0945] Data processing: Programs written in Python, Java, etc.

[0946] The system processing flow is as follows:

[0947] The server receives the data sent from the terminal and first applies OCR technology to recognize characters from the input data. In this step, character recognition is performed from an image using the Tesseract OCR engine. For example, character data is extracted from a captured image of a paper document uploaded by the user.

[0948] The server then applies pre-defined modification rules based on the recognized strings, which may include replacing specific strings, removing unnecessary information, or adding necessary information, such as replacing "old service name" with "new service name."

[0949] The server then uses natural language processing technology to analyze the context surrounding the recognized string. This allows for appropriate corrections to be made that are consistent with the context. In this step, a generative AI model is used to perform the appropriate context analysis. Specifically, the generative AI model is prompted with the following instruction: "Please make appropriate corrections so that the entire paragraph flows properly."

[0950] The server then marks the corrections and automatically generates a correction instruction sheet, which includes details of the corrections and specific correction content, and is usually generated in PDF or Word format, allowing the user to receive clear correction instructions.

[0951] Finally, the terminal receives the generated correction instructions from the server via an HTTP response and provides them to the user, who can then check the correction instructions and perform the actual correction work.

[0952] Specific examples

[0953] Rename a web page

[0954] User action: The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[0955] What the device does: Sends the URL in JSON format to the server.

[0956] Server process: Obtain HTML and analyze the text with Tesseract. Replace "old service name" with "new service name" and adjust using context analysis. Generate a PDF of the correction instructions.

[0957] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[0958] Changes in paper media conditions

[0959] User Action: Upload a captcha image.

[0960] Device operation: Sends image data to the server.

[0961] Server processing: Recognize text in the image using Tesseract. Replace "old conditions" with "new conditions" and adjust using context analysis. Generate a PDF of correction instructions.

[0962] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[0963] Through the above process, this system is able to carry out correction work efficiently and with high accuracy.

[0964] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0965] Step 1:

[0966] The user enters the data to be modified. The user selects the URL of a web page or a captured image of a paper document in the system interface and clicks the submit button. For example, the user can enter the URL "https: / / example.com / page1" or upload a captured image file. This operation inputs the web page URL or image data into the system.

[0967] input:

[0968] Webpage URL

[0969] Captured image data

[0970] output:

[0971] Send button pressed event

[0972] Step 2:

[0973] The terminal sends the entered data to the server. The terminal converts the entered URL or captured image into JSON or binary format and sends it to the server via an HTTP request. The event of pressing the send button triggers the sending of JSON format data to the server.

[0974] input:

[0975] Web page URL or captured image data

[0976] Send button pressed event

[0977] output:

[0978] HTTP request (sent by server)

[0979] Step 3:

[0980] The server analyzes the data using OCR technology. If the received URL is a webpage, the server retrieves the HTML and applies OCR technology to extract the text data. If the captured image is a text file, the server uses the Tesseract engine to recognize characters from the image. This process generates the text data.

[0981] input:

[0982] HTTP request (web page URL and captured image data)

[0983] output:

[0984] Text data

[0985] Step 4:

[0986] The server applies pre-defined correction rules. Based on the recognized text data, the server replaces specific strings with other strings, deletes unnecessary information, and adds necessary information. At this stage, the server refers to correction rules stored in a specific memory or database. For example, it replaces "old service name" with "new service name."

[0987] input:

[0988] Text data

[0989] output:

[0990] Corrected text data

[0991] Step 5:

[0992] The server analyzes the context using natural language processing technology. The server uses a generative AI model to analyze the context of the corrected text data and make appropriate adjustments to maintain consistency. The prompt used is "Please make appropriate corrections so that the entire paragraph flows properly."

[0993] input:

[0994] Corrected text data

[0995] output:

[0996] Contextually aligned text data

[0997] Step 6:

[0998] The server creates a correction instruction document. The server marks all the corrections and generates a correction instruction document that summarizes the specific corrections. This instruction document is usually created in PDF or Word format. A PDF generation library is used to generate the instruction document.

[0999] input:

[1000] Contextually aligned text data

[1001] output:

[1002] Correction Instructions (PDF or Word)

[1003] Step 7:

[1004] The terminal receives the correction instructions and provides them to the user. The terminal receives the HTTP response from the server and displays the correction instructions to the user. The user uses these as a reference to perform the actual correction work.

[1005] input:

[1006] Correction Instructions (PDF or Word)

[1007] output:

[1008] Display of correction instructions

[1009] By following the above steps, the system can efficiently carry out correction work.

[1010] (Application example 1)

[1011] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1012] Corrections to work instructions and manuals in factories are often done manually, which can lead to problems such as missing corrections and inconsistencies in notation. Furthermore, the significant time and effort required for corrections reduces work efficiency. There is a need for a system that can solve these problems and streamline the process of correcting work instructions in factories.

[1013] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1014] In this invention, the server includes: a means for a user to input the URL of a web page to be corrected or a captured image of a paper medium into the system; a means for a terminal to send the input URL or image data to the server; a means for the server to recognize characters from the input data using OCR technology; a means for the server to apply specified correction rules based on the recognized character strings; a means for the server to analyze the context and generate appropriate correction instructions; a means for the server to mark the corrections and create a correction instruction; a means for the terminal to receive the created correction instruction from the server and provide it to the user as a factory work instruction; and a means for using a camera of a factory robot to take a capture image of the paper medium and input it into the system. This prevents omissions and inconsistencies in work instructions and enables efficient correction work.

[1015] A "user" is a person who uses the system to input the URL of a web page to be modified or a captured image of a paper medium.

[1016] A "terminal" is a device that sends URLs and image data entered by a user to a server and receives correction instructions from the server.

[1017] The "server" is a system that uses OCR technology to recognize characters from input data, applies specified correction rules to analyze the context, and generates correction instructions and creates a correction instruction document.

[1018] "OCR technology" is a technology that optically recognizes characters in an image and converts them into digital text information.

[1019] "Correction rules" are rules that are applied based on recognized character strings and define operations for replacing specific character strings with other character strings, or for adding or deleting necessary information.

[1020] "Context analysis" is a process in which the context surrounding the recognized character string is analyzed using natural language processing technology and appropriate corrections are made.

[1021] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and marks the areas that require correction.

[1022] The "factory robot camera" is a camera built into a robot used in a factory, and is a device that takes captured images of paper media.

[1023] A "factory work instruction" is a document that describes work instructions within a factory, and provides workers with the latest work content based on the correction instruction.

[1024] This invention is a system for streamlining the work of correcting work instructions and manuals in factories. This system automates a series of processes, from the stage where a user inputs the URL of a web page to be corrected or a captured image of a paper medium, to the stage where correction instructions are generated and the updated work instructions are provided within the factory.

[1025] The system includes a user, a terminal, a server, and a factory robot's camera. The user enters the URL to be corrected or a captured image of the paper media into the system's interface. The input data is sent by the terminal to the server. The server uses OCR technology to recognize characters from the input data and then applies specified correction rules based on the recognized characters. The server then analyzes the context and generates appropriate correction instructions. Finally, it creates a correction instruction sheet containing the marked-up corrections and sends it to the terminal. The terminal then provides the created correction instruction sheet to the user as a factory work instruction sheet.

[1026] The server uses the following hardware and software: The Tesseract OCR engine is used for OCR processing, and PIL (Python Imaging Library) is used as the image processing library. The requests library is used to send and receive HTTP requests. A general NLP library is used for context analysis using natural language processing (NLP).

[1027] The terminal has a function to send data entered by the user to the server in JSON or binary format, receive correction instructions from the server, and provide them to the user as factory work instructions.The factory robot's camera is also used to take capture images of paper media and input the image data into the system.

[1028] For example, if a manual used in a factory contains a description of "old process A" and needs to be changed to a new "new process B," a captured image can be uploaded to the system. The system then automatically corrects "old process A" to "new process B," and a robot presents the new work instructions to the worker. Examples of input prompts for the generative AI model include the following:

[1029] "Please modify the following text. Original text: For this task, please use old process A. Old part X is also required. Modification instructions: Please modify old process A to new process B and old part X to new part Y."

[1030] As described above, the system of the present invention can improve the efficiency of work instruction correction work in factories and prevent oversight of corrections and inconsistencies in notation.

[1031] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1032] Step 1:

[1033] The user inputs the URL of the web page to be edited or a captured image of the paper document into the system. The input can be the URL of the web page (e.g., "https: / / example.com / page1") or a captured image of the paper document. The terminal receives this input data.

[1034] Step 2:

[1035] The device sends the entered URL and image data to the server. The URL and image data are sent to the server in JSON or binary format via an HTTP request. The server receives them and begins processing.

[1036] Step 3:

[1037] The server uses OCR technology to recognize characters from the input data. For example, the Tesseract OCR engine is used to extract text information from captured images of paper media. The input is image data, and the output is a string of characters.

[1038] Step 4:

[1039] The server applies the specified modification rules based on the recognized string. The modification rules include, for example, replacing "old process A" with "new process B." The input is the recognized string, and the output is the modified string.

[1040] Step 5:

[1041] The server analyzes the context and generates appropriate correction instructions. It uses natural language processing technology to analyze the context and make corrections appropriate to the context. The input is the corrected string, and the output is the appropriate correction string after context analysis.

[1042] Step 6:

[1043] The server marks the corrections and creates a correction instruction sheet. The correction instruction sheet contains details of the corrections and specific correction content, and is generated in PDF or Word format, for example. The input is the correction string after context analysis, and the output is the correction instruction sheet.

[1044] Step 7:

[1045] The terminal receives the created correction instructions from the server and provides them to the user as factory work instructions. The user refers to these to perform the actual correction work. The input is the correction instructions, and the output is the factory work instructions that the user refers to.

[1046] Step 8:

[1047] A factory robot's camera is used to capture an image of the paper medium and input it into the system. The captured image data is then subjected to OCR processing as a target for correction. The input is the image of the paper medium, and the output is the image data input into the system.

[1048] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1049] The present invention combines a system for efficiently editing web pages and paper media with an emotion engine that recognizes the user's emotions and responds accordingly. The system is implemented as follows:

[1050] System Embodiments

[1051] 1. User data entry

[1052] The user accesses the system interface and enters the URL of the web page or a captured image of the paper to be edited. For example, the user can enter the URL of the web page "https: / / example.com / page1" or upload an image of the paper.

[1053] 2. Data transmission by the terminal

[1054] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[1055] 3. Emotional engine for recognizing user emotions

[1056] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, and the emotion engine analyzes these to recognize the user's emotions. The emotion engine uses facial recognition and voice analysis technologies to identify the user's emotional state, such as joy, anger, sadness, or happiness.

[1057] 4. OCR processing by the server

[1058] The server receives the input data, retrieves the HTML of the corresponding web page in the case of a URL, or prepares the image for analysis in the case of a paper image. The server then uses OCR technology to recognize characters within the data. For example, an OCR engine such as Tesseract can be used to extract the character data.

[1059] 5. Server applies modification rules

[1060] The server applies the configured modification rules based on the recognized string. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[1061] 6. Context Analysis by the Server

[1062] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[1063] 7. Server creates correction instructions

[1064] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[1065] 8. Adjusting correction instructions with the emotion engine

[1066] The emotion engine adjusts the content and format of the correction instructions based on the user's emotions, for example, simplifying the instructions or providing additional support information if the user is feeling stressed.

[1067] 9. Receiving correction instructions via terminal

[1068] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[1069] Specific examples

[1070] Rename a web page

[1071] User Action:

[1072] The user enters "https: / / example.com / page1" into the system interface and presses the submit button. The video and audio recorded during the input are captured and analyzed by the emotion engine.

[1073] Terminal behavior:

[1074] The device sends the URL "https: / / example.com / page1" to the server.

[1075] Server Action:

[1076] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[1077] Emotion Engine Processing:

[1078] The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations and support if the user is feeling anxious.

[1079] Terminal behavior:

[1080] The terminal receives the correction instructions and displays them to the user.

[1081] Changes in paper media conditions

[1082] User Action:

[1083] Users upload captured images of paper to the system, and the emotion engine analyzes the user's emotions as they are input.

[1084] Terminal behavior:

[1085] The terminal transmits the image data to the server.

[1086] Server Action:

[1087] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[1088] Emotion Engine Processing:

[1089] An emotion engine adjusts the format and content of correction instructions based on the user's emotional state, for example generating simple instructions if the user is frustrated.

[1090] Terminal behavior:

[1091] The terminal receives the correction instructions and displays them to the user.

[1092] As described above, by combining an emotion engine, it becomes possible to provide correction instructions that take into account the user's emotional state, thereby realizing a more user-friendly system.

[1093] The processing flow will be explained below.

[1094] Step 1:

[1095] A user accesses the system interface and enters the URL of the web page to be edited or a captured image of the paper. For example, the user enters the URL of the web page "https: / / example.com / page1" or uploads an image of the paper.

[1096] Step 2:

[1097] The device sends the entered URL or captured image to the server via an HTTP request, and the data is usually sent in JSON or binary format.

[1098] Step 3:

[1099] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, thereby obtaining the user's emotional data.

[1100] Step 4:

[1101] The device transmits the acquired emotion data to the emotion engine, which analyzes it and identifies the user's emotion.

[1102] Step 5:

[1103] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[1104] Step 6:

[1105] The server uses OCR technology to recognize characters from the HTML of the web page or the captured image, for example, by using an OCR engine such as Tesseract to extract the character data.

[1106] Step 7:

[1107] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[1108] Step 8:

[1109] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[1110] Step 9:

[1111] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[1112] Step 10:

[1113] The emotion engine adjusts the content and format of the correction instructions based on the user's emotional data. For example, if the user is feeling stressed, the instructions may be simplified or additional support information may be provided.

[1114] Step 11:

[1115] The terminal receives the generated correction instructions from the server as an HTTP response.

[1116] Step 12:

[1117] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[1118] Through these steps, the system efficiently creates correction instructions and takes into account the user's emotional state to prevent omissions and inconsistencies in spelling. Furthermore, by responding to emotions, the system becomes more user-friendly.

[1119] Example 2

[1120] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1121] Conventional editing work on web pages and paper documents is time-consuming and inefficient. Furthermore, no system exists that takes into account the user's emotions, and it is not possible to appropriately address the stress and anxiety users feel while editing. Therefore, there is a need for a user-friendly editing system.

[1122] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1123] In this invention, the server

[1124] means for recognizing characters from the input data using optical character recognition techniques;

[1125] means for applying specified modification rules based on the recognized characters;

[1126] a means for analyzing the context of the recognized character using natural language processing technology;

[1127] and means for recognizing the user's emotional state using emotion recognition technology and adjusting the content and format of the correction instructions.

[1128] This makes it possible to carry out efficient and appropriate correction work that takes into account the user's feelings.

[1129] A "user" is an entity that operates the system and inputs digital information to be modified.

[1130] A "terminal" is a device operated by a user, such as a PC or smartphone, and is responsible for sending input data to a server.

[1131] A "server" is a central processing unit that receives data sent from a terminal and performs various processes.

[1132] Optical character recognition (OCR) is a technology that recognizes characters from image data and converts them into text data.

[1133] "Characters" are elements such as letters and symbols recognized by OCR technology.

[1134] A "correction rule" is an operation rule such as substitution, addition, or deletion that is applied to a recognized character string.

[1135] "Context" refers to the text that exists before and after a particular string of characters, and is information that helps determine its meaning and appropriateness.

[1136] "Natural language processing technology" is a set of technologies that enable computers to understand, analyze, and generate human language.

[1137] "Emotion recognition technology" is a technology for identifying and classifying a user's emotional state from facial expressions, voice, etc.

[1138] A "correction instruction sheet" is a document that marks the corrections and clearly states the corrections.

[1139] The present invention is an efficient system for modifying digital information that takes into account the user's feelings, and is implemented in the following manner.

[1140] First, the user accesses the system interface and enters the digital information to be modified (this can be a web page URL, e.g., "https: / / example.com / page1," or a captured image of a paper document). The user enters this information and presses the submit button.

[1141] Next, the device (user's PC or smartphone) sends the entered URL and image data to the server. The data is usually sent in JSON or binary format. This allows the data to be passed to the server.

[1142] The server analyzes the received URL or image data. In the case of a URL, it retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares the image for analysis. This analysis uses optical character recognition (OCR) technology such as Tesseract to recognize characters from the input data. It then applies correction rules based on the recognized characters. Correction rules can include replacing specific strings with other strings, or adding or deleting necessary information. This is done using Python scripts or regular expressions (RegEx).

[1143] The server then analyzes the surrounding context using natural language processing techniques (such as SpaCy or NLTK) to determine whether the revised text is contextually appropriate. If not, it makes further adjustments.

[1144] Additionally, the present invention incorporates emotion recognition technology. The device captures the user's facial expressions and voice through a camera and microphone, which are then analyzed by the emotion engine. Here, facial expression recognition technology (OpenCV) and voice analysis technology (Google Cloud Speech-to-Text) are used to recognize the user's emotions. The emotion engine then adjusts the content and format of the correction instructions based on the user's emotions. For example, if the device detects that the user is anxious, it can simplify the instructions or provide additional explanations.

[1145] Once all the processing is complete, the correction instructions are generated in a format such as PDF or Word and sent to the terminal. The user can refer to this and carry out the necessary corrections.

[1146] Specific examples

[1147] Rename a web page

[1148] User operation: The user enters "https: / / example.com / page1" into the system interface and presses the submit button. The video and audio input are captured and analyzed by the emotion engine.

[1149] Device behavior: The device sends the URL "https: / / example.com / page1" to the server.

[1150] Server processing: The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[1151] Emotion engine processing: The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations or support if the user is feeling anxious.

[1152] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[1153] Example prompt sentence:

[1154] "Enter a webpage URL to generate correction instructions. For example, enter "https: / / example.com / page1". Please keep your facial expressions and voice natural while capturing. Emotional data will be used to refine your correction instructions."

[1155] Changes in paper media conditions

[1156] User operation: The user uploads a captured image of a paper document to the system, and the emotion engine analyzes the user's emotions as they are input.

[1157] Terminal operation: The terminal sends image data to the server.

[1158] Server processing: The server uses OCR technology to recognize the characters in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[1159] Emotion engine processing: The emotion engine adjusts the format and content of the correction instructions based on the user's emotional state. For example, if the user is frustrated, it will generate simple instructions.

[1160] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[1161] Example prompt sentence:

[1162] "Upload a paper capture image and replace the old conditions with the new ones. Please make sure the image is clear and you remain calm while typing. Emotional data will affect the conciseness of the correction instructions."

[1163] As described above, the system of the present invention makes it possible to carry out correction work efficiently while taking into consideration the user's feelings.

[1164] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1165] Step 1:

[1166] The user accesses the system interface and enters the digital information to be modified (a web page URL or a captured image of a paper document).

[1167] Input: For web pages, a URL such as "https: / / example.com / page1". For paper documents, a capture image file.

[1168] Specific operation: The user opens a browser and enters a URL into the input form, or takes a photo of a paper document with the smartphone camera and uploads the image file.

[1169] Output: The input URL or image data.

[1170] Step 2:

[1171] The device sends the entered URL and image data to the server.

[1172] Input: URLs and captcha images entered by the user.

[1173] Specific operation: The terminal creates data in JSON or binary format and sends it to the server address using a POST request.

[1174] Output: URL or image data sent to the server.

[1175] Step 3:

[1176] The server analyzes the URL or captcha image received.

[1177] Input: URL or image data received by the server.

[1178] Specific operation: In the case of a URL, the server retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares for OCR processing.

[1179] Output: HTML content of a web page or image data to be OCR processed.

[1180] Step 4:

[1181] The emotion engine captures the user's facial expressions and voice through the device and recognizes their emotional state.

[1182] Input: Captured facial and voice data of the user.

[1183] Specific operation: Use OpenCV to obtain facial expression data. Use Google Cloud Speech-to-Text to analyze audio data.

[1184] Output: The user's emotional state (e.g., happiness, anger, sadness, anxiety, irritation).

[1185] Step 5:

[1186] The server uses optical character recognition technology (OCR) to recognize characters within the image data.

[1187] Input: Image data to be OCR processed.

[1188] What it does: The server calls the Tesseract library to extract text from the image, using functions like pytesseract.image_to_string(image) .

[1189] Output: OCR recognized text data.

[1190] Step 6:

[1191] The server applies the specified modification rules based on the recognized string.

[1192] Input: Text data recognized by OCR.

[1193] Specific operation: Using Python scripts and regular expressions (RegEx), for example, replace "old service name" with "new service name". Implemented as follows: re.sub(r'\bold service name\b', 'new service name', text).

[1194] Output: Text data with correction rules applied.

[1195] Step 7:

[1196] The server uses natural language processing technology to analyze the context and make appropriate adjustments.

[1197] Input: Text data with correction rules applied.

[1198] Specific operation: Use SpaCy or NLTK to analyze the context and identify areas that need fine-tuning. Execute nlp = spacy.load('en_core_web_sm'); doc = nlp(text) etc.

[1199] Output: Contextually adjusted text data.

[1200] Step 8:

[1201] The server marks the corrections and generates a correction instruction.

[1202] Input: Contextually adjusted text data.

[1203] What it does: Generate a PDF or Word document with the revisions highlighted. For example, generate a PDF using ReportLab. Use canvas.drawString(x, y, revised_text) or similar.

[1204] Output: Generated correction instructions (PDF and Word formats).

[1205] Step 9:

[1206] An emotional engine adjusts the content and format of revision instructions based on the user's emotional state.

[1207] Input: Generated correction instructions and the user's emotional state.

[1208] Specific behavior: If the user is feeling anxious, simplify the instructions or provide additional support information. Implemented as follows: if user_emotion == 'anxiety': add_support_info(document) .

[1209] Output: Adjusted correction instructions.

[1210] Step 10:

[1211] The terminal receives the generated correction instructions from the server and displays them to the user.

[1212] Input: Correction instructions sent from the server.

[1213] Specific operation: Receive the correction instructions in the HTTP response and display them to the user in a browser or dedicated app. For example, use the download_document(response) function.

[1214] Output: Correction instructions displayed to the user.

[1215] The above is the specific processing flow of this system.

[1216] (Application example 2)

[1217] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1218] In factories, there is a demand for accurate and fast correction work on web pages and paper documents. However, conventional systems do not take user emotions into consideration, which can result in poor user-friendliness and reduced efficiency. The present invention aims to provide a technology that recognizes user emotions and responds accordingly, thereby improving the efficiency of correction work and user satisfaction.

[1219] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input a URL or digital image of an information source to be corrected to the system; means for a terminal to transmit the input URL or image data to the server; means for the server to recognize characters from the input data using character recognition technology; means for the server to apply specified correction rules based on the recognized character string; means for the server to analyze the context and generate appropriate correction instructions; means for the server to mark the corrections and create a correction instruction; means for the terminal to receive the created correction instruction from the server; means for the system to recognize the user's emotions; and means for the server to adjust the content and format of the correction instruction based on the emotion data. This makes it possible to provide correction instructions that take the user's emotions into consideration, thereby improving the efficiency of correction work on the factory floor and user satisfaction.

[1220] "User" means the person or operator who uses the system to enter information to be corrected and receives correction instructions.

[1221] A "URL" is an address that indicates the location of a resource on the Internet, such as a web page.

[1222] A "digital image" is an image that captures a paper document or other visual information and stores it as an electronic file.

[1223] "System" refers to a set of hardware and software that performs correction work based on information entered by the user and generates and provides correction instructions.

[1224] "Terminal" means an electronic device used by a User to input and transmit information and receive correction instructions.

[1225] A "server" is a central device that receives and analyzes data sent from a terminal and generates correction instructions.

[1226] "Character recognition technology" is a technology that identifies character information in an image and converts it into text data.

[1227] "Modification rules" are a set of rules for string substitution or modification based on recognized strings.

[1228] "Context" refers to the relationships between the sentences and words surrounding the recognized string.

[1229] "Emotion" refers to the psychological state or feedback that a user shows when entering information.

[1230] "Emotional Data" refers to data collected and analyzed to express a user's emotions.

[1231] A "correction instruction sheet" is a document generated by the system that lists the corrections to be made and the specific corrections to be made.

[1232] An "emotion engine" is a software component for analyzing a user's emotions.

[1233] An embodiment of the present invention will now be described. The system starts when the user inputs the URL or digital image of the information source to be corrected. The user accesses the system interface and inputs and uploads the URL of the web page to be corrected or a captured image of paper media. The terminal is responsible for sending the input URL or image data to the server.

[1234] The server receives the transmitted data and uses the character recognition technology "pytesseract" to recognize characters from the input data. This makes it possible to extract text information from captured images of paper media. The server also applies specified correction rules based on the recognized strings. These correction rules include replacing specific strings with other strings.

[1235] Additionally, the server uses a pipeline of "transformers" as natural language processing technology to analyze the context surrounding the recognized string, determining whether the corrected text is contextually appropriate and making further adjustments if necessary.

[1236] Meanwhile, the system incorporates an "emotion engine," which uses the device's camera and microphone to recognize the user's emotions during input. This emotion engine uses the "DeepFace" library to analyze the user's facial expressions and generate emotion data, which is then sent to the server to indicate the user's emotions.

[1237] The server adjusts the content and format of the correction instructions based on the emotion data. For example, if the user is frustrated, the server can simplify the correction instructions to reduce stress. If the user is anxious, the server can add detailed explanations to provide reassurance.

[1238] Finally, the server marks the corrections and automatically generates a correction instruction sheet. The generated correction instruction sheet is sent to the terminal as an HTTP response and provided to the user. The user can refer to this correction instruction sheet and perform the actual correction work.

[1239] As a concrete example, the following prompt sentence is shown.

[1240] Example prompt sentence:

[1241] "I'd like you to fix this instruction manual. I'm very busy and frustrated."

[1242] Thus, the present invention takes into account the user's feelings and provides an efficient and user-friendly correction system.

[1243] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1244] Step 1:

[1245] The user inputs the URL or digital image of the source to be modified into the system, and the input URL or captured image is sent to the server through the terminal interface.

[1246] Step 2:

[1247] The device sends the input URL or image data to the server. The sent data is transferred to the server in JSON or binary format. The server receives the input data and starts the next process.

[1248] Step 3:

[1249] The server analyzes the received data using optical character recognition (OCR). Specifically, it uses the "pytesseract" library to extract text from digital images. If the input data is a URL, it retrieves the relevant text from the HTML of the web page.

[1250] Step 4:

[1251] The server applies modification rules based on the recognized strings. The server performs operations to replace certain strings with other strings. For example, it replaces "old service name" with "new service name".

[1252] Step 5:

[1253] The server uses natural language processing techniques to analyze the context of the string, using a pipeline of "transformers" to determine whether the revised text is contextually appropriate and make further adjustments if necessary.

[1254] Step 6:

[1255] The system's terminal uses a camera and microphone to recognize the user's emotions. The emotion engine uses the "DeepFace" library to analyze emotions from the user's facial expressions and voice. The analyzed emotion data is sent to the server.

[1256] Step 7:

[1257] The server adjusts the content and format of the correction instructions based on the emotion data, for example, if the user is frustrated, the server adjusts the correction instructions to be briefer and less stressful.

[1258] Step 8:

[1259] The server marks the corrections and automatically generates a correction instruction sheet, which is created in PDF, Word, or other formats.

[1260] Step 9:

[1261] The terminal receives the generated correction instructions from the server and displays them to the user, who can then refer to the correction instructions to carry out the actual correction work.

[1262] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1263] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1264] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1265] [Fourth embodiment]

[1266] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1267] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1268] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1269] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1270] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1271] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1272] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1273] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1274] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1275] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1276] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1277] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1278] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1279] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented in the following manner.

[1280] System Embodiments

[1281] 1. User data entry

[1282] Through the system interface, the user selects the URL of the web page or a captured image of the paper document to be edited and inputs it into the system. For example, the user can upload the URL of the web page "https: / / example.com / page1" or a captured image of the paper document.

[1283] 2. Data transmission by the terminal

[1284] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[1285] 3. OCR processing by the server

[1286] The server analyzes the received data and uses OCR (Optical Character Recognition) technology to recognize the characters within the data. In the case of a URL, it retrieves the HTML of the web page, and in the case of an image, it analyzes the characters within the image. This process is typically performed using an OCR engine such as Tesseract.

[1287] 4. Server applies modification rules

[1288] The server applies pre-defined modification rules based on the recognized strings. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[1289] 5. Context Analysis by the Server

[1290] The server analyzes the context surrounding the recognized string and applies appropriate corrections, using natural language processing techniques to adjust the context before and after the correction to ensure consistency.

[1291] 6. Server creates correction instructions

[1292] The server marks the corrections and automatically generates a correction instruction sheet that reflects the specific correction instructions. The correction instruction sheet contains details of the corrections and the specific correction content. The correction instruction sheet is usually generated in a format such as PDF or Word.

[1293] 7. Receiving correction instructions via terminal

[1294] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[1295] Specific examples

[1296] Rename a web page

[1297] User Action:

[1298] The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[1299] Terminal behavior:

[1300] The device sends the URL "https: / / example.com / page1" to the server.

[1301] Server Action:

[1302] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[1303] Terminal behavior:

[1304] The terminal receives the correction instructions and displays them to the user.

[1305] Changes in paper media conditions

[1306] User Action:

[1307] The user uploads a captured image of the paper media to the system.

[1308] Terminal behavior:

[1309] The terminal transmits the image data to the server.

[1310] Server Action:

[1311] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[1312] Terminal behavior:

[1313] The terminal receives the correction instructions and displays them to the user.

[1314] As described above, the system of the present invention prevents omissions of corrections and inconsistencies in notation, and realizes efficient creation of correction instructions.

[1315] The processing flow will be explained below.

[1316] Step 1:

[1317] A user accesses the system interface and inputs the URL of the web page to be modified or a captured image of the paper medium. For example, the user inputs the URL of the web page "https: / / example.com / page1" or uploads an image of the paper medium.

[1318] Step 2:

[1319] The device sends the entered URL or captured image to the server via an HTTP request, with the data usually sent in JSON or binary format.

[1320] Step 3:

[1321] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[1322] Step 4:

[1323] The server uses OCR technology to recognize characters from HTML pages and captured images. In the case of images, an OCR engine such as Tesseract is used to extract character data.

[1324] Step 5:

[1325] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[1326] Step 6:

[1327] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[1328] Step 7:

[1329] The server marks the areas to be corrected and automatically generates a correction instruction sheet, which clearly describes the identified areas to be corrected and the specific corrections to be made.

[1330] Step 8:

[1331] The terminal receives the generated correction instructions from the server as an HTTP response.

[1332] Step 9:

[1333] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[1334] Through the above steps, this system efficiently creates correction instructions and prevents omissions of corrections and inconsistencies in notation.

[1335] Example 1

[1336] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1337] Conventional correction work is often done manually, which can lead to problems such as missing corrections, inconsistencies in spelling, and inconsistencies in context. Furthermore, there is a lack of systems that can efficiently correct both paper and digital content. Therefore, there is a need to improve the efficiency and accuracy of correction work.

[1338] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1339] In this invention, the server includes means for recognizing characters from input data using OCR technology, means for applying specified correction rules based on the recognized character string, and means for analyzing the context using natural language processing technology, which makes it possible to prevent omissions and inconsistencies in spelling and automatically make corrections that are consistent with the context.

[1340] "User" refers to the person or end user who uses the system to input data to be corrected and checks the correction results.

[1341] "Content identification information" refers to information that uniquely identifies the digital data to be modified, such as the URL of a web page or the ID of a digital document.

[1342] "Digital data" refers to data expressed in a format that can be processed by a computer, and includes, for example, text, images, audio, and video.

[1343] "Terminal" refers to a device used by a user, such as a computer, smartphone, or tablet, that has the ability to communicate with the system and send and receive data.

[1344] "Server" refers to a computer system that receives requests from terminals, processes the input information, and generates correction instructions.

[1345] "OCR technology" is an abbreviation for Optical Character Recognition, and refers to the technology that analyzes characters in an image and converts them into text data.

[1346] The "modification rule" refers to a rule for replacing a specific character string or information with another character string or information, or a rule for adding necessary information or deleting unnecessary information.

[1347] "Natural language processing technology" refers to computer technology for understanding, analyzing, and generating human language, and includes, for example, algorithms for context analysis and semantic understanding.

[1348] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and is usually generated in a digital format such as PDF or Word.

[1349] A "prompt" is text used to input instructions or questions to a model, and refers to a document that specifically instructs an AI model on what to do.

[1350] An "AI model" refers to an algorithm or neural network that learns from data and automatically performs specific tasks.

[1351] The present invention relates to a system for efficiently performing revision work on web pages and paper media, and is implemented as follows.

[1352] The system consists of multiple components: users, terminals, and servers. Users access the system using their terminals and enter the data to be corrected. The data to be corrected can be a web page URL or a captured image of a paper document.

[1353] The device (PC, smartphone, etc.) provides a means to send the URL or captured image entered by the user to the server. Data is sent in JSON or binary format using an HTTP request.

[1354] The server performs various processes using the following hardware and software:

[1355] Hardware: A server with a powerful processor, sufficient memory and storage

[1356] software:

[1357] OCR Technology: Tesseract OCR Engine

[1358] Natural language processing technology: NLP libraries (e.g., SpaCy, NLTK)

[1359] Data processing: Programs written in Python, Java, etc.

[1360] The system processing flow is as follows:

[1361] The server receives the data sent from the terminal and first applies OCR technology to recognize characters from the input data. In this step, character recognition is performed from an image using the Tesseract OCR engine. For example, character data is extracted from a captured image of a paper document uploaded by the user.

[1362] The server then applies pre-defined modification rules based on the recognized strings, which may include replacing specific strings, removing unnecessary information, or adding necessary information, such as replacing "old service name" with "new service name."

[1363] The server then uses natural language processing technology to analyze the context surrounding the recognized string. This allows for appropriate corrections to be made that are consistent with the context. In this step, a generative AI model is used to perform the appropriate context analysis. Specifically, the generative AI model is prompted with the following instruction: "Please make appropriate corrections so that the entire paragraph flows properly."

[1364] The server then marks the corrections and automatically generates a correction instruction sheet, which includes details of the corrections and specific correction content, and is usually generated in PDF or Word format, allowing the user to receive clear correction instructions.

[1365] Finally, the terminal receives the generated correction instructions from the server via an HTTP response and provides them to the user, who can then check the correction instructions and perform the actual correction work.

[1366] Specific examples

[1367] Rename a web page

[1368] User action: The user enters "https: / / example.com / page1" into the system interface and presses the submit button.

[1369] What the device does: Sends the URL in JSON format to the server.

[1370] Server process: Obtain HTML and analyze the text with Tesseract. Replace "old service name" with "new service name" and adjust using context analysis. Generate a PDF of the correction instructions.

[1371] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[1372] Changes in paper media conditions

[1373] User Action: Upload a captcha image.

[1374] Device operation: Sends image data to the server.

[1375] Server processing: Recognize text in the image using Tesseract. Replace "old conditions" with "new conditions" and adjust using context analysis. Generate a PDF of correction instructions.

[1376] Terminal operation: Receives the HTTP response and displays correction instructions to the user.

[1377] Through the above process, this system is able to carry out correction work efficiently and with high accuracy.

[1378] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1379] Step 1:

[1380] The user enters the data to be modified. The user selects the URL of a web page or a captured image of a paper document in the system interface and clicks the submit button. For example, the user can enter the URL "https: / / example.com / page1" or upload a captured image file. This operation inputs the web page URL or image data into the system.

[1381] input:

[1382] Webpage URL

[1383] Captured image data

[1384] output:

[1385] Send button pressed event

[1386] Step 2:

[1387] The terminal sends the entered data to the server. The terminal converts the entered URL or captured image into JSON or binary format and sends it to the server via an HTTP request. The event of pressing the send button triggers the sending of JSON format data to the server.

[1388] input:

[1389] Web page URL or captured image data

[1390] Send button pressed event

[1391] output:

[1392] HTTP request (sent by server)

[1393] Step 3:

[1394] The server analyzes the data using OCR technology. If the received URL is a webpage, the server retrieves the HTML and applies OCR technology to extract the text data. If the captured image is a text file, the server uses the Tesseract engine to recognize characters from the image. This process generates the text data.

[1395] input:

[1396] HTTP request (web page URL and captured image data)

[1397] output:

[1398] Text data

[1399] Step 4:

[1400] The server applies pre-defined correction rules. Based on the recognized text data, the server replaces specific strings with other strings, deletes unnecessary information, and adds necessary information. At this stage, the server refers to correction rules stored in a specific memory or database. For example, it replaces "old service name" with "new service name."

[1401] input:

[1402] Text data

[1403] output:

[1404] Corrected text data

[1405] Step 5:

[1406] The server analyzes the context using natural language processing technology. The server uses a generative AI model to analyze the context of the corrected text data and make appropriate adjustments to maintain consistency. The prompt used is "Please make appropriate corrections so that the entire paragraph flows properly."

[1407] input:

[1408] Corrected text data

[1409] output:

[1410] Contextually aligned text data

[1411] Step 6:

[1412] The server creates a correction instruction document. The server marks all the corrections and generates a correction instruction document that summarizes the specific corrections. This instruction document is usually created in PDF or Word format. A PDF generation library is used to generate the instruction document.

[1413] input:

[1414] Contextually aligned text data

[1415] output:

[1416] Correction Instructions (PDF or Word)

[1417] Step 7:

[1418] The terminal receives the correction instructions and provides them to the user. The terminal receives the HTTP response from the server and displays the correction instructions to the user. The user uses these as a reference to perform the actual correction work.

[1419] input:

[1420] Correction Instructions (PDF or Word)

[1421] output:

[1422] Display of correction instructions

[1423] By following the above steps, the system can efficiently carry out correction work.

[1424] (Application example 1)

[1425] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1426] Corrections to work instructions and manuals in factories are often done manually, which can lead to problems such as missing corrections and inconsistencies in notation. Furthermore, the significant time and effort required for corrections reduces work efficiency. There is a need for a system that can solve these problems and streamline the process of correcting work instructions in factories.

[1427] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1428] In this invention, the server includes: a means for a user to input the URL of a web page to be corrected or a captured image of a paper medium into the system; a means for a terminal to send the input URL or image data to the server; a means for the server to recognize characters from the input data using OCR technology; a means for the server to apply specified correction rules based on the recognized character strings; a means for the server to analyze the context and generate appropriate correction instructions; a means for the server to mark the corrections and create a correction instruction; a means for the terminal to receive the created correction instruction from the server and provide it to the user as a factory work instruction; and a means for using a camera of a factory robot to take a capture image of the paper medium and input it into the system. This prevents omissions and inconsistencies in work instructions and enables efficient correction work.

[1429] A "user" is a person who uses the system to input the URL of a web page to be modified or a captured image of a paper medium.

[1430] A "terminal" is a device that sends URLs and image data entered by a user to a server and receives correction instructions from the server.

[1431] The "server" is a system that uses OCR technology to recognize characters from input data, applies specified correction rules to analyze the context, and generates correction instructions and creates a correction instruction document.

[1432] "OCR technology" is a technology that optically recognizes characters in an image and converts them into digital text information.

[1433] "Correction rules" are rules that are applied based on recognized character strings and define operations for replacing specific character strings with other character strings, or for adding or deleting necessary information.

[1434] "Context analysis" is a process in which the context surrounding the recognized character string is analyzed using natural language processing technology and appropriate corrections are made.

[1435] A "correction instruction sheet" is a document that details the corrections to be made and the specific corrections to be made, and marks the areas that require correction.

[1436] The "factory robot camera" is a camera built into a robot used in a factory, and is a device that takes captured images of paper media.

[1437] A "factory work instruction" is a document that describes work instructions within a factory, and provides workers with the latest work content based on the correction instruction.

[1438] This invention is a system for streamlining the work of correcting work instructions and manuals in factories. This system automates a series of processes, from the stage where a user inputs the URL of a web page to be corrected or a captured image of a paper medium, to the stage where correction instructions are generated and the updated work instructions are provided within the factory.

[1439] The system includes a user, a terminal, a server, and a factory robot's camera. The user enters the URL to be corrected or a captured image of the paper media into the system's interface. The input data is sent by the terminal to the server. The server uses OCR technology to recognize characters from the input data and then applies specified correction rules based on the recognized characters. The server then analyzes the context and generates appropriate correction instructions. Finally, it creates a correction instruction sheet containing the marked-up corrections and sends it to the terminal. The terminal then provides the created correction instruction sheet to the user as a factory work instruction sheet.

[1440] The server uses the following hardware and software: The Tesseract OCR engine is used for OCR processing, and PIL (Python Imaging Library) is used as the image processing library. The requests library is used to send and receive HTTP requests. A general NLP library is used for context analysis using natural language processing (NLP).

[1441] The terminal has a function to send data entered by the user to the server in JSON or binary format, receive correction instructions from the server, and provide them to the user as factory work instructions.The factory robot's camera is also used to take capture images of paper media and input the image data into the system.

[1442] For example, if a manual used in a factory contains a description of "old process A" and needs to be changed to a new "new process B," a captured image can be uploaded to the system. The system then automatically corrects "old process A" to "new process B," and a robot presents the new work instructions to the worker. Examples of input prompts for the generative AI model include the following:

[1443] "Please modify the following text. Original text: For this task, please use old process A. Old part X is also required. Modification instructions: Please modify old process A to new process B and old part X to new part Y."

[1444] As described above, the system of the present invention can improve the efficiency of work instruction correction work in factories and prevent oversight of corrections and inconsistencies in notation.

[1445] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1446] Step 1:

[1447] The user inputs the URL of the web page to be edited or a captured image of the paper document into the system. The input can be the URL of the web page (e.g., "https: / / example.com / page1") or a captured image of the paper document. The terminal receives this input data.

[1448] Step 2:

[1449] The device sends the entered URL and image data to the server. The URL and image data are sent to the server in JSON or binary format via an HTTP request. The server receives them and begins processing.

[1450] Step 3:

[1451] The server uses OCR technology to recognize characters from the input data. For example, the Tesseract OCR engine is used to extract text information from captured images of paper media. The input is image data, and the output is a string of characters.

[1452] Step 4:

[1453] The server applies the specified modification rules based on the recognized string. The modification rules include, for example, replacing "old process A" with "new process B." The input is the recognized string, and the output is the modified string.

[1454] Step 5:

[1455] The server analyzes the context and generates appropriate correction instructions. It uses natural language processing technology to analyze the context and make corrections appropriate to the context. The input is the corrected string, and the output is the appropriate correction string after context analysis.

[1456] Step 6:

[1457] The server marks the corrections and creates a correction instruction sheet. The correction instruction sheet contains details of the corrections and specific correction content, and is generated in PDF or Word format, for example. The input is the correction string after context analysis, and the output is the correction instruction sheet.

[1458] Step 7:

[1459] The terminal receives the created correction instructions from the server and provides them to the user as factory work instructions. The user refers to these to perform the actual correction work. The input is the correction instructions, and the output is the factory work instructions that the user refers to.

[1460] Step 8:

[1461] A factory robot's camera is used to capture an image of the paper medium and input it into the system. The captured image data is then subjected to OCR processing as a target for correction. The input is the image of the paper medium, and the output is the image data input into the system.

[1462] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1463] The present invention combines a system for efficiently editing web pages and paper media with an emotion engine that recognizes the user's emotions and responds accordingly. The system is implemented as follows:

[1464] System Embodiments

[1465] 1. User data entry

[1466] The user accesses the system interface and enters the URL of the web page or a captured image of the paper to be edited. For example, the user can enter the URL of the web page "https: / / example.com / page1" or upload an image of the paper.

[1467] 2. Data transmission by the terminal

[1468] The device (the user's PC or smartphone) sends the entered URL or captured image to the server via an HTTP request. The data is usually sent in JSON or binary format.

[1469] 3. Emotional engine for recognizing user emotions

[1470] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, and the emotion engine analyzes these to recognize the user's emotions. The emotion engine uses facial recognition and voice analysis technologies to identify the user's emotional state, such as joy, anger, sadness, or happiness.

[1471] 4. OCR processing by the server

[1472] The server receives the input data, retrieves the HTML of the corresponding web page in the case of a URL, or prepares the image for analysis in the case of a paper image. The server then uses OCR technology to recognize characters within the data. For example, an OCR engine such as Tesseract can be used to extract the character data.

[1473] 5. Server applies modification rules

[1474] The server applies the configured modification rules based on the recognized string. Modification rules include replacing specific strings with other strings, adding or deleting necessary information, for example, replacing "old service name" with "new service name."

[1475] 6. Context Analysis by the Server

[1476] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[1477] 7. Server creates correction instructions

[1478] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[1479] 8. Adjusting correction instructions with the emotion engine

[1480] The emotion engine adjusts the content and format of the correction instructions based on the user's emotions, for example, simplifying the instructions or providing additional support information if the user is feeling stressed.

[1481] 9. Receiving correction instructions via terminal

[1482] The terminal receives the generated correction instructions from the server as an HTTP response and provides them to the user, who can refer to them to perform the actual correction work.

[1483] Specific examples

[1484] Rename a web page

[1485] User Action:

[1486] The user enters "https: / / example.com / page1" into the system interface and presses the submit button. The video and audio recorded during the input are captured and analyzed by the emotion engine.

[1487] Terminal behavior:

[1488] The device sends the URL "https: / / example.com / page1" to the server.

[1489] Server Action:

[1490] The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[1491] Emotion Engine Processing:

[1492] The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations and support if the user is feeling anxious.

[1493] Terminal behavior:

[1494] The terminal receives the correction instructions and displays them to the user.

[1495] Changes in paper media conditions

[1496] User Action:

[1497] Users upload captured images of paper to the system, and the emotion engine analyzes the user's emotions as they are input.

[1498] Terminal behavior:

[1499] The terminal transmits the image data to the server.

[1500] Server Action:

[1501] The server uses OCR technology to recognize the text in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[1502] Emotion Engine Processing:

[1503] An emotion engine adjusts the format and content of correction instructions based on the user's emotional state, for example generating simple instructions if the user is frustrated.

[1504] Terminal behavior:

[1505] The terminal receives the correction instructions and displays them to the user.

[1506] As described above, by combining an emotion engine, it becomes possible to provide correction instructions that take into account the user's emotional state, thereby realizing a more user-friendly system.

[1507] The processing flow will be explained below.

[1508] Step 1:

[1509] A user accesses the system interface and enters the URL of the web page to be edited or a captured image of the paper. For example, the user enters the URL of the web page "https: / / example.com / page1" or uploads an image of the paper.

[1510] Step 2:

[1511] The device sends the entered URL or captured image to the server via an HTTP request, and the data is usually sent in JSON or binary format.

[1512] Step 3:

[1513] When a user inputs data, the device captures the user's facial expressions and voice through a camera and microphone, thereby obtaining the user's emotional data.

[1514] Step 4:

[1515] The device transmits the acquired emotion data to the emotion engine, which analyzes it and identifies the user's emotion.

[1516] Step 5:

[1517] The server receives the input data and, in the case of a URL, retrieves the HTML of the corresponding web page, or, in the case of a paper image, prepares to analyze the image.

[1518] Step 6:

[1519] The server uses OCR technology to recognize characters from the HTML of the web page or the captured image, for example, by using an OCR engine such as Tesseract to extract the character data.

[1520] Step 7:

[1521] The server then applies the configured correction rules to the recognized strings, using predefined rules to find the text that needs to be changed and making the necessary substitutions, additions, or deletions.

[1522] Step 8:

[1523] The server analyzes the context surrounding the recognized string and uses natural language processing techniques to determine whether the corrected text is contextually appropriate and makes further adjustments if necessary.

[1524] Step 9:

[1525] The server marks the corrections and automatically generates a correction instruction sheet, which clearly describes the identified corrections and the specific corrections. The correction instruction sheet is usually generated in a format such as PDF or Word.

[1526] Step 10:

[1527] The emotion engine adjusts the content and format of the correction instructions based on the user's emotional data. For example, if the user is feeling stressed, the instructions may be simplified or additional support information may be provided.

[1528] Step 11:

[1529] The terminal receives the generated correction instructions from the server as an HTTP response.

[1530] Step 12:

[1531] The terminal displays the received correction instructions to the user, who can then proceed with the actual correction work based on the presented correction instructions.

[1532] Through these steps, the system efficiently creates correction instructions and takes into account the user's emotional state to prevent omissions and inconsistencies in spelling. Furthermore, by responding to emotions, the system becomes more user-friendly.

[1533] Example 2

[1534] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1535] Conventional editing work on web pages and paper documents is time-consuming and inefficient. Furthermore, no system exists that takes into account the user's emotions, and it is not possible to appropriately address the stress and anxiety users feel while editing. Therefore, there is a need for a user-friendly editing system.

[1536] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1537] In this invention, the server

[1538] means for recognizing characters from the input data using optical character recognition techniques;

[1539] means for applying specified modification rules based on the recognized characters;

[1540] a means for analyzing the context of the recognized character using natural language processing technology;

[1541] and means for recognizing the user's emotional state using emotion recognition technology and adjusting the content and format of the correction instructions.

[1542] This makes it possible to carry out efficient and appropriate correction work that takes into account the user's feelings.

[1543] A "user" is an entity that operates the system and inputs digital information to be modified.

[1544] A "terminal" is a device operated by a user, such as a PC or smartphone, and is responsible for sending input data to a server.

[1545] A "server" is a central processing unit that receives data sent from a terminal and performs various processes.

[1546] Optical character recognition (OCR) is a technology that recognizes characters from image data and converts them into text data.

[1547] "Characters" are elements such as letters and symbols recognized by OCR technology.

[1548] A "correction rule" is an operation rule such as substitution, addition, or deletion that is applied to a recognized character string.

[1549] "Context" refers to the text that exists before and after a particular string of characters, and is information that helps determine its meaning and appropriateness.

[1550] "Natural language processing technology" is a set of technologies that enable computers to understand, analyze, and generate human language.

[1551] "Emotion recognition technology" is a technology for identifying and classifying a user's emotional state from facial expressions, voice, etc.

[1552] A "correction instruction sheet" is a document that marks the corrections and clearly states the corrections.

[1553] The present invention is an efficient system for modifying digital information that takes into account the user's feelings, and is implemented in the following manner.

[1554] First, the user accesses the system interface and enters the digital information to be modified (this can be a web page URL, e.g., "https: / / example.com / page1," or a captured image of a paper document). The user enters this information and presses the submit button.

[1555] Next, the device (user's PC or smartphone) sends the entered URL and image data to the server. The data is usually sent in JSON or binary format. This allows the data to be passed to the server.

[1556] The server analyzes the received URL or image data. In the case of a URL, it retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares the image for analysis. This analysis uses optical character recognition (OCR) technology such as Tesseract to recognize characters from the input data. It then applies correction rules based on the recognized characters. Correction rules can include replacing specific strings with other strings, or adding or deleting necessary information. This is done using Python scripts or regular expressions (RegEx).

[1557] The server then analyzes the surrounding context using natural language processing techniques (such as SpaCy or NLTK) to determine whether the revised text is contextually appropriate. If not, it makes further adjustments.

[1558] Additionally, the present invention incorporates emotion recognition technology. The device captures the user's facial expressions and voice through a camera and microphone, which are then analyzed by the emotion engine. Here, facial expression recognition technology (OpenCV) and voice analysis technology (Google Cloud Speech-to-Text) are used to recognize the user's emotions. The emotion engine then adjusts the content and format of the correction instructions based on the user's emotions. For example, if the device detects that the user is anxious, it can simplify the instructions or provide additional explanations.

[1559] Once all the processing is complete, the correction instructions are generated in a format such as PDF or Word and sent to the terminal. The user can refer to this and carry out the necessary corrections.

[1560] Specific examples

[1561] Rename a web page

[1562] User operation: The user enters "https: / / example.com / page1" into the system interface and presses the submit button. The video and audio input are captured and analyzed by the emotion engine.

[1563] Device behavior: The device sends the URL "https: / / example.com / page1" to the server.

[1564] Server processing: The server retrieves the HTML of the web page and analyzes the text content using OCR technology. Next, it replaces the "old service name" with the "new service name" and analyzes the context to make adjustments. Finally, it marks the corrections and generates a correction instruction.

[1565] Emotion engine processing: The emotion engine adjusts the content of the correction instructions based on the user's emotions, for example, providing detailed explanations or support if the user is feeling anxious.

[1566] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[1567] Example prompt sentence:

[1568] "Enter a webpage URL to generate correction instructions. For example, enter "https: / / example.com / page1". Please keep your facial expressions and voice natural while capturing. Emotional data will be used to refine your correction instructions."

[1569] Changes in paper media conditions

[1570] User operation: The user uploads a captured image of a paper document to the system, and the emotion engine analyzes the user's emotions as they are input.

[1571] Terminal operation: The terminal sends image data to the server.

[1572] Server processing: The server uses OCR technology to recognize the characters in the image, replaces the "old conditions" with the "new conditions," analyzes the context, makes any necessary adjustments, and generates a correction instruction document.

[1573] Emotion engine processing: The emotion engine adjusts the format and content of the correction instructions based on the user's emotional state. For example, if the user is frustrated, it will generate simple instructions.

[1574] Terminal operation: The terminal receives the correction instructions and displays them to the user.

[1575] Example prompt sentence:

[1576] "Upload a paper capture image and replace the old conditions with the new ones. Please make sure the image is clear and you remain calm while typing. Emotional data will affect the conciseness of the correction instructions."

[1577] As described above, the system of the present invention makes it possible to carry out correction work efficiently while taking into consideration the user's feelings.

[1578] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1579] Step 1:

[1580] The user accesses the system interface and enters the digital information to be modified (a web page URL or a captured image of a paper document).

[1581] Input: For web pages, a URL such as "https: / / example.com / page1". For paper documents, a capture image file.

[1582] Specific operation: The user opens a browser and enters a URL into the input form, or takes a photo of a paper document with the smartphone camera and uploads the image file.

[1583] Output: The input URL or image data.

[1584] Step 2:

[1585] The device sends the entered URL and image data to the server.

[1586] Input: URLs and captcha images entered by the user.

[1587] Specific operation: The terminal creates data in JSON or binary format and sends it to the server address using a POST request.

[1588] Output: URL or image data sent to the server.

[1589] Step 3:

[1590] The server analyzes the URL or captcha image received.

[1591] Input: URL or image data received by the server.

[1592] Specific operation: In the case of a URL, the server retrieves the HTML of the corresponding web page. In the case of a paper image, it prepares for OCR processing.

[1593] Output: HTML content of a web page or image data to be OCR processed.

[1594] Step 4:

[1595] The emotion engine captures the user's facial expressions and voice through the device and recognizes their emotional state.

[1596] Input: Captured facial and voice data of the user.

[1597] Specific operation: Use OpenCV to obtain facial expression data. Use Google Cloud Speech-to-Text to analyze audio data.

[1598] Output: The user's emotional state (e.g., happiness, anger, sadness, anxiety, irritation).

[1599] Step 5:

[1600] The server uses optical character recognition technology (OCR) to recognize characters within the image data.

[1601] Input: Image data to be OCR processed.

[1602] What it does: The server calls the Tesseract library to extract text from the image, using functions like pytesseract.image_to_string(image) .

[1603] Output: OCR recognized text data.

[1604] Step 6:

[1605] The server applies the specified modification rules based on the recognized string.

[1606] Input: Text data recognized by OCR.

[1607] Specific operation: Using Python scripts and regular expressions (RegEx), for example, replace "old service name" with "new service name". Implemented as follows: re.sub(r'\bold service name\b', 'new service name', text).

[1608] Output: Text data with correction rules applied.

[1609] Step 7:

[1610] The server uses natural language processing technology to analyze the context and make appropriate adjustments.

[1611] Input: Text data with correction rules applied.

[1612] Specific operation: Use SpaCy or NLTK to analyze the context and identify areas that need fine-tuning. Execute nlp = spacy.load('en_core_web_sm'); doc = nlp(text) etc.

[1613] Output: Contextually adjusted text data.

[1614] Step 8:

[1615] The server marks the corrections and generates a correction instruction.

[1616] Input: Contextually adjusted text data.

[1617] What it does: Generate a PDF or Word document with the revisions highlighted. For example, generate a PDF using ReportLab. Use canvas.drawString(x, y, revised_text) or similar.

[1618] Output: Generated correction instructions (PDF and Word formats).

[1619] Step 9:

[1620] An emotional engine adjusts the content and format of revision instructions based on the user's emotional state.

[1621] Input: Generated correction instructions and the user's emotional state.

[1622] Specific behavior: If the user is feeling anxious, simplify the instructions or provide additional support information. Implemented as follows: if user_emotion == 'anxiety': add_support_info(document) .

[1623] Output: Adjusted correction instructions.

[1624] Step 10:

[1625] The terminal receives the generated correction instructions from the server and displays them to the user.

[1626] Input: Correction instructions sent from the server.

[1627] Specific operation: Receive the correction instructions in the HTTP response and display them to the user in a browser or dedicated app. For example, use the download_document(response) function.

[1628] Output: Correction instructions displayed to the user.

[1629] The above is the specific processing flow of this system.

[1630] (Application example 2)

[1631] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1632] In factories, there is a demand for accurate and fast correction work on web pages and paper documents. However, conventional systems do not take user emotions into consideration, which can result in poor user-friendliness and reduced efficiency. The present invention aims to provide a technology that recognizes user emotions and responds accordingly, thereby improving the efficiency of correction work and user satisfaction.

[1633] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to input a URL or digital image of an information source to be corrected to the system; means for a terminal to transmit the input URL or image data to the server; means for the server to recognize characters from the input data using character recognition technology; means for the server to apply specified correction rules based on the recognized character string; means for the server to analyze the context and generate appropriate correction instructions; means for the server to mark the corrections and create a correction instruction; means for the terminal to receive the created correction instruction from the server; means for the system to recognize the user's emotions; and means for the server to adjust the content and format of the correction instruction based on the emotion data. This makes it possible to provide correction instructions that take the user's emotions into consideration, thereby improving the efficiency of correction work on the factory floor and user satisfaction.

[1634] "User" means the person or operator who uses the system to enter information to be corrected and receives correction instructions.

[1635] A "URL" is an address that indicates the location of a resource on the Internet, such as a web page.

[1636] A "digital image" is an image that captures a paper document or other visual information and stores it as an electronic file.

[1637] "System" refers to a set of hardware and software that performs correction work based on information entered by the user and generates and provides correction instructions.

[1638] "Terminal" means an electronic device used by a User to input and transmit information and receive correction instructions.

[1639] A "server" is a central device that receives and analyzes data sent from a terminal and generates correction instructions.

[1640] "Character recognition technology" is a technology that identifies character information in an image and converts it into text data.

[1641] "Modification rules" are a set of rules for string substitution or modification based on recognized strings.

[1642] "Context" refers to the relationships between the sentences and words surrounding the recognized string.

[1643] "Emotion" refers to the psychological state or feedback that a user shows when entering information.

[1644] "Emotional Data" refers to data collected and analyzed to express a user's emotions.

[1645] A "correction instruction sheet" is a document generated by the system that lists the corrections to be made and the specific corrections to be made.

[1646] An "emotion engine" is a software component for analyzing a user's emotions.

[1647] An embodiment of the present invention will now be described. The system starts when the user inputs the URL or digital image of the information source to be corrected. The user accesses the system interface and inputs and uploads the URL of the web page to be corrected or a captured image of paper media. The terminal is responsible for sending the input URL or image data to the server.

[1648] The server receives the transmitted data and uses the character recognition technology "pytesseract" to recognize characters from the input data. This makes it possible to extract text information from captured images of paper media. The server also applies specified correction rules based on the recognized strings. These correction rules include replacing specific strings with other strings.

[1649] Additionally, the server uses a pipeline of "transformers" as natural language processing technology to analyze the context surrounding the recognized string, determining whether the corrected text is contextually appropriate and making further adjustments if necessary.

[1650] Meanwhile, the system incorporates an "emotion engine," which uses the device's camera and microphone to recognize the user's emotions during input. This emotion engine uses the "DeepFace" library to analyze the user's facial expressions and generate emotion data, which is then sent to the server to indicate the user's emotions.

[1651] The server adjusts the content and format of the correction instructions based on the emotion data. For example, if the user is frustrated, the server can simplify the correction instructions to reduce stress. If the user is anxious, the server can add detailed explanations to provide reassurance.

[1652] Finally, the server marks the corrections and automatically generates a correction instruction sheet. The generated correction instruction sheet is sent to the terminal as an HTTP response and provided to the user. The user can refer to this correction instruction sheet and perform the actual correction work.

[1653] As a concrete example, the following prompt sentence is shown.

[1654] Example prompt sentence:

[1655] "I'd like you to fix this instruction manual. I'm very busy and frustrated."

[1656] Thus, the present invention takes into account the user's feelings and provides an efficient and user-friendly correction system.

[1657] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1658] Step 1:

[1659] The user inputs the URL or digital image of the source to be modified into the system, and the input URL or captured image is sent to the server through the terminal interface.

[1660] Step 2:

[1661] The device sends the input URL or image data to the server. The sent data is transferred to the server in JSON or binary format. The server receives the input data and starts the next process.

[1662] Step 3:

[1663] The server analyzes the received data using optical character recognition (OCR). Specifically, it uses the "pytesseract" library to extract text from digital images. If the input data is a URL, it retrieves the relevant text from the HTML of the web page.

[1664] Step 4:

[1665] The server applies modification rules based on the recognized strings. The server performs operations to replace certain strings with other strings. For example, it replaces "old service name" with "new service name".

[1666] Step 5:

[1667] The server uses natural language processing techniques to analyze the context of the string, using a pipeline of "transformers" to determine whether the revised text is contextually appropriate and make further adjustments if necessary.

[1668] Step 6:

[1669] The system's terminal uses a camera and microphone to recognize the user's emotions. The emotion engine uses the "DeepFace" library to analyze emotions from the user's facial expressions and voice. The analyzed emotion data is sent to the server.

[1670] Step 7:

[1671] The server adjusts the content and format of the correction instructions based on the emotion data, for example, if the user is frustrated, the server adjusts the correction instructions to be briefer and less stressful.

[1672] Step 8:

[1673] The server marks the corrections and automatically generates a correction instruction sheet, which is created in PDF, Word, or other formats.

[1674] Step 9:

[1675] The terminal receives the generated correction instructions from the server and displays them to the user, who can then refer to the correction instructions to carry out the actual correction work.

[1676] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1677] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1678] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1679] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1680] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1681] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1682] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1683] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1684] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1685] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1686] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1687] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1688] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1689] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1690] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1691] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1692] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1693] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1694] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1695] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1696] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1697] The following is further disclosed regarding the above embodiment.

[1698] (Claim 1)

[1699] A means for a user to input the URL of a web page to be modified or a captured image of a paper medium into the system;

[1700] A means for transmitting the input URL and image data to a server by the terminal;

[1701] a means for the server to recognize characters from input data using OCR technology;

[1702] means for the server to apply specified modification rules based on the recognized string;

[1703] means for the server to analyze the context and generate appropriate correction instructions;

[1704] A means for the server to mark the correction portion and create a correction instruction;

[1705] A means for receiving the created correction instruction from the server by the terminal;

[1706] A system including:

[1707] (Claim 2)

[1708] 10. The system of claim 1, wherein the server includes means for executing the correction rules and for search-and-replace of recognized strings.

[1709] (Claim 3)

[1710] 10. The system of claim 1, wherein the server includes means for analyzing the context of the recognized string using natural language processing techniques.

[1711] "Example 1"

[1712] (Claim 1)

[1713] a means for a user to input into the system an identification or digital data of the content to be modified;

[1714] A means for transmitting input identification information and digital data from the terminal to a server;

[1715] a means for the server to recognize characters from input data using OCR technology;

[1716] means for the server to apply specified modification rules based on the recognized string;

[1717] A means for the server to analyze the context surrounding the recognized character string using natural language processing technology;

[1718] A means for the server to mark the correction portion and create a correction instruction;

[1719] A means for receiving the created correction instruction from the server by the terminal;

[1720] A system including:

[1721] (Claim 2)

[1722] 10. The system of claim 1, further comprising means for the server to utilize prompt sentences to make contextually consistent modifications.

[1723] (Claim 3)

[1724] 10. The system of claim 1, wherein the server comprises means for performing contextual analysis of the recognized string using an AI model.

[1725] "Application Example 1"

[1726] (Claim 1)

[1727] A means for a user to input the URL of a web page to be modified or a captured image of a paper medium into the system;

[1728] A means for transmitting the input URL and image data to a server by the terminal;

[1729] a means for the server to recognize characters from input data using OCR technology;

[1730] means for the server to apply specified modification rules based on the recognized string;

[1731] means for the server to analyze the context and generate appropriate correction instructions;

[1732] A means for the server to mark the correction portion and create a correction instruction;

[1733] a means for receiving the created correction instruction from the server by the terminal and providing it to the user as a factory work instruction;

[1734] A means for taking a capture image of the paper medium using a camera of the factory robot and inputting the captured image into the system;

[1735] A system including:

[1736] (Claim 2)

[1737] 10. The system of claim 1, wherein the server includes means for executing the correction rules and for search-and-replace of recognized strings.

[1738] (Claim 3)

[1739] 10. The system of claim 1, wherein the server includes means for analyzing the context of the recognized string using natural language processing techniques.

[1740] "Example 2: Combining Emotion Engines"

[1741] (Claim 1)

[1742] A means for a user to input unbalanced information of the digital information to be corrected into the system;

[1743] A means for transmitting the input unbalanced additional information and image data from the terminal to the server;

[1744] means for the server to recognize characters from the input data using optical character recognition technology;

[1745] a means for the server to apply specified modification rules based on the recognized characters;

[1746] a means for the server to analyze the context and generate optimal correction instructions;

[1747] A means for the server to mark the correction portion and create a correction instruction;

[1748] A means for receiving the created correction instruction from the server by the terminal;

[1749] a means for recognizing the emotional state of the user using emotion recognition technology and adjusting the content and format of the correction instructions;

[1750] A system including:

[1751] (Claim 2)

[1752] 10. The system of claim 1, wherein the server includes means for executing the correction rules and searching and replacing recognized characters.

[1753] (Claim 3)

[1754] 10. The system of claim 1, wherein the server includes means for analyzing the context of the recognized characters using natural language processing techniques.

[1755] "Application example 2 when combining emotion engines"

[1756] (Claim 1)

[1757] a means for a user to input into the system a URL or digital image of a source to be modified;

[1758] A means for transmitting the input URL and image data to a server by the terminal;

[1759] a means for the server to recognize characters from input data using character recognition technology;

[1760] means for the server to apply specified modification rules based on the recognized string;

[1761] means for the server to analyze the context and generate appropriate correction instructions;

[1762] A means for the server to mark the correction portion and create a correction instruction;

[1763] A means for receiving the created correction instruction from the server by the terminal;

[1764] a means for the system to recognize the user's emotions;

[1765] a means for the server to adjust the content and format of the correction instruction based on the emotion data;

[1766] A system including:

[1767] (Claim 2)

[1768] 10. The system of claim 1, wherein the server includes means for executing the modification rules and for search-and-replace of recognized strings.

[1769] (Claim 3)

[1770] 10. The system of claim 1, wherein the server includes means for analyzing the context of the recognized string using natural language processing techniques.

[1771] (Claim 4)

[1772] 10. The system of claim 1, further comprising means for the system to analyze the user's emotions using an emotion engine and transmit the results to a server.

[1773] (Claim 5)

[1774] 10. The system of claim 1, wherein the server comprises means for adjusting content of the revision instructions based on the user's emotional data analyzed by the emotion engine. [Explanation of symbols]

[1775] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to input the URL of a web page to be modified or a captured image of a paper medium into the system; A means for transmitting the input URL and image data to a server by the terminal; a means for the server to recognize characters from input data using OCR technology; means for the server to apply specified modification rules based on the recognized string; means for the server to analyze the context and generate appropriate correction instructions; A means for the server to mark the correction portion and create a correction instruction; A means for receiving the created correction instruction from the server by the terminal; A system including:

2. 10. The system of claim 1, wherein the server includes means for executing the correction rules and for search-and-replace of recognized strings.

3. 2. The system of claim 1, wherein the server includes means for analyzing the context of the recognized string using natural language processing techniques.

Images