Information processing system

Abstract

The present invention provides an information processing system comprising: means for receiving information; means for analyzing the received information and detecting confidential information and personal information; means for automatically replacing or deleting the detected confidential information and personal information; means for transmitting the processed information to an external service; provided is a device for recording a processing process.

Description

Technical Field

[0001] The technology disclosed herein relates to an information processing system. Background Technology

[0002] Japanese Patent Application Publication No. 2022-180282 discloses a method for controlling a role-based chatbot executed by at least one processor. The method includes the following steps: receiving a user's speech; adding the user's speech to a prompt word, the prompt word containing instruction statements associated with an explanation of the chatbot's role; encoding the prompt word; and inputting the encoded prompt word into a language model to generate a chatbot response to the user's speech.

[0003] When enterprises and other organizations utilize generative artificial intelligence services, there is a risk of confidential and personal information being leaked through external networks. Furthermore, existing information processing methods largely rely on manual intervention or simple filtering, making it difficult to effectively and automatically detect and process sensitive information in text, resulting in insufficient information security and increasing data leakage and compliance risks. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides an information processing system, comprising: a device for receiving information; a device for parsing the received information and detecting confidential and personal information; a device for automatically replacing or deleting the detected confidential and personal information; a device for sending the processed information to an external service; and a device for recording the processing process. The system can automatically identify and process sensitive information in text using natural language processing technology, and replace confidential information with predefined placeholders, effectively preventing the leakage of sensitive data. Simultaneously, it automates processing steps and makes the process traceable, improving information security and operational efficiency.

[0005] "A device for receiving information" refers to a hardware or software component that is capable of receiving data or documents from a user, terminal or other system.

[0006] "A device for parsing information" refers to a software or hardware unit used to analyze the content of input information and process the text structure or semantics.

[0007] "A device for detecting confidential and personal information" refers to a software module or hardware unit that can identify sensitive data such as names, addresses, and telephone numbers contained in the information.

[0008] "Automatic replacement or deletion device" refers to a functional module used to automatically replace detected confidential and personal information with placeholders or delete it from the information.

[0009] "A device for sending data to an external service" refers to a system component used to securely transmit processed data to an external generative artificial intelligence service or other third-party service.

[0010] "A device for recording the processing process" refers to a hardware or software unit that can save data such as the various steps of information processing, content changes, and time as a log for subsequent auditing and tracking.

[0011] Natural Language Processing (NLP) technology refers to a series of algorithms and models that use computers to analyze, understand, and process human natural language.

[0012] "Placeholders" are special characters or words that are defined in advance to replace confidential or personal information in text in order to prevent the leakage of original sensitive information. Attached Figure Description

[0013] Figure 1 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the first embodiment.

[0014] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0015] Figure 3 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the second embodiment.

[0016] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0017] Figure 5 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the third embodiment.

[0018] Figure 6 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and head-mounted terminal according to the third embodiment.

[0019] Figure 7 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the fourth embodiment.

[0020] Figure 8 This is a conceptual diagram illustrating an example of the main functions of the data processing device and robot according to the fourth embodiment.

[0021] Figure 9 This represents an emotion map that maps multiple emotions.

[0022] Figure 10 This represents an emotion map that maps multiple emotions.

[0023] Figure 11 This is a sequence diagram illustrating the processing flow of the data processing system of Embodiment 1.

[0024] Figure 12 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 1.

[0025] Figure 13 This is a sequence diagram illustrating the processing flow of the data processing system in Embodiment 2.

[0026] Figure 14 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 2. Detailed Implementation

[0027] Hereinafter, an example of an implementation of the system according to the present disclosure will be described with reference to the accompanying drawings.

[0028] First, let me explain the terminology used in the following instructions.

[0029] In the following embodiments, the processor (hereinafter referred to as "processor") with reference numerals may be a single computing device or a combination of multiple computing devices. Furthermore, the processor may be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0030] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory that temporarily stores information and is used as working memory by the processor.

[0031] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices that store various programs and parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disks (e.g., hard disks), or magnetic tapes.

[0032] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface that includes a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. As an example of a communication specification applicable to the communication I / F, wireless communication specifications such as 5G (5th Generation Mobile Communication System), Wi-Fi (wireless fidelity) (registered trademark), or Bluetooth (registered trademark) can be listed.

[0033] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects to express more than three items, the same interpretation as "A and / or B" applies.

[0034] First Implementation Method Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0035] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. A server can be cited as an example of the data processing device 12.

[0036] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. In addition, the receiving device 38, output device 40, camera 42, and communication I / F 44 are also connected to the bus 52.

[0038] The receiving device 38 includes a touchscreen 38A and a microphone 38B, and receives user input. The touchscreen 38A receives user input via touch by detecting contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input via sound by detecting the user's voice. The control unit 46A in the processor 46 sends data representing the user input received by the touchscreen 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data representing the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting data in a form perceptible to the user 20 (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0041] Figure 2 The diagram shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0043] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0044] In the smart device 14, the processor 46 performs the acceptance output processing. The memory 50 stores the acceptance output program 60. The acceptance output program 60 is used in conjunction with the data processing system 10 and the specific processing program 56. The processor 46 reads the acceptance output program 60 from the memory 50 and executes the read acceptance output program 60 on the RAM 48. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48. Furthermore, the smart device 14 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48.

[0045] Alternatively, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-held terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing of the data processing system 10 of the first embodiment will be described.

[0046] Example 1 The flow of a specific process in Example 1 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. Furthermore, the data processing device 12 is referred to as the "server," and the smart device 14 is referred to as the "terminal."

[0047] With the widespread application of generative artificial intelligence models in various data processing tasks, enterprises and organizations face a high risk of confidential and personal information leakage when utilizing external AI services. This is particularly true when processing reports, documents, and other textual information in batches; traditional data protection methods struggle to balance security with the processing efficiency of generative AI models, leading to the easy leakage of sensitive information during transmission and processing, hindering efficient and compliant data utilization. Therefore, there is an urgent need for an information processing system that can automatically detect, de-identify, and securely utilize generative AI models while ensuring information security and privacy.

[0048] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 1 is achieved by the following means.

[0049] In this invention, the server includes a device for receiving information data, a device for verifying the integrity of the received information data, a device for parsing the received information data and extracting confidential information and personal attribute information, a device for anonymizing the extracted confidential information and personal attribute information based on higher-level concepts by automatically replacing strings or deleting data, a device for sending the anonymized information data, the anonymized content, and a description to an external information processing model, and a device for recording the entire processing process and the data replacement content with time information. This allows for the secure and efficient processing and output of user-required information using a generative artificial intelligence model, while ensuring that sensitive data is not leaked, thereby improving the security compliance and business efficiency of data utilization.

[0050] "Information data" refers to data content in various forms that are input or generated by users, terminals or external systems for transmission, storage or processing, including text, numbers, symbols, etc.

[0051] "Integrity verification" refers to the process of using specific algorithms to calculate received information data in order to determine whether the data has been tampered with or lost during transmission or storage.

[0052] "Analysis" refers to the process of analyzing received information data, identifying and classifying its content, and extracting specific types of information from it.

[0053] "Confidential information" refers to sensitive information that requires strict protection and is not allowed to be disclosed in specific application scenarios, such as business-sensitive content and internal company information.

[0054] "Personal attribute information" refers to data content that can identify an individual, either alone or in combination with other information, such as name, telephone number, address, and ID number.

[0055] "Superordinate concept" refers to a broad term that is abstracted and summarized from specific information content or data category, such as "name" or "telephone number" as abstract representations of detailed personal information.

[0056] "Anonymization" refers to the process of making data information unidentifiable as a specific individual or sensitive entity through operations such as replacement, masking, or deletion.

[0057] "External information processing model" refers to an information technology platform located outside this system that has the ability to generate, analyze, or process data, including generative artificial intelligence models, etc.

[0058] "Identifier" refers to a uniform format, abstract symbol, or representative character used in data anonymization to replace the original sensitive information.

[0059] "Records" refers to the systematic saving of operation steps, processing procedures, data changes, and time information that occur in the system.

[0060] To facilitate understanding of the present invention, the specific forms in which the present invention can be implemented are described in detail below based on the content of the claims.

[0061] The information processing system of this invention consists of a server, a terminal, and an external information processing model. The server can be a physical server or a virtual cloud host, running a general-purpose operating system (such as Linux) and web server software such as Nginx or Apache. The database can be MySQL or other relational databases. The software environment for data processing and communication can use the Python language and its ecosystem libraries (such as Flask as the server-side API framework, the requests library for HTTP communication, and hashlib for verification and calculation). The terminal can be a smart mobile device, a personal computer, etc., and must have an operating system installed (such as Windows, Linux, or Android) and be able to run Python and related natural language processing libraries, such as spaCy and NLTK.

[0062] Users upload document data to the server through a terminal interface. The terminal interface can be a web frontend, desktop application, or mobile application, and includes functions such as file upload and text input. Upon receiving the data, the server first performs integrity verification, using algorithms such as hashlib to calculate SHA-256 checksums to ensure security and consistency during data transmission.

[0063] After receiving instructions from the server, the terminal uses natural language processing tools (such as spaCy and NLTK) to perform content analysis and entity recognition on the information data. It automatically detects confidential information and personal attribute information, such as names, phone numbers, and ID card numbers, through keyword lists and regular expressions. A pre-built pattern library in the system can include identifiers for various higher-level concepts, such as "name," "number," and "address." Detected sensitive information is automatically replaced with corresponding identifiers or deleted according to rules, achieving anonymization.

[0064] After anonymization, the terminal returns the de-identified data to the server. The server re-verifies the data security and automatically generates a prompt for subsequent external information processing models (i.e., generative artificial intelligence models, such as large language model APIs). For example, the server might generate a prompt like: "Please correct the following document; names and phone numbers in the text have been anonymized." Based on a security protocol (such as HTTPS), the server calls the external AI model API, submitting the prompt and the de-identified document together to obtain the external processing results.

[0065] During this process, the server will also record in detail each processing step, the content of sensitive information replacement, and the operation time, and save them in the server-side database or log file for traceability, compliance auditing, and system maintenance.

[0066] For example, a user needs to use a generative AI model to correct a report containing employee names and contact information. The user uploads the report through their terminal, and the system, after being detected by spaCy, automatically replaces "Wang Xiaoming, Phone: 13912345678" with "Name, Phone Number" and completes the anonymization. The server then generates the following prompt statement and submits it along with the anonymized text to the external AI model: "Please correct the following document; names and phone numbers have been anonymized." Please translate the following report. Note that all names and contact information have been replaced with placeholders. Therefore, the system of the present invention can ensure that sensitive information of an enterprise or organization is fully protected when using external services such as generative artificial intelligence models, while improving the convenience of artificial intelligence model application and data compliance.

[0067] use Figure 11 The processing flow is explained.

[0068] Step 1: Users input or upload data documents containing confidential and personal information through the interface on the terminal. Input can be text input, file upload, etc. The terminal temporarily stores the received raw data in local storage or memory. The input is the data content provided by the user, and the output is the raw data temporarily stored locally on the terminal.

[0069] Step 2: The terminal sends the raw data packet to the server via HTTPS encrypted communication. The terminal appends user identification information and metadata such as upload time to the data packet. The input is the raw data temporarily stored locally, and the output is the uploaded data packet containing metadata.

[0070] Step 3: The server receives data uploaded by the terminal and uses tools such as hashlib to perform integrity checks on the data content, calculating and comparing checksums to prevent data tampering during transmission. The input is the data packet uploaded by the terminal, and the output is the integrity check result and the original data to be parsed.

[0071] Step 4: After the server verifies the data, it sends instructions to the terminal for data parsing and anonymization, specifying the sensitive data types to be detected and processed. The input is the original data after integrity verification, and the output is the processing instructions for the specific data.

[0072] Step 5: The terminal responds to server commands, using natural language processing tools such as spaCy and NLTK to parse the raw data. It automatically detects confidential information and personal attributes (such as names and phone numbers) in the text and replaces or deletes them with preset identifiers according to rules, achieving anonymization. The input is server commands and raw data; the output is anonymized data and a report on sensitive information processing.

[0073] Step 6: The terminal replaces the report with anonymized data and detailed sensitive information, and returns it to the server via an HTTPS secure channel. The input is the de-identified data and the processing report; the output is the anonymized data packet uploaded to the server.

[0074] Step 7: After receiving the anonymized data, the server performs a secondary check on the data structure and security to confirm that no sensitive information remains. The server then automatically generates a prompt suitable for external generative AI models, such as "Please correct the following document; names and phone numbers in the text have been anonymized." The inputs are the anonymized data and a sensitive information report; the outputs are the generated prompt and the final de-anonymized data.

[0075] Step 8: The server calls the API of an external generative artificial intelligence model, submitting prompts and anonymized data along with the prompts, and obtains the processing results (such as text correction, summarization, or translation) returned by the AI ​​model. The input is anonymized data and prompts, and the output is the document result processed by the AI.

[0076] Step 9: The server returns the processed document results, security prompts, operation logs, and security audit data to the terminal via an HTTPS secure channel. The input is the AI's returned results and related logs; the output is the AI-processed document and processing records displayed to the terminal.

[0077] Step 10: The terminal receives information from the server, displays the AI-generated text, and notifies the user that data processing has been anonymized. The user can choose to download, copy, edit, or save the processed document. The input is the AI-processed result returned by the server; the output is the user interface display and subsequent user actions.

[0078] Application Example 1 The process flow corresponding to the specific processing in Use Case 1 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. Furthermore, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0079] With the development of information and communication technologies, users often input personal information or confidential business data such as names, addresses, and contact information when interacting with external generative artificial intelligence models through terminal devices. Current technologies struggle to effectively prevent the leakage of this confidential information during data transmission and AI processing while ensuring data processing efficiency and intelligence levels. Furthermore, there is a lack of systematic and secure technical means to adjust automated feedback based on the user's emotional state to improve user experience when interacting with AI systems. Therefore, developing an information processing system that can automatically detect, identify, and anonymize information requiring protection, while incorporating emotion recognition capabilities to combine security, confidentiality, and intelligent response, has become a pressing technical challenge.

[0080] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is achieved by the following means.

[0081] In this invention, the server includes a device for receiving input information, a device for parsing the received input information and detecting information requiring protection, a device for automatically replacing or deleting the detected information requiring protection with a higher-level category identifier, a device for parsing and replacing the input information using natural language processing technology, a device for sending the processed input information to an external information processing service, a device for receiving the response from the external information processing service again and controlling the prompting of information to the user based on the response content, a device for recording and managing all information and processing procedures related to the processing, and a device for performing sentiment analysis on the user input content, appending the sentiment analysis results to the data sent by the external information processing service, or controlling the prompting method of the response data based on the sentiment analysis results. This allows for flexible and efficient intelligent interaction with external artificial intelligence systems while ensuring the security of confidential and personal information, preventing the leakage of sensitive information, and improving the user experience and response quality in automated information processing.

[0082] "Input information" refers to the data content submitted to the system by the user through the terminal device, including but not limited to text, voice, images and other types of information.

[0083] "Information requiring protection" refers to data elements that may reveal personal privacy, trade secrets, or other sensitive content in the input information, including names, contact information, addresses, account information, etc.

[0084] "Superior category identifier" refers to a general label or abstract representation used to replace specific information that needs to be protected, such as "name", "phone number", "address", etc., without specific personal or business details.

[0085] Natural Language Processing (NLP) technology refers to the methods and tools used by computers to analyze, understand, and process natural language text, including word segmentation, entity recognition, and sentiment analysis.

[0086] "External information processing services" refer to data processing, analysis, and generation services provided by computing resources or platforms outside this system, such as generative artificial intelligence models.

[0087] "Generative artificial intelligence models" refer to artificial intelligence algorithm systems that can automatically generate text, summaries, answers, and other content based on input content, such as large-scale language models.

[0088] "Sentiment analysis" refers to the process of analyzing the semantics or expression of input information to determine the user's emotional state (such as happiness, anger, sadness, etc.).

[0089] "Prompt statements" refer to input instructions and explanatory statements used to indicate processing objectives, limit output, or specify special rules when submitting data to external information processing services (especially generative artificial intelligence models).

[0090] "Terminal device" refers to the hardware device used by users to input, receive and display information, including smartphones, tablets, smart glasses, etc.

[0091] A "server" is a computing device in a network that undertakes functions such as receiving, processing, storing, and interacting with external services.

[0092] "Anonymization" refers to the process of replacing or deleting the original information that needs to be protected, transforming it into content that cannot be directly identified as specific to an individual or business.

[0093] "Response data" refers to the output content generated by an external information processing service based on input information and returned to the user or system.

[0094] To illustrate the invention more clearly, the following detailed description of how the invention can be implemented is provided in conjunction with specific embodiments.

[0095] The information processing system of this invention mainly includes a server and terminal devices. Terminal devices can be computing devices with information input and display functions, such as smartphones, tablets, laptops, and smart glasses. The server is a network server with data receiving, processing, storage, and external service interaction capabilities, on which a complete set of information processing and security management software can be deployed.

[0096] The terminal is used to receive user input. Users can input various text information through the terminal's input interface, such as business applications, meeting minutes, and feedback. The terminal is equipped with natural language processing software, such as spaCy and NLTK, which can perform detailed parsing of user input and identify information requiring protection, including but not limited to names, contact information, addresses, and account numbers. The terminal utilizes capabilities such as Named Entity Recognition (NER) and regular expressions to automatically detect confidential and personal information in the text and replace or anonymize it with higher-level category identifiers (such as "name," "phone number," and "address") to eliminate the risk of direct identification.

[0097] The terminal also integrates a sentiment analysis module, such as a sentiment recognition model based on PyTorch or TensorFlow, which classifies user-input text into categories such as "happy," "angry," "sad," and "neutral." This sentiment analysis result can be uploaded along with the anonymized information.

[0098] The terminal and server send and receive data via encrypted communication protocols (such as HTTPS) to ensure the security of the transmission process. After receiving information from the terminal, the server records each data processing step, event time, processing result, and sentiment analysis, and saves this information in a database or log file for system auditing and subsequent retrieval. The server can also perform a secondary review of the received anonymized data. If any residual sensitive information is found, further blocking or administrator intervention can be implemented.

[0099] The server uses established data transmission protocols (such as RESTful APIs) or message queues to pass anonymized and sentiment-tagged data to external information processing services, typically generative artificial intelligence models, such as the currently popular large language model systems. The server can also adjust the interaction methods and style of the generated content based on sentiment analysis results; for example, when a user expresses dissatisfaction, it may prioritize returning comforting or emotionally soothing response templates.

[0100] The external AI service returns the processing results to the server, which then performs further security filtering for sensitive words or content before pushing the final output to the terminal. The terminal displays the AI ​​service's response, providing users with personalized, automated responses that do not expose sensitive information, while simultaneously displaying security prompts such as "Your personal information has been protected."

[0101] The aforementioned information processing system supports both local processing capabilities on the device side and flexible expansion with cloud computing power. All data processing-related processes are logged for future review, monitoring, and system optimization.

[0102] The following are specific application scenarios and interaction examples: The user entered the following into the terminal: "Wang Xiaoming's phone number is 13512345678. He is very angry about the device's handling result!" The terminal will automatically recognize "Wang Xiaoming" as the name and "13512345678" as the phone number, and replace them with "name" and "phone number" respectively, generating the anonymized text "The phone number of the name is the phone number, he is very angry about the device's processing result!" and determine the emotion as "anger".

[0103] This information, along with sentiment tags, is then sent to the server and transmitted to an external AI model.

[0104] When the system invokes a generative artificial intelligence model, the following example of a prompt statement can be used: "Please replace all personal information in the text below with tags (such as name and phone number) and analyze the sentiment type in the text." "Please generate a gentle and comforting auto-response based on the following anonymized and sentiment-tagged content." Through the above methods, the information processing system of the present invention can effectively prevent the leakage of personal information and confidential data, and achieve more adaptive automatic intelligent interaction based on user emotions.

[0105] use Figure 12 The processing flow is explained.

[0106] Step 1: Users input information through the terminal. Users enter text content on terminal devices such as smartphones and tablets, for example, "Mr. Zhang's email address is zhangsan@email.com, and he is very dissatisfied." The input is raw text data. The terminal temporarily stores the raw data entered by the user in its local memory.

[0107] Step 2: The terminal parses the input information and identifies confidential information. It calls natural language processing libraries (such as spaCy and NLTK) to perform word segmentation and named entity recognition on the user input data, detecting protected information such as names, email addresses, and phone numbers in the text. The input is raw text, and the output is structured data marked with protected information.

[0108] Step 3: The terminal performs information anonymization. Based on the recognition results from the previous step, the terminal replaces specific information requiring protection in the text with general category labels (such as "name and email address"), or replaces unrecognized formats using regular expression matching. The input is tokenized data, and the output is the anonymized text data. For example, the output might become: "The email address of the name is an email address, and he is very dissatisfied." Step 4: The terminal performs sentiment analysis. It uses a sentiment classification model (such as a PyTorch-based text sentiment analysis network) to determine the sentiment of anonymized text, identifying the type of emotion expressed by the user's text (such as "anger," "happiness," etc.). The input is anonymized text, and the output is the corresponding sentiment tag (such as "anger").

[0109] Step 5: The terminal assembles the data and sends it to the server. The terminal packages the anonymized text with sentiment tags and sends it to the server via an encrypted communication protocol (such as HTTPS). The input is anonymized text and sentiment data; the output is an encrypted data packet.

[0110] Step 6: The server receives and reviews uploaded data. It receives data packets submitted by the terminal, verifies their legality and integrity, and can perform a secondary check on the anonymization effect to confirm that no residual information requiring protection remains. The input is an encrypted data packet, and the output is the reviewed and approved data (or sends an error feedback).

[0111] Step 7: The server records processing logs. The server writes received information and related processing data (such as processing time, processing results, and sentiment classification) to a database or log file for subsequent auditing and tracking. Input is processing details, and output is the stored log record.

[0112] Step 8: The server sends data to an external generative AI model. The server calls the generative AI model API, submitting anonymized data along with sentiment tags, and can add prompts as needed (e.g., "Please replace all sensitive information with tags and generate a response based on sentiment."). The input consists of anonymized text, sentiment tags, and prompts; the output is the response text automatically generated by the AI ​​model.

[0113] Step 9: The server receives and processes the results generated by the AI ​​model. After receiving the AI's response, the server checks for any illegal information or security risks and determines the final response style based on sentiment analysis results. The input is the AI ​​model's response, and the output is the approved and styled response.

[0114] Step 10: The server sends the final response to the terminal. The server pushes the AI-processed and approved response to the user's terminal via a secure channel. The input is the prepared response content, and the output is the response data received by the terminal.

[0115] Step 11: The terminal displays the results and prompts the user. The terminal receives feedback from the server, displays the processing results on the user interface, and provides information such as "Personal information is protected" and "This response is in a mild style." The input is the server's response, and the output is a visual display and operation prompts for the user.

[0116] Alternatively, an emotion engine for inferring user emotions can be combined. That is, the specific processing unit 290 can also use the emotion-specific model 59 to infer user emotions and perform specific processing using user emotions.

[0117] Example 2 The flow of a specific process in Example 2 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 as the "terminal."

[0118] In enterprise information processing and user interaction, accurately understanding user emotions and adjusting system responses accordingly, while securely handling confidential and personal information, is a common challenge faced by existing systems. Previous technologies have shortcomings in user emotion recognition and response tone adjustment, and pose risks of information leakage when detecting and protecting sensitive information, making it difficult to balance user experience and information security. Therefore, there is an urgent need for a system capable of dynamically recognizing user emotions, automatically processing sensitive information, and securely calling external generative artificial intelligence models to improve the intelligence and security level of information processing.

[0119] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 2 is achieved by the following means.

[0120] In this invention, the server includes a device for receiving data, a device for parsing the received data and detecting personal attribute information and sensitive attribute information, a device for automatically replacing or deleting personal attribute information and sensitive attribute information with abstract attribute elements, a device for recognizing user emotions using natural language processing methods during the parsing process, a device for adjusting data content or tone of voice based on the recognized user emotions, a device for sending the adjusted data to external generative artificial intelligence resources, and a device for recording each processing stage. This enables timely and automated protection of user input information by sensing emotions and protecting sensitive attribute information. While improving information security, it also allows for dynamic generation and optimization of response content based on user emotions, effectively enhancing user experience and system intelligence.

[0121] "Data" refers to all kinds of information in the system, including text, symbols, tags, etc., which are input by users, received by terminals, processed by servers, and generated by external artificial intelligence resources.

[0122] "Personal attribute information" refers to various types of information that can identify or infer the identity of a specific individual, including name, contact information, ID card number, etc.

[0123] "Sensitive attribute information" refers to important data that involves personal privacy or requires strict protection, such as financial information, account passwords, addresses, medical records, etc.

[0124] "Abstract attribute elements" refer to data representations that replace actual personal attribute information or sensitive attribute information with general, untraceable personal identity information, such as placeholders and general descriptions.

[0125] "Natural Language Processing methods" refer to information processing technologies used to understand, analyze, or generate natural language text, including algorithms and processes such as word segmentation, sentiment analysis, and keyword extraction.

[0126] "User sentiment" refers to the psychological state or emotional tendency expressed by users when inputting data, such as pleasure, doubt, dissatisfaction, anger, and expectation.

[0127] "Expression tone" refers to the speaking style and emotional tone reflected in the content of the system-generated response data, such as comforting, encouraging, explaining, and polite expression styles.

[0128] "External generative AI resources" refer to external intelligent systems or models that are invoked through network interfaces to understand, analyze, or generate text from input data, such as natural language generation models based on deep learning.

[0129] The "processing stage" refers to the overall process of data flow, analysis, modification, sending, and storage within the system.

[0130] The system implemented in this invention consists of a server, a terminal, and external generative artificial intelligence resources. Users input prompts through the terminal, which is equipped with a natural language processing module and a sentiment engine. Software such as HanLP, spaCy, and Baidu Sentiment Analysis API can be used to parse natural language text and identify sentiments. The terminal identifies the emotions expressed by the user in the input text (e.g., "doubt," "anger," "interest") and transmits the sentiment-tagged input data to the server.

[0131] The server can be deployed on general-purpose computing devices, such as x86-based server hardware, and the operating system can be Linux or Windows. The server-side can be developed using Python, with backend frameworks such as Flask or Django, and a sensitive information detection module (e.g., a regular expression library, a Pii detection library, or Azure Content Moderator) to perform confidentiality and personal attribute information checks on received data. If personal or sensitive attribute information is detected in user input or subsequently generated content, the server will automatically replace it with abstract attribute elements (such as "username" or "ID") or delete it.

[0132] After sensing the user's specific emotions, the server automatically adjusts the response content and tone of voice according to predefined rules or templates. For example, for data expressing "doubt" or "dissatisfaction," the server will use comforting, explanatory, or encouraging expressions; for "happiness" or "anticipation," it will match a positive and affirmative style. After completing the above processing, the server securely transmits the data via encrypted protocols such as HTTPS to an external generative artificial intelligence model interface, such as through cloud API calls like OpenAI GPT-4 or Baidu Wenxin Yiyan, to generate the final response content.

[0133] After receiving the statements generated by the external generative artificial intelligence model, the server may perform secondary checks and replacements of sensitive information on the response text if necessary to ensure no information leakage risk occurs. Once processed, the server returns the final response to the terminal via secure communication.

[0134] After receiving the response, the terminal presents it to the user in a clear and user-friendly interface, ensuring a good user experience while achieving emotionally adaptive information interaction. The entire data flow and key processing actions can be recorded in a database (such as MySQL, MongoDB, or cloud storage such as AWS S3, Azure Blob, etc.) for later traceability, auditing, and function optimization.

[0135] Specific application examples: A user entered the following message into the company's internal intelligent customer service platform: "The system keeps giving me errors lately, what should I do?" The terminal uses Baidu's sentiment analysis API to identify the user's sentiment as "confusion and dissatisfaction," and submits the analysis results along with the original text to the server. The server uses a Python regular expression module to detect sensitive words such as "account" and "name" in the content, converting "Zhang San's account login failed" to "Username's account login failed." Based on the sentiment tags, the server selects a response template that fits a comforting tone and combines it with the following prompts to pass to the external AI model: Please provide a detailed and reassuring answer to the user's question: "My system keeps giving me errors lately, what should I do?" After processing by an external generative AI model, the returned text is checked by the server for sensitive information and the result is promptly pushed to the terminal. The terminal interface displays the reply: "We apologize for the inconvenience. Please try restarting the system. If the problem persists, please contact technical support." Another example of a prompt: "What are the features of this product?" After the terminal identifies the emotion as "interest" and uploads it, the server directly generates positive and detailed instructions. Once the external AI model returns the content, the terminal clearly displays the product features to the user.

[0136] Through the implementation methods provided by this invention, users can obtain an intelligent interactive experience that is both emotionally compatible and informationally secure. The software and hardware at each stage can flexibly select appropriate general components and mainstream open-source software tools according to actual needs.

[0137] use Figure 13 The processing flow is explained.

[0138] Step 1: Users input prompts via the terminal. The input is natural language text entered by the user, such as "Why can't I log in to my account?" The output is the raw text data to be processed. The specific action involves the user directly typing the question or command into the client page or application.

[0139] Step 2: The terminal receives text data input by the user and uses a natural language processing module to parse and perform sentiment recognition on the text. The input is the user's raw text data; the data processing includes word segmentation, syntactic analysis, and sentiment classification; the output is structured text data with sentiment tags. Specifically, the terminal calls a sentiment analysis API to obtain something like "question text + sentiment result (such as confusion, anger, etc.)".

[0140] Step 3: The terminal packages the sentiment-tagged data and sends it to the server via an encrypted communication protocol. The input consists of structured text and sentiment-tagged data; through data packaging and encryption, the output is a securely transmitted data packet. Specifically, the terminal formats JSON and sends a request to the server interface via HTTPS.

[0141] Step 4: The server optimizes the response content and tone based on the user's sentiment tags and the content, using preset templates or rules. The input is text containing anonymized content and sentiment tags; the data processing involves matching an appropriate response style and embedding specific content. The output is a response message adapted to the user's sentiment. Specifically, the server selects tone templates such as "comfort," "explanation," and "affirmation" to generate initial response content.

[0142] Step 5: The server sends the adjusted response content to an external generative AI model via an interface. The input is the server-generated response information, and the data processing involves interface encapsulation and API calls. The output is a detailed response generated by the external AI model. Specifically, the server obtains the output by calling the generative model through APIs such as OpenAI GPT or other cloud services.

[0143] Step 6: After receiving responses from an external generative AI model, the server performs secondary sensitive information checks and a final review. The input is the text returned by the external AI; the data processing involves further checking and replacing sensitive information. The output is a final, safe answer suitable for display. Specifically, the server further abstracts and processes information containing "identity" and "account."

[0144] Step 7: The server securely returns the final generated response to the terminal. The input is the final generated and anonymized response data, which is then packaged and encrypted for output via a secure data channel. Specifically, the server invokes the secure channel to return the data.

[0145] Step 8: After receiving the server's response, the terminal displays the final answer on the user interface. The input is the text data returned by the server, which requires no further processing and is directly output as user-readable content. Specifically, the terminal displays the response as a pop-up window, dialog box, or list on the application interface.

[0146] Step 9: The server and terminal jointly record logs of the entire processing process. Input includes each step of the operation, data change records, and key processing results. Data processing involves summarizing and formatting the log content, with the final output being traceable and auditable log data. Specifically, this involves automatically writing to a local database or a cloud-based logging system.

[0147] Application Example 2 The process flow corresponding to the specific processing in Use Case 2 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. In addition, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0148] Existing information processing systems often fail to adequately consider users' emotional states when processing user information, potentially increasing user stress during information delivery and feedback. Furthermore, achieving efficient detection and secure processing of sensitive and personal information while simultaneously improving user experience presents a challenging technical hurdle. This is particularly complex in scenarios requiring generative AI models to generate personalized feedback based on user emotions, making automated sensitive information protection and emotionally adaptive output even more intricate.

[0149] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 2 is achieved by the following means.

[0150] In this invention, the server includes a device for receiving external data, a device for parsing and detecting specific information and personal attribute information, a device for automatically converting or deleting the detected information into a predetermined format, a device for acquiring user behavior status and characteristic information, a device for automatically generating adaptive prompts based on the acquired characteristic information and sending them to a generative information processing device, a device for sending prompts and data to the generative information processing device and acquiring response results, a device for outputting response results to the user, and a device for recording the entire processing process. This allows for the automatic generation and output of personalized feedback that aligns with the user's emotional state, while simultaneously protecting sensitive and personal information security. This effectively enhances data security and user experience, and reduces psychological stress for users during information interaction.

[0151] "A device for receiving data acquired from an external source" refers to a hardware or software module capable of acquiring and receiving data from a terminal, network, or other system.

[0152] "A device for parsing attribute information records contained in received data and detecting specific information and personal attribute information" refers to a functional component that can analyze and process received data to identify specific sensitive information and personal information contained therein.

[0153] "A device for transforming or deleting detected specific information and personal attribute information into a predetermined format" refers to a hardware or software unit that can replace detected sensitive information or personal information with a uniform placeholder or delete it directly according to pre-defined rules.

[0154] "A device for acquiring user behavior status and characteristic information" refers to a system component that can collect characteristic information such as user emotional state, facial expressions, and voice through sensing devices or other means.

[0155] "A device for automatically generating adaptive prompts based on acquired feature information and sending them to a generative information processing device" refers to a component that can automatically create prompts suitable for the current context based on user behavior characteristics and send them to an artificial intelligence-based generative information processing module.

[0156] "A device for sending automatically generated prompts and data to a generative information processing device and obtaining response results" refers to a communication and processing unit that can synchronously send prompts and data to a generative artificial intelligence model or device and receive feedback information generated by it.

[0157] "A device for outputting the obtained response results to the user" refers to an output module that can present the feedback content of the generative information processing device to the user through display, voice, or other means.

[0158] "A device for recording the various processing steps such as data acquisition, parsing, transformation, and output" refers to a device that can store and log information from all data processing stages in a system.

[0159] The present invention can be implemented through the following embodiments.

[0160] Servers and terminals are interconnected via a network. Terminals can be smart wearable devices (such as smart glasses) equipped with cameras, microphones, displays, and speakers. Servers can be computing devices with data processing and storage capabilities. Data exchange between servers and terminals is conducted wirelessly or via wired communication.

[0161] The terminal is responsible for collecting the user's emotional characteristics. Specifically, the terminal captures the user's facial expressions in real time through a camera and collects the user's voice through a microphone. Using the Python programming language, combined with software such as OpenCV and TensorFlow, the terminal processes and analyzes the collected image and audio data to identify the user's emotional state, such as "happy," "sad," or "angry." The terminal then sends the extracted emotional tags and the collected user text information to the server.

[0162] After receiving data from the terminal, the server first uses natural language processing libraries such as NLTK and SpaCy to perform semantic analysis on the text data. Based on predefined rules, the server automatically detects specific information and personal attribute information (such as phone number, email address, name, etc.) in the text. For the detected information, the server replaces or deletes it using predefined formats (such as placeholders for [phone number], [email address], etc.), thus automatically shielding and protecting sensitive information.

[0163] Subsequently, the server combines the user's sentiment tags and anonymized text content to automatically generate a prompt that reflects the user's current mood. The server then sends this prompt along with the text content to a generative AI model (such as a dialogue generation engine based on a large model). The generative AI model then generates a context-appropriate response based on the user's emotional state and specific input, enhancing the user experience.

[0164] The server returns the generated response message to the terminal. The terminal then displays the response content directly to the user in text form on the screen, or plays the content aloud using speech synthesis software (such as pyttsx3) through the speaker, allowing the user to fully perceive the feedback. Furthermore, the terminal can also decide how to notify the user based on user settings.

[0165] The entire data processing process (including data acquisition, information detection, data transformation, AI generation, terminal output, etc.) will be fully recorded by the server's database or log system, facilitating post-event review and system maintenance.

[0166] For example, when a user enters "I think this plan is unreasonable and it also leaks my phone number 123-4567-8910" on the terminal, expressing anger, the terminal analyzes the facial expression and voice to generate the emotion tag "angry". Upon receiving this, the server automatically replaces the sensitive information with "[phone number]" and generates the following prompt: "The user's current emotion is anger. Please generate a reply to the following message in a gentle and considerate tone: 'I think this plan is very unreasonable, and it also leaked my [phone number].'" The server sends the prompt and content to the generative artificial intelligence model. The AI ​​model returns a reassuring response, such as, "We take your feedback very seriously and have arranged for relevant personnel to handle it promptly. Please rest assured that your information is secure." The terminal then presents this response to the user in both text and voice format.

[0167] Through the above methods, the present invention not only achieves automatic protection and full-process recording of sensitive information, but also improves information security and user experience through emotion perception and adaptive interaction.

[0168] use Figure 14 The processing flow is explained.

[0169] Step 1: The terminal uses a camera and microphone to capture the user's facial expressions and voice data in real time. The input consists of the user's current image and audio streams. The terminal uses Python in conjunction with OpenCV and TensorFlow to perform facial landmark recognition and voice sentiment analysis on the captured data, outputting timestamped sentiment tags (such as "happy," "angry," and "sad") and raw or transcribed text information. Simultaneously, the terminal sends the captured text and sentiment tags to the server.

[0170] Step 2: The server receives text data and sentiment tags from the terminal. The input consists of user text and sentiment tags. The server uses NLTK and SpaCy to perform word segmentation, named entity recognition, and regular expression matching on the text, detecting sensitive information such as phone numbers, email addresses, and real names. Then, the server replaces or deletes the detected sensitive information using a preset format (e.g., [phone number], [email address]). The output is the anonymized text.

[0171] Step 3: The server automatically generates a prompt message tailored to the user's current emotion based on the received sentiment tags and anonymized text. The input consists of sentiment tags and anonymized text. The server creates a prompt message based on the user's emotional state, such as, "The user's current emotion is anger. Please generate a response to the following message in a gentle and considerate tone: 'I think this plan is unreasonable and it also leaks my [phone number].'" The server then passes this prompt message and anonymized text to the generative AI model. The output is the personalized response generated by the AI ​​model.

[0172] Step 4: The server sends the response from the generative artificial intelligence model back to the terminal. The input is text information generated by the AI ​​model. After receiving the content, the terminal first displays the text content to the user on the screen, and simultaneously calls speech synthesis software (such as pyttsx3) to convert the response into speech, which is then broadcast through the speaker. The output consists of visual feedback visible to the user and audible audio feedback.

[0173] Step 5: Throughout the entire process, the server continuously records information about each step of data acquisition, processing, de-identification, generation, and output. The input consists of the intermediate and final data generated at each step. The server writes detailed operations and results to a log database, and the output is a log record available for subsequent auditing and tracking.

[0174] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires sound representing user input regarding the result of the specific processing. The control unit 46A sends the sound data representing user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0175] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0176] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0177] For example, the collection unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart device 14 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0178] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart device 14.

[0179] Second Implementation Method Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0180] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. A server can be cited as an example of the data processing device 12.

[0181] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0182] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and communication I / F 44 are also connected to the bus 52.

[0183] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0184] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0185] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0186] Figure 4 This illustrates an example of the main functions of the data processing device 12 and the smart glasses 214. For example... Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0187] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0188] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0189] In the smart glasses 214, the processor 46 performs reception and output processing. The memory 50 stores the reception and output program 60. The processor 46 reads the reception and output program 60 from the memory 50 and executes the read reception and output program 60 on the RAM 48. The reception and output processing is implemented by the processor 46 operating as a control unit 46A according to the reception and output program 60 executed on the RAM 48. Furthermore, the smart glasses 214 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290.

[0190] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart glasses 214. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0191] Example 1 The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0192] Application Example 1 The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0193] Example 2 The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0194] Application Example 2 The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0195] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A outputs the result of the specific processing to the speaker 240. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0196] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0197] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0198] For example, the collection unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart glasses 214 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0199] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart glasses 214.

[0200] Third Implementation Method Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0201] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. A server can be cited as an example of the data processing device 12.

[0202] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0203] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, display 343, and communication I / F 44 are also connected to the bus 52.

[0204] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0205] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0206] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0207] Figure 6 This illustrates an example of the main functions of the data processing device 12 and the head-mounted terminal 314. For example... Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0208] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0209] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0210] In the head-mounted terminal 314, the processor 46 performs the acceptance / output processing. The memory 50 stores the acceptance / output program 60. The processor 46 reads the acceptance / output program 60 from the memory 50 and executes the read acceptance / output program 60 on the RAM 48. The acceptance / output processing is implemented by the processor 46 operating as a control unit 46A according to the acceptance / output program 60 executed on the RAM 48.

[0211] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the head-mounted terminal 314. In the following description, the data processing device 12 will be referred to as the "server" and the head-mounted terminal 314 will be referred to as the "terminal".

[0212] Example 1 The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0213] Application Example 1 The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0214] Example 2 The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0215] Application Example 2 The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0216] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A outputs the result of the specific processing to the speaker 240 and the display 343. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0217] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 includes prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0218] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0219] For example, the collection unit is implemented by the control unit 46A of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the head-mounted terminal 314 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 to analyze the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 to generate a menu using a generation AI. For example, the serving unit is implemented by the speaker 240 and display 343 of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12 to provide the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0220] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the head-mounted terminal 314.

[0221] Fourth Implementation Method Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0222] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. A server can be cited as an example of the data processing device 12.

[0223] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0224] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, controlled object 443, and communication I / F 44 are also connected to the bus 52.

[0225] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0226] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to photograph the area around robot 414 (e.g., the field of view defined by a perspective equivalent to the field of vision of an average healthy person).

[0227] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0228] The controlled object 443 includes a display device, LEDs (light-emitting diodes) for the eyes, and motors for driving the arms, hands, and feet. The posture or movement of the robot 414 is controlled by controlling the motors in the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0229] Figure 8 This illustrates an example of the main functions of the data processing device 12 and the robot 414. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0230] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0231] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0232] In robot 414, the processor 46 performs the acceptance and output processing. The memory 50 stores the acceptance and output program 60. The processor 46 reads the acceptance and output program 60 from the memory 50 and executes the read acceptance and output program 60 on RAM 48. The acceptance and output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance and output program 60 executed on RAM 48.

[0233] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the robot 414. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal".

[0234] Example 1 The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0235] Application Example 1 The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0236] Example 2 The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0237] Application Example 2 The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0238] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A outputs the result of the specific processing to the speaker 240 and the controlled object 443. The microphone 238 acquires sound input from the user representing the result of the specific processing. The control unit 46A sends the sound data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0239] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0240] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0241] For example, the collection unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the robot 414 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the robot 414 and the control object 443 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0242] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the robot 414.

[0243] Furthermore, the emotion-specific model 59, acting as an emotion engine, can determine a user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine a user's emotion based on an emotion graph that serves as a specific mapping (see [reference]). Figure 9 The emotion-specific model 59 can also determine the robot's emotion, and the specific processing unit 290 performs specific processing based on the robot's emotions.

[0244] Figure 9 This is a diagram representing an emotion map 400 that maps multiple emotions. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotion is. On the outer side of the concentric circles, emotions representing states or behaviors arising from mood are arranged. Emotions are concepts that include feelings and mental states. Emotions generated by reactions occurring in the brain are arranged roughly to the left of the concentric circles. Emotions derived from situational judgments are arranged roughly to the right of the concentric circles. Emotions generated by reactions occurring in the brain and derived from situational judgments are arranged roughly above and below the concentric circles. Furthermore, "pleasant" emotions are arranged above the concentric circles, and "unpleasant" emotions are arranged below them. Thus, in the emotion map 400, multiple emotions are mapped based on the structure that generates emotions, and emotions that are likely to occur simultaneously are mapped close to each other.

[0245] These emotions are distributed at the three o'clock position of the emotion map 400, typically fluctuating between peace and anxiety. In the right half of the emotion map 400, situational awareness dominates over internal sensation, thus resulting in an impression of calm.

[0246] The inner side of the emotion map 400 represents the inner state, while the outer side represents behavior. Therefore, the further outward you are from the emotion map 400, the more visible the emotion becomes (manifested in behavior).

[0247] Here, human emotions are based on various balances such as posture and blood sugar levels. When these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotions in robots, cars, motorcycles, etc., can also be created in the following way: based on various balances such as posture and remaining battery power, when these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a Brain Physiological Signal Analysis System for Voice Emotion Recognition and Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the sensory-dominated region, called "response," are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the situational cognition-dominated region, called "situation," are arranged.

[0248] In the emotion map, two types of emotions that promote learning are defined. One is a negative emotion on the situational side, in the middle or peripheral region of "repentance" or "reflection." This occurs when the robot experiences negative emotions such as "I don't want to experience this feeling again" or "I don't want to be blamed again." The other is a positive emotion on the response side, near the "desire" region. This occurs when there are positive feelings such as "wanting more" or "wanting to know more."

[0249] The emotion-specific model 59 inputs user input into a pre-trained neural network to obtain emotion values ​​representing each emotion shown in the emotion map 400, thereby determining the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network... Figure 10 As shown in the sentiment graph 900, it was trained in a way that sentiments that are configured close to each other have similar values. Figure 10 The text shows examples of emotions such as "peace of mind", "stability", and "reassurance" that have similar emotion values.

[0250] The above description focuses on the functions of the data processing device 12, but the system of this disclosure is not necessarily installed on a server. The system of this disclosure can also be installed as a general information processing system. This disclosure can also be installed, for example, as a software program running on a personal computer, an application running on a smartphone, etc. The method of this disclosure can also be provided to users in the form of SaaS (Software as a Service).

[0251] In the above embodiments, an example of a specific process being performed by a single computer 22 is given. However, the technology disclosed herein is not limited to this, and the specific process can also be distributed among multiple computers, including computer 22. For example, the data generation model 58 can be located on an external device of the data processing apparatus 12, where data is generated based on the input data.

[0252] In the above embodiments, examples of storing a specific processing program 56 in the memory 32 have been described, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0253] Alternatively, a specific processing program 56 may be pre-stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 according to the requirements of the data processing device 12.

[0254] In addition, it is not necessary to store all the specific processing program 56 in the storage device such as the server connected to the data processing device 12 via the network 54 or in the memory 32; a portion of the specific processing program 56 may be stored in advance.

[0255] As hardware resources for performing specific processes, various processors, as shown below, can be used. For example, a CPU can be listed as a processor, which functions as a general-purpose processor that performs specific processes by executing software, i.e., a program. Furthermore, processors can be listed as special-purpose circuits such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application-Specific Integrated Circuits), which are processors with circuitry specifically designed to perform specific processes. Each processor has built-in or connected memory, and each processor executes specific processes using that memory.

[0256] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resources for performing a specific process can be a single processor.

[0257] As an example of a single processor, there are two approaches: First, a processor is composed of a combination of one or more CPUs and software, which functions as a hardware resource to perform a specific process; second, as represented by a SoC (System-on-a-chip), a processor is used to implement the functionality of the entire system, which includes multiple hardware resources for performing a specific process, using a single IC (Integrated Circuit) chip. In this way, the specific process is implemented by using one or more of the aforementioned processors as hardware resources.

[0258] Furthermore, the hardware architecture of these various processors, more specifically, can utilize circuits that combine semiconductor elements and other circuit components. Moreover, the specific process described above is just one example. Therefore, without departing from the main point, unnecessary steps can certainly be deleted, new steps added, or the processing order changed.

[0259] The descriptions and illustrations above are detailed explanations of a portion of the technology disclosed herein, and are merely one example of the technology disclosed herein. For example, the above descriptions of the structure, function, effect, and results are just one example of the structure, function, effect, and results of a portion of the technology disclosed herein. Therefore, without departing from the spirit of the technology disclosed herein, unnecessary parts may be deleted, new elements added, or replacements may be made to the descriptions and illustrations above. Furthermore, to avoid confusion and facilitate understanding of a portion of the technology disclosed herein, explanations of common technical knowledge that do not require special explanation under the premise of being able to implement the technology disclosed herein have been omitted from the descriptions and illustrations above.

[0260] All documents, patent applications and technical specifications set forth in this specification are incorporated herein by reference to the same extent that each document, patent application and technical specification is specifically and individually described therein and referenced by reference.

[0261] In addition, the following notes are provided in response to the above explanation.

[0262] Example 1 (Note 1) An information processing system includes: a device for receiving information data; a device for verifying the integrity of the received information data; a device for parsing the received information data and extracting confidential information and personal attribute information; a device for anonymizing the extracted confidential information and personal attribute information based on a higher-level concept by automatically replacing or deleting strings; a device for sending the anonymized information data, the anonymized content, and a description to an external information processing model; and a device for recording the entire processing process and the data replacement content with time information.

[0263] (Note 2) According to the information processing system described in Appendix 1, during the parsing process, natural language understanding processing technology and rule-based information recognition technology are used to extract confidential information and personal attribute information.

[0264] (Note 3) According to the information processing system described in Note 1, in the anonymization of confidential information and personal attribute information, predefined identifiers or abstract symbols based on higher-level concepts are used to replace data.

[0265] Application Example 1 (Note 1) An information processing system includes: a device for receiving input information; a device for parsing the received input information and detecting information requiring protection; a device for automatically replacing the detected information requiring protection with a higher-level category identifier or deleting it; a device for parsing and replacing the input information using natural language processing technology; a device for sending the processed input information to an external information processing service; a device for receiving a response from the external information processing service again and controlling the prompting of information to the user based on the response content; a device for recording and managing all information and processing procedures related to the processing; and a device for performing sentiment analysis on the user input content, appending the sentiment analysis results to the data sent by the external information processing service, or controlling the prompting method of the response data based on the sentiment analysis results.

[0266] (Note 2) According to the information processing system described in Note 1, the external information processing service is a generative artificial intelligence model, and when data is sent to the model, all highly confidential elements are replaced with higher-level category expressions or identifiers.

[0267] (Note 3) According to the information processing system described in Appendix 1, during the process of parsing or anonymizing input information, prompt statements are automatically generated or edited as input instructions for generative artificial intelligence models.

[0268] Example 2 (Note 1) An information processing system includes: a means for receiving data; a means for parsing the received data and detecting personal attribute information and sensitive attribute information; a means for automatically replacing or deleting personal attribute information and sensitive attribute information with abstract attribute elements; a means for recognizing user emotions using natural language processing methods during the parsing process; a means for adjusting data content or tone of voice based on the recognized user emotions; a means for sending the adjusted data to external generative artificial intelligence resources; and a means for recording each processing stage.

[0269] (Note 2) According to the information processing system described in Appendix 1, when inputting a prompt statement, the terminal device uses natural language processing technology to identify the user's emotions in the input data and incorporates the identification result into the processing procedure.

[0270] (Note 3) According to the information processing system described in Note 1, predefined abstract elements are used in the process of replacing personal attribute information and sensitive attribute information.

[0271] Application Example 2 (Note 1) An information processing system includes: a device for receiving data acquired from an external source; a device for parsing attribute information records contained in the received data and detecting specific information and personal attribute information; a device for transforming the detected specific information and personal attribute information into a predetermined format or deleting it; a device for acquiring user behavior status and characteristic information; a device for automatically generating adaptive prompt statements based on the acquired characteristic information and sending them to a generative information processing device; a device for sending the automatically generated prompt statements and data to the generative information processing device and acquiring response results; a device for outputting the acquired response results to the user; and a device for recording each processing step, such as data acquisition, parsing, transformation, and output.

[0272] (Note 2) According to the information processing system described in Appendix 1, when parsing data, language information processing technology is used to detect specific information and personal attribute information.

[0273] (Note 3) According to the information processing system described in Appendix 1, when performing transformation processing on specific information and personal attribute information, a predefined format is used for expression.

Claims

1. An information processing system, characterized in that, include: A device for receiving information; Device for parsing received information and detecting confidential and personal information; A device for automatically replacing or deleting detected confidential and personal information; A device for sending processed information to external services; and A device for recording the processing procedure.

2. The information processing system according to claim 1, characterized in that, The device for detecting confidential and personal information employs natural language processing technology during information parsing.

3. The information processing system according to claim 1, characterized in that, The device for automatically replacing confidential and personal information uses predefined placeholders during the replacement process.

Images