System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a voice and a sentence input in real time; means for analyzing and matching the voice and the sentence using a generative AI model; and means for generating an alert message and transmitting the alert message to a user when a possibility of abuse of authority is detected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's work environment, power harassment (hereafter referred to as "power harassment") has become a serious problem, significantly impacting the health of a company's work environment and employee motivation. In particular, there has been an increase in cases where the perpetrator is unaware that their remarks constitute power harassment, creating a need for effective countermeasures. Conventional methods have been unable to monitor or prevent power harassment in real time and have been limited to post-event responses, making it difficult to recognize harassment early and correct behavior. To improve this situation, a system is needed that can detect potential power harassment in real time and immediately alert the perpetrator. [Means for solving the problem]

[0005] The present invention solves the above problem by providing a power harassment prevention system that includes a terminal equipped with a generative artificial intelligence model, means for accepting voice and text input in real time, means for transmitting preprocessed data to a server, which then analyzes and compares the data using the generative artificial intelligence model, and means for generating an alert message and transmitting it to the terminal to notify the user when the possibility of power harassment is detected.

[0006] First, the device accepts the user's voice and text input in real time and preprocesses it into an appropriate format. Next, the preprocessed data is sent to the server via the endpoint. A generative AI model on the server analyzes the received data and compares it with past cases of power harassment. If it determines that there is a possibility of power harassment, the server generates an alert message containing the specific reason and sends it to the device using a secure communication method. The device notifies the user of this alert, providing an opportunity for the user to review their own comments and text, enabling the prevention and early response of power harassment.

[0007] A "generative artificial intelligence model" is an artificial intelligence (AI) algorithm that can learn using large amounts of past data and perform analysis and predictions based on new data.

[0008] A "terminal" is an electronic device operated by a user, and refers to a wide range of devices such as smartphones and personal computers (PCs) that have the ability to input voice and text data and send it to a server.

[0009] "Voice and text input" refers to voice data spoken by a user through a terminal and text data input as text.

[0010] "Preprocessing" refers to a series of processes that convert audio data into text and format text data into a form that is easier to analyze.

[0011] A "server" is a central computing unit that receives data sent from terminals via a network and performs analysis and processing based on that data.

[0012] "Analysis" refers to the process of evaluating received data using generative artificial intelligence models to identify specific patterns or features.

[0013] "Matching" refers to the process by which the server compares newly received data with past cases of power harassment stored in a database to assess the possibility of power harassment.

[0014] An "alert message" is a notification containing the content of a warning that is generated when it is determined that there is a possibility of power harassment.

[0015] "User" refers to an individual who operates a terminal and is a user of the system. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that uses a generative artificial intelligence model to detect statements or text that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[0038] Voice and text input and preprocessing

[0039] The device (smartphone or personal computer) accepts speech during meetings, everyday communication (voice input), and text input (sentence input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and the sentence data is preprocessed using a grammar analysis tool to make it easier to analyze.

[0040] Data transmission and analysis

[0041] The preprocessed data is sent to a server via the internet. The server is equipped with a generative artificial intelligence model that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[0042] Matching and alert message generation

[0043] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[0044] Alerting and Notifications

[0045] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0046] User response

[0047] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[0048] Specific examples

[0049] Example 1: Audio alert during a meeting

[0050] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[0051] 2. The preprocessed text data is sent to the server.

[0052] 3. The server analyzes the data and detects the statement, "You're slow at work."

[0053] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment."

[0054] 5. An alert message is sent to the terminal to notify the user.

[0055] 6. The user checks the alert and reviews their comments.

[0056] Example 2: Email alert

[0057] 1. The device retrieves the text of the email being composed in real time.

[0058] 2. The preprocessed text data is sent to the server.

[0059] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[0060] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment."

[0061] 5. An alert message is sent to the terminal to notify the user.

[0062] 6. The user checks the alert and corrects the email content.

[0063] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] The terminal accepts voice and text input from the user.

[0067] In the case of voice input, speeches made during the meeting are recorded using a microphone.

[0068] When entering text, text data is obtained through a keyboard or touch screen.

[0069] The input data is temporarily stored in a buffer.

[0070] Step 2:

[0071] Preprocess the data acquired by the device.

[0072] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[0073] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[0074] The converted text data is formatted into a format that is easy to parse.

[0075] Step 3:

[0076] The terminal transmits the preprocessed text data to the server.

[0077] To ensure secure data transfer through the endpoints, encryption is provided using the SSL / TLS protocol.

[0078] Executes the appropriate API calls to send the data to the server.

[0079] Step 4:

[0080] The server parses the received data.

[0081] It receives input data and passes it to its own data analysis engine.

[0082] It uses generative artificial intelligence models to analyze and identify words and phrases within the text.

[0083] Past cases of power harassment are referenced from the database and cross-checked.

[0084] Step 5:

[0085] The server evaluates the possibility of power harassment.

[0086] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[0087] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[0088] Step 6:

[0089] The server generates an alert message.

[0090] If possible power harassment is detected, an alert message is generated.

[0091] It identifies specific problems in the text and the reasons for them. It generates content such as, "This statement is overbearing and may cause stress to the recipient, so it constitutes power harassment."

[0092] Step 7:

[0093] The server generates an alert message and sends it to the terminal.

[0094] An alert message is sent to the terminal using secure communications means.

[0095] Send data to the device's receiving API.

[0096] Step 8:

[0097] The device displays an alert notification to the user.

[0098] Received alert messages are displayed as push notifications or pop-up notifications.

[0099] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[0100] Step 9:

[0101] The user checks the alert message and takes action.

[0102] Check the alert message and review your comments and writings.

[0103] For example, take action such as revising the content of the email or improving the content of the statement.

[0104] This allows the system to detect potential power harassment in real time, providing users with the opportunity to respond quickly.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Power harassment is a serious problem in modern workplaces, often having a negative impact on employees' mental health and the work environment. Therefore, it is important to quickly identify potential power harassment in statements or texts and respond appropriately. However, current approaches make it difficult to monitor and notify in real time, and it is also difficult to identify specific problems. Therefore, there is a need for a system that can detect power harassment in real time and promptly notify users.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting the accepted voice into text data using voice recognition technology, means for preprocessing the accepted text data using a grammar analysis tool, means for transmitting the preprocessed data to the server, means for the server to analyze the received data using a generative artificial intelligence model and evaluate the possibility of power harassment, means for generating an alert message if the possibility of power harassment is detected, and means for transmitting the generated alert message to the terminal to notify the user. This makes it possible to detect the possibility of power harassment in real time and quickly notify the user of an alert message explaining the specific problem.

[0110] "Terminal" refers to any device that accepts voice and text input in real time.

[0111] "Speech recognition technology" refers to the technology that converts voice into text data.

[0112] A "grammar analysis tool" refers to software that preprocesses text into a form that is easier to analyze.

[0113] "Preprocessing" refers to the process of formatting the original data into a form that is easier to analyze.

[0114] "Server" refers to a computer system that performs processes such as data analysis and alert message generation.

[0115] A "generative artificial intelligence model" refers to an artificial intelligence model that learns from large amounts of data and analyzes input data.

[0116] "Analysis" refers to the process of evaluating input data and understanding and classifying its contents.

[0117] "Power harassment" refers to the act of using one's power or position to inflict psychological or physical pain on others.

[0118] "Alert Message" means a notification message sent to warn of a particular event or problem.

[0119] "Notification" refers to the act of informing a user of specific information.

[0120] "Real-time" refers to the process from receiving data to providing results occurring instantly.

[0121] The present invention is a system that uses a generative artificial intelligence model to detect statements or sentences that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[0122] Voice and text input and preprocessing

[0123] The terminals are devices such as smartphones and personal computers. They accept voice input during meetings and everyday communication, as well as text input for emails and chats, in real time. Voice data is converted into text data using speech recognition technology within the terminal (for example, Google Recognition API). Text data is preprocessed into a form that is easy to analyze using a grammar analysis tool (for example, SpaCy or NLTK).

[0124] Data transmission and analysis

[0125] The preprocessed data is sent over the internet to a server, which is equipped with a generative artificial intelligence model (e.g., GPT-4) that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[0126] Matching and alert message generation

[0127] If the received data is judged to be a possible example of power harassment, the server will generate an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[0128] Alerting and Notifications

[0129] The generated alert message is sent back from the server to the device. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0130] User response

[0131] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[0132] Specific examples

[0133] Example 1: Audio alert during a meeting

[0134] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[0135] 2. The preprocessed text data is sent to the server.

[0136] 3. The server analyzes the data and detects the statement, "You're slow at work."

[0137] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is non-constructive and constitutes power harassment."

[0138] 5. An alert message is sent to the terminal to notify the user.

[0139] 6. The user checks the alert and reviews their comments.

[0140] Example 2: Email alert

[0141] 1. The device retrieves the text of the email being composed in real time.

[0142] 2. The preprocessed text data is sent to the server.

[0143] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[0144] 4. The server determines that this sentence is power harassment and generates an alert message such as, "This sentence has been recognized as a personal attack and constitutes power harassment."

[0145] 5. An alert message is sent to the terminal to notify the user.

[0146] 6. The user checks the alert and corrects the email content.

[0147] Prompt Sentence Examples

[0148] "Evaluate whether the following statement constitutes power harassment and explain why: 'Your work is completely worthless.'"

[0149] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1:

[0152] The device accepts voice and text input in real time. Specifically, the device's microphone is used to record what is said during a meeting. Text data is also obtained via email or chat applications. The input data can be voice or text data.

[0153] Step 2:

[0154] The device converts voice data into text data using speech recognition technology. Specifically, it analyzes the recorded voice using the Google Recognition API and outputs it as text data. The input here is voice data, and the output is the converted text data.

[0155] Step 3:

[0156] The text data received by the terminal is preprocessed using a grammar analysis tool. Specifically, grammar analysis tools such as SpaCy and NLTK are used to tokenize and parse the text, which then formats it into a format suitable for analysis. The input is raw text data, and the output is text data that has undergone preprocessing.

[0157] Step 4:

[0158] The terminal sends the preprocessed text data to the server. Specifically, it uses an HTTP POST request to send data to the server in JSON format. The input is the preprocessed text data, and the output is the result sent to the server.

[0159] Step 5:

[0160] The server analyzes the received data using a generative artificial intelligence model. Specifically, it uses GPT-4 installed on the server to evaluate whether words and phrases in the data constitute power harassment. The input is preprocessed text data, and the output is the analysis results.

[0161] Step 6:

[0162] The server generates an alert message based on the analysis results. Specifically, for detected cases of possible power harassment, it creates a message that explains in detail why the case constitutes power harassment. The input is the analysis results, and the output is the alert message.

[0163] Step 7:

[0164] The server sends the generated alert message to the terminal again. Specifically, it sends the message to the terminal using an HTTP POST request. The input is the alert message, and the output is the result sent to the terminal.

[0165] Step 8:

[0166] The device notifies the user of the alert message received. Specifically, it notifies the user using a push notification, a popup display, or an email. The input is the alert message, and the output is the notification sent to the user.

[0167] Step 9:

[0168] Users receive an alert message and reevaluate their statements or writing. Specifically, if they make an overbearing remark during a meeting, they reflect on their remarks and pay more attention to what they say in the future. In the case of emails or chats, they correct the content before sending. The input is the alert message, and the output is the corrected remarks or writing.

[0169] (Application example 1)

[0170] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0171] In physical stores, there is a need to detect power harassment that may occur during communication between store staff and customers in real time and deal with it on the spot.An effective system is needed to reduce customer dissatisfaction and store staff stress and maintain a healthy working environment.

[0172] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0173] In this invention, the server includes: means for a terminal equipped with a generative artificial intelligence model to accept voice and text input in real time; means for sending preprocessed data to the server, which analyzes and compares the data using the generative artificial intelligence model; means for generating an alert message and sending it to the terminal to notify the user when possible power harassment is detected; and means for collecting voice data using a device worn by store clerks and analyzing it in real time based on the generative artificial intelligence model. This allows conversations between store clerks and customers in a physical store to be monitored in real time, making it possible to detect and address signs of power harassment early on.

[0174] A "generative artificial intelligence model" is an artificial intelligence algorithm that can learn from huge data sets and understand and generate new input data.

[0175] A "terminal" is a device that a user operates to input or output data, and includes smartphones, computers, and the like.

[0176] "Server" means a computer system that receives and processes data over a network and transmits the analysis results.

[0177] "Power harassment" refers to the act of using one's position in the workplace to inflict mental or physical pain on another.

[0178] An "alert message" is notification information that alerts the user when a specific condition is met.

[0179] The "device" refers to the hardware worn by the store clerk for collecting voice data.

[0180] The present invention is a system that monitors voice communications between store clerks and customers in real time, and quickly detects and alerts users when statements that may constitute power harassment. The specific system configuration and processing procedures for implementing the present invention are described below.

[0181] Hardware configuration

[0182] This system uses the following hardware:

[0183] Device: Refers to smart glasses or smartphones for voice input and real-time notifications.

[0184] Server: A high-performance computer system that runs generative artificial intelligence models.

[0185] Device: Smart glasses worn by store staff and a microphone for collecting audio data.

[0186] Software configuration

[0187] This system consists of the following software:

[0188] Speech Recognition Software: Software for converting speech to text, such as the Google Cloud Speech-to-Text API.

[0189] Generative artificial intelligence model: An AI model for detecting power harassment using Transformer-based models such as BERT.

[0190] Communication Protocol: SSL / TLS protocol for secure data communication.

[0191] Data processing and calculation procedures

[0192] The device records conversations between store staff and customers in real time, and the recorded audio data is converted into text data by voice recognition software within the smart glasses, which is then pre-processed and sent to the server via a secure communication protocol.

[0193] The server analyzes the received text data using a generative artificial intelligence model (such as BERT). This analysis evaluates whether the phrases and words in the data constitute power harassment. If the evaluation results indicate the possibility of power harassment, the server generates an alert message pointing out the specific problem.

[0194] The generated alert message is sent to the terminal and notified to the user (store clerk) as a push notification or a pop-up display on the smart glasses or smartphone, allowing the store clerk to recognize the problematic part of their own statement in real time and correct their response appropriately.

[0195] Specific examples

[0196] For example, if a store clerk at a brick-and-mortar convenience store says to a customer, "This is simple, hurry up, the next person is waiting," the system will analyze the remark in real time and display a warning on the smart glasses saying, "This remark is overbearing and constitutes power harassment." This will enable the store clerk to immediately review their remark and correct it to respond appropriately to the customer.

[0197] Prompt Sentence Examples

[0198] TXT

[0199] <audio_file.wav>

[0200] -store_interaction

[0201] -convert_audio_to_text

[0202] -check_for_power_harassment

[0203] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0204] Step 1:

[0205] The device records conversations between store staff and customers in real time, using the microphone built into the smart glasses to collect audio data.

[0206] Input: Voice conversation between store clerk and customer

[0207] Output: Audio data file (.wav, etc.)

[0208] Step 2:

[0209] The device converts the collected voice data into text data using the Google Cloud Speech-to-Text API, and the converted text is preprocessed.

[0210] Input: Audio data file

[0211] Output: Text data

[0212] Step 3:

[0213] The terminal transmits the preprocessed text data to the server using a secure communication protocol (SSL / TLS).

[0214] Input: Preprocessed text data

[0215] Output: Sending text data

[0216] Step 4:

[0217] The server analyzes the received text data using a generative AI model (such as BERT), which analyzes words and phrases in the input text data and performs data calculations to evaluate whether they constitute power harassment.

[0218] Input: Transmitted text data

[0219] Output: Analysis results (power harassment judgment)

[0220] Step 5:

[0221] If the server detects possible power harassment, it generates an alert message explaining why a particular statement or sentence constitutes power harassment.

[0222] Input: Analysis result (power harassment judgment)

[0223] Output: Alert message

[0224] Step 6:

[0225] The server then sends the generated alert message back to the device, again using a secure protocol.

[0226] Input: Alert message

[0227] Output: Sending an alert message

[0228] Step 7:

[0229] The device notifies the user (store clerk) of the received alert message via a pop-up display on the smart glasses display, a push notification, or other means.

[0230] Input: Alert message

[0231] Output: User notification

[0232] Step 8:

[0233] The user checks the alert message, reviews their comments, and, if necessary, corrects the content of their comments and takes appropriate action.

[0234] Input: Alert message

[0235] Output: User behavior modification

[0236] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0237] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect the possibility of power harassment in real time and quickly issue an alert to the user. A specific embodiment of this system is described below.

[0238] Voice and text input and preprocessing

[0239] The device (smartphone or personal computer) accepts comments made during meetings, everyday communication (voice input), and text input (text input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and text data is preprocessed using grammar analysis tools to make it easier to analyze. An emotion engine is also used, and user emotion data is simultaneously generated from the voice and text.

[0240] Data transmission and analysis

[0241] The preprocessed text data and generated emotion data are sent to a server via the internet. The server's generative AI model analyzes the received data. This AI model has previously studied a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[0242] Matching and alert message generation

[0243] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[0244] Alerting and Notifications

[0245] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0246] User response

[0247] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[0248] Specific examples

[0249] Example 1: Voice alerts and emotion recognition during meetings

[0250] 1. The device records what is said during the meeting, converts it into text using voice recognition, and generates emotional data using an emotion engine.

[0251] 2. The preprocessed text data and emotion data are sent to the server.

[0252] 3. The server analyzes the data and detects the statement, "You're slow at work."

[0253] 4. The server determines that this comment constitutes power harassment and, taking into account the emotional data, generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotion has been recognized as irritation."

[0254] 5. An alert message is sent to the terminal to notify the user.

[0255] 6. The user checks the alert and reviews their comments.

[0256] Example 2: Email alerts and emotion recognition

[0257] 1. The device obtains the text of the email being composed in real time and generates emotion data using the emotion engine.

[0258] 2. The preprocessed text data and emotion data are sent to the server.

[0259] 3. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes the emotional data as dissatisfaction.

[0260] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[0261] 5. An alert message is sent to the terminal to notify the user.

[0262] 6. The user checks the alert and corrects the email content.

[0263] This invention allows users to recognize problems with their own statements, writing, and emotions in real time and have the opportunity to correct them, thereby more effectively preventing power harassment and improving the working environment.

[0264] The processing flow will be explained below.

[0265] Step 1:

[0266] The terminal accepts voice and text input from the user.

[0267] In the case of voice input, speech during the meeting is recorded using a microphone, and the recorded audio is stored in the device.

[0268] In the case of text input, the text data entered by the user via a keyboard or touch screen is acquired.

[0269] The input data is temporarily stored in a buffer.

[0270] Step 2:

[0271] Preprocess the data acquired by the device.

[0272] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[0273] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[0274] The converted text data is formatted into a format that is easy to parse.

[0275] Step 3:

[0276] The device generates emotion data.

[0277] In the case of audio data, the emotion engine analyzes it to extract the tone, speed, and emotional expression of the voice.

[0278] For text data, the emotion engine analyzes the context and keywords of the text to generate an emotion classification (e.g., anger, frustration, joy, etc.).

[0279] Step 4:

[0280] The terminal transmits the preprocessed text data and emotion data to the server.

[0281] The endpoints use SSL / TLS encryption to ensure secure data transfer.

[0282] Executes the appropriate API calls to send the data to the server.

[0283] Step 5:

[0284] The server parses the received data.

[0285] It receives text data and emotion data and passes it to its own data analysis engine.

[0286] Using a generative artificial intelligence model, words and phrases within text are identified and analyzed along with sentiment data.

[0287] Past cases of power harassment are referenced from the database and cross-checked.

[0288] Step 6:

[0289] The server evaluates the possibility of power harassment.

[0290] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[0291] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[0292] Step 7:

[0293] The server generates an alert message.

[0294] If possible power harassment is detected, an alert message is generated.

[0295] It describes specific problems in the text and the reasons for them. It also includes feedback based on emotional data, such as, "This statement is overbearing and may cause stress to the recipient. Furthermore, your emotion is recognized as anger."

[0296] Step 8:

[0297] The server generates an alert message and sends it to the terminal.

[0298] An alert message is sent to the terminal using secure communications means.

[0299] Send data to the device's receiving API.

[0300] Step 9:

[0301] The device displays an alert notification to the user.

[0302] Received alert messages are displayed as push notifications or pop-up notifications.

[0303] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[0304] Step 10:

[0305] The user checks the alert message and takes action.

[0306] Check the alert message and review your statements, writing, and emotions.

[0307] For example, take action such as revising the content of the email or improving the content of the statement.

[0308] As a result of this trend, the system not only detects potential power harassment in real time and provides users with an opportunity to respond quickly, but also takes into account the user's emotions, promoting more effective communication improvements.

[0309] Example 2

[0310] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0311] In today's work environment, mental stress and reduced motivation due to power harassment have become serious problems. Real-time monitoring and specific alerts are necessary to prevent this from happening, but conventional systems have difficulty in detecting power harassment in real time and generating appropriate alerts. Therefore, there is a need to establish a system that can detect power harassment in real time, including recognizing emotions, and provide appropriate feedback.

[0312] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0313] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting voice data into text data using voice recognition technology, means for generating user emotion data using an emotion engine, means for transmitting the preprocessed text data and emotion data to the server, means for the server to analyze the preprocessed data using a generative artificial intelligence model and evaluate whether it constitutes power harassment, and means for generating an alert message and transmitting it to the terminal to notify the user if the possibility of power harassment is detected. This allows users to review their own comments and text in real time and receive feedback based on the emotion data.

[0314] A "terminal" is a device (such as a smartphone or personal computer) that allows a user to input voice or text.

[0315] "Speech recognition technology" is a technology that converts voice data into text data.

[0316] An "emotion engine" is a software technology that analyzes and classifies a user's emotions from input voice or text.

[0317] "Preprocessing" is a series of operations to prepare input data in a form that is easy to analyze.

[0318] A "generative artificial intelligence model" is an artificial intelligence system that learns from huge amounts of data and analyzes and evaluates input data.

[0319] "Power harassment" is the act of using one's position or status in the workplace to inflict mental or physical pain on others.

[0320] An "alert message" is a warning message that is generated and notified to the user when a specific condition (in this case, the possibility of power harassment) is detected.

[0321] A "server" is a central processing unit that receives and analyzes data, and generates and sends alert messages.

[0322] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect potential power harassment in real time and quickly issue an alert to the user.

[0323] Voice and text input and preprocessing

[0324] Devices (smartphones or personal computers) accept comments made during meetings and everyday communication (voice input), as well as text input (text input) for emails and chats in real time. Voice data is converted into text data using voice recognition technology (such as Google Speech-to-Text API). Text data is preprocessed into an easy-to-analyze form using a grammar analysis tool (such as SpaCy). In addition, an emotion engine (such as Amazon Comprehend) is used in combination to generate user emotion data from the voice and text.

[0325] Data transmission and analysis

[0326] The preprocessed text data and generated emotion data are sent to a server via the internet. The server runs a generative AI model (such as GPT-4) to analyze the received data. The generative AI model has learned from a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[0327] Matching and alert message generation

[0328] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[0329] Alerting and Notifications

[0330] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0331] User response

[0332] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[0333] Specific examples

[0334] Example 1: Voice alerts and emotion recognition during meetings

[0335] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[0336] 2. The device generates emotion data using an emotion engine.

[0337] 3. The preprocessed text data and emotion data are sent to the server.

[0338] 4. The server analyzes the data and detects the statement, "You're slow at work."

[0339] 5. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotions have been recognized as irritation."

[0340] 6. An alert message is sent to the terminal to notify the user.

[0341] 7. The user checks the alert and reviews their comments.

[0342] Example 2: Email alerts and emotion recognition

[0343] 1. The device retrieves the text of the email being composed in real time.

[0344] 2. The device generates emotion data using an emotion engine.

[0345] 3. The preprocessed text data and emotion data are sent to the server.

[0346] 4. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes it as dissatisfaction.

[0347] 5. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[0348] 6. An alert message is sent to the terminal to notify the user.

[0349] 7. The user checks the alert and corrects the email content.

[0350] Prompt Sentence Examples

[0351] The following prompt sentences are input to the generative AI model to analyze the data and generate an alert message:

[0352] Please rate whether the following statements constitute power harassment:

[0353] Saying: "You're slow at work"

[0354] Emotion Data: {"User Emotion": "Irritated"}

[0355] Additionally, generate an alert message that includes the reasons and feedback on why this statement is problematic.

[0356] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0357] Step 1:

[0358] The device collects user conversations and everyday communication as voice data. Specifically, it records voice using the microphone of a smartphone or personal computer. The input is voice data, which is collected in real time. The output is the voice data before conversion.

[0359] Step 2:

[0360] The device converts the collected voice data into text data using voice recognition technology. The technology used here is a voice recognition API (e.g., Google Speech-to-Text). The input is voice data, and the output is converted text data. Specifically, the voice is converted sequentially into recognizable text.

[0361] Step 3:

[0362] The device collects text data as sentence data. For example, emails and chat messages entered by users in real time are input here. The output is text data before preprocessing. The input strings are captured sequentially and prepared as data for analysis.

[0363] Step 4:

[0364] The terminal preprocesses the text data. Using a grammar analysis tool (e.g., SpaCy), it appropriately processes punctuation and special characters and converts them into a format that is easy to analyze. The input is text data before preprocessing, and the output is preprocessed text data. Specifically, the analysis tool is used to remove unnecessary elements and organize the data grammatically.

[0365] Step 5:

[0366] The device generates emotion data using an emotion engine. The technology used here is an emotion recognition engine (e.g., Amazon Comprehend). The input is preprocessed text data, and the output is emotion data. Specifically, the emotion engine analyzes emotions in the text and assigns labels such as anger or joy.

[0367] Step 6:

[0368] The terminal transmits the preprocessed text data and generated emotion data to a server via the Internet. The input is the preprocessed text data and emotion data, and the output is the transmitted data. Specifically, the terminal transmits the data to the server via an Internet connection.

[0369] Step 7:

[0370] The server analyzes the received text data and emotion data. The model used is a generative artificial intelligence model (e.g., GPT-4). The input is the transmitted data, and the output is the analysis results. Specifically, the AI ​​model detects specific words and phrases in the data and evaluates them along with the emotion data.

[0371] Step 8:

[0372] The server detects power harassment based on the analysis results. The input is the analysis results, and the output is an assessment of the possibility of power harassment. Specifically, the detection algorithm compares the analysis results with the assessment criteria to identify harmful remarks and behavior.

[0373] Step 9:

[0374] The server generates an alert message when there is a possibility of power harassment. This message includes feedback based on the specific problem and emotional data. The input is the evaluation result of power harassment, and the output is the alert message. Specifically, the alert generation algorithm creates a warning message based on the evaluation result.

[0375] Step 10:

[0376] The server sends the generated alert message to the terminal. The input is the alert message, and the output is the sent message. As a specific operation, the alert is delivered to the user's terminal via the message sending function.

[0377] Step 11:

[0378] The device notifies the user of the received alert message. Possible notification methods include push notification, pop-up display, and email. The input is the received alert message, and the output is the notification to the user. Specifically, the notification system displays the alert.

[0379] Step 12:

[0380] The user checks the alert message and reevaluates their own statements and writing. For example, if they make an overbearing statement during a meeting, they can reflect on it and be more careful about what they say in the future. The input is the alert message, and the output is the corrected statement or writing. The specific operation is for the user to review the content and make corrections.

[0381] (Application example 2)

[0382] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] Power harassment is becoming a serious problem in modern workplaces, causing deterioration in employees' mental health and the work environment. Effective measures to prevent power harassment, particularly in communication between superiors and subordinates, are needed. However, a system that can detect power harassment in real time and respond quickly has not yet been established. In addition, advanced analysis that takes into account not only voice and text data analysis but also emotional changes is required. However, conventional methods are often inadequate.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0385] In this invention, the server includes a means for a terminal equipped with a generative artificial intelligence model to receive voice and text input in real time, a means for transmitting preprocessed data to the server, which then analyzes and collates the data using the generative artificial intelligence model, a means for generating an alert message and transmitting it to the terminal to notify the user when possible power harassment is detected, a means for recording conversations in real time using smart glasses and generating emotion data, and a means for sending push notifications using a smartphone, thereby enabling the detection of power harassment and rapid notification.

[0386] A "generative artificial intelligence model" is an AI technology that learns from vast amounts of data and generates and analyzes natural, human-like sentences and speech.

[0387] A "terminal" is an electronic device capable of accepting, processing, and displaying voice and text input in real time.

[0388] "Preprocessing" is the process of converting input voice data or text data into a form that is easier to analyze.

[0389] A "server" is a computer system installed to process data over a network.

[0390] "Matching" is the process of comparing the analyzed data with existing databases or standards.

[0391] An "alert message" is a warning message that notifies the user when a possible case of power harassment is detected.

[0392] "Smart glasses" are wearable devices that have built-in AR-compatible displays, cameras, and microphones, and can display information and collect data in real time.

[0393] "Real-time" means that processing and data acquisition are immediate.

[0394] "Emotional data" is data that indicates the user's emotional state, generated from analyzed speech or text.

[0395] "Push notification" is a technology that sends instant notifications from a server to a device.

[0396] The present invention is an advanced system for real-time detection and notification of workplace power harassment, utilizing a generative artificial intelligence model and an emotion engine. The system accepts and analyzes voice and text input in real time, and alerts the user if a risk of power harassment is detected. A specific embodiment of the system is described in detail below.

[0397] Hardware and software used

[0398] Hardware

[0399] Smart glasses: AR-enabled, equipped with a camera and microphone

[0400] Smartphone: iOS / Android compatible

[0401] software

[0402] Speech recognition engine: Google Cloud Speech-to-Text

[0403] Emotion engine: IBM Watson Tone Analyzer

[0404] Generative AI model: OpenAI's GPT series

[0405] Communications technology: Firebase, AWS

[0406] Push notification system: Firebase Cloud Messaging

[0407] System Operation

[0408] 1. Data Collection:

[0409] The server acquires audio and video data from the smart glasses and smartphone in real time.

[0410] This allows meetings and daily communications to be recorded.

[0411] 2. Pretreatment:

[0412] The server converts the voice data into text data using Google Cloud Speech-to-Text.

[0413] The text data is pre-processed to make it easier to interpret and analyze.

[0414] 3. Emotion data generation:

[0415] The server uses IBM Watson Tone Analyzer to generate emotion data from text data.

[0416] Emotional data indicates the user's emotional state (eg, anger, frustration, happiness).

[0417] 4. Data Analysis:

[0418] The server performs analysis using a generative artificial intelligence model (GPT series) based on the preprocessed text data and emotion data.

[0419] The server detects specific words and phrases in the input data and compares them with emotional data to assess the risk of power harassment.

[0420] 5. Alert Generation:

[0421] The server generates an alert message if a risk of power harassment is detected.

[0422] The alert message includes feedback on specific utterances or sentence problems and corresponding emotional data.

[0423] For example, "This statement is overbearing and may cause stress to the other person. Furthermore, your emotion is being perceived as anger."

[0424] 6. Notice:

[0425] The server sends the generated alert message to the smart glasses and the smartphone.

[0426] Users can view alerts via a pop-up display on their smart glasses or a push notification on their smartphone.

[0427] Specific examples

[0428] Example 1: Real-time detection during a meeting

[0429] Say: "How could you make such a basic mistake?"

[0430] Emotion data: Anger

[0431] Alert message: "This statement puts pressure on the other person and poses a risk of power harassment. We have also confirmed that your emotion is anger."

[0432] Example 2: Real-time chat monitoring

[0433] Say: "This project has been delayed entirely because of you."

[0434] Emotional data: Dissatisfied

[0435] Alert message: "This comment has been perceived as a personal attack and constitutes power harassment. Furthermore, your feelings of dissatisfaction have been confirmed."

[0436] Prompt Sentence Examples

[0437] Analyze the following statements and determine whether they pose a risk of power harassment. Also, consider the emotional data associated with the statements. Statement: "You're really useless. How many times do I have to tell you?" Emotional data: Anger

[0438] In this way, the present invention has the ability to detect the risk of power harassment in real time and promptly notify the user, thereby promoting a healthy work environment and protecting the mental health of employees.

[0439] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0440] Step 1:

[0441] The terminal collects conversation and audio data in real time through smart glasses and smartphones. The input is audio data and video data. The output is raw audio and video data collected in real time.

[0442] Step 2:

[0443] The voice data collected by the device is converted into text data using Google Cloud Speech-to-Text. The input is voice data. Data processing involves converting it into text data using voice recognition technology. The output is the converted text data.

[0444] Step 3:

[0445] The server receives the preprocessed text data from the terminal. Secure communication technologies such as AWS and Firebase are used for secure data transfer. The input is the converted text data. The output is the received text data.

[0446] Step 4:

[0447] Emotion data is generated from text data using IBM Watson Tone Analyzer. The input is the received text data. Data processing involves extracting and classifying the user's emotional state using emotion analysis technology. The output is the generated emotion data.

[0448] Step 5:

[0449] The server integrates the preprocessed text data and emotion data and performs analysis using a generative artificial intelligence model (OpenAI's GPT series). The input is the integrated text data and emotion data. Data calculation involves comparing the data with existing cases of power harassment and assessing the risk. The output is the analysis results.

[0450] Step 6:

[0451] The server evaluates the possibility of power harassment based on the analysis results and generates an alert message. The input is the analysis results. Data processing involves creating an alert message based on specific remarks and problems with the sentences, as well as feedback on emotional data. The output is the generated alert message.

[0452] Step 7:

[0453] The server uses Firebase Cloud Messaging to send the generated alert message to the devices (smart glasses and smartphones). The input is the generated alert message. The output is the sent alert message.

[0454] Step 8:

[0455] The user confirms the alert message through the device (a pop-up display on the smart glasses or a push notification on the smartphone). The input is the received alert message. The output is the user's confirmed alert message.

[0456] Step 9:

[0457] The user reevaluates their own statements and attitudes based on the alert content and makes corrections if necessary. The input is the confirmed alert message. The output is the corrected statements and attitudes.

[0458] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0459] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0460] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0461] [Second embodiment]

[0462] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0463] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0464] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0465] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0466] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0467] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0468] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0469] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0470] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0471] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0472] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0473] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0474] The present invention is a system that uses a generative artificial intelligence model to detect statements or text that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[0475] Voice and text input and preprocessing

[0476] The device (smartphone or personal computer) accepts speech during meetings, everyday communication (voice input), and text input (sentence input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and the sentence data is preprocessed using a grammar analysis tool to make it easier to analyze.

[0477] Data transmission and analysis

[0478] The preprocessed data is sent to a server via the internet. The server is equipped with a generative artificial intelligence model that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[0479] Matching and alert message generation

[0480] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[0481] Alerting and Notifications

[0482] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0483] User response

[0484] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[0485] Specific examples

[0486] Example 1: Audio alert during a meeting

[0487] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[0488] 2. The preprocessed text data is sent to the server.

[0489] 3. The server analyzes the data and detects the statement, "You're slow at work."

[0490] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment."

[0491] 5. An alert message is sent to the terminal to notify the user.

[0492] 6. The user checks the alert and reviews their comments.

[0493] Example 2: Email alert

[0494] 1. The device retrieves the text of the email being composed in real time.

[0495] 2. The preprocessed text data is sent to the server.

[0496] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[0497] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment."

[0498] 5. An alert message is sent to the terminal to notify the user.

[0499] 6. The user checks the alert and corrects the email content.

[0500] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[0501] The processing flow will be explained below.

[0502] Step 1:

[0503] The terminal accepts voice and text input from the user.

[0504] In the case of voice input, speeches made during the meeting are recorded using a microphone.

[0505] When entering text, text data is obtained through a keyboard or touch screen.

[0506] The input data is temporarily stored in a buffer.

[0507] Step 2:

[0508] Preprocess the data acquired by the device.

[0509] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[0510] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[0511] The converted text data is formatted into a format that is easy to parse.

[0512] Step 3:

[0513] The terminal transmits the preprocessed text data to the server.

[0514] To ensure secure data transfer through the endpoints, encryption is provided using the SSL / TLS protocol.

[0515] Executes the appropriate API calls to send the data to the server.

[0516] Step 4:

[0517] The server parses the received data.

[0518] It receives input data and passes it to its own data analysis engine.

[0519] It uses generative artificial intelligence models to analyze and identify words and phrases within the text.

[0520] Past cases of power harassment are referenced from the database and cross-checked.

[0521] Step 5:

[0522] The server evaluates the possibility of power harassment.

[0523] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[0524] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[0525] Step 6:

[0526] The server generates an alert message.

[0527] If possible power harassment is detected, an alert message is generated.

[0528] It identifies specific problems in the text and the reasons for them. It generates content such as, "This statement is overbearing and may cause stress to the recipient, so it constitutes power harassment."

[0529] Step 7:

[0530] The server generates an alert message and sends it to the terminal.

[0531] An alert message is sent to the terminal using secure communications means.

[0532] Send data to the device's receiving API.

[0533] Step 8:

[0534] The device displays an alert notification to the user.

[0535] Received alert messages are displayed as push notifications or pop-up notifications.

[0536] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[0537] Step 9:

[0538] The user checks the alert message and takes action.

[0539] Check the alert message and review your comments and writings.

[0540] For example, take action such as revising the content of the email or improving the content of the statement.

[0541] This allows the system to detect potential power harassment in real time, providing users with the opportunity to respond quickly.

[0542] Example 1

[0543] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0544] Power harassment is a serious problem in modern workplaces, often having a negative impact on employees' mental health and the work environment. Therefore, it is important to quickly identify potential power harassment in statements or texts and respond appropriately. However, current approaches make it difficult to monitor and notify in real time, and it is also difficult to identify specific problems. Therefore, there is a need for a system that can detect power harassment in real time and promptly notify users.

[0545] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0546] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting the accepted voice into text data using voice recognition technology, means for preprocessing the accepted text data using a grammar analysis tool, means for transmitting the preprocessed data to the server, means for the server to analyze the received data using a generative artificial intelligence model and evaluate the possibility of power harassment, means for generating an alert message if the possibility of power harassment is detected, and means for transmitting the generated alert message to the terminal to notify the user. This makes it possible to detect the possibility of power harassment in real time and quickly notify the user of an alert message explaining the specific problem.

[0547] "Terminal" refers to any device that accepts voice and text input in real time.

[0548] "Speech recognition technology" refers to the technology that converts voice into text data.

[0549] A "grammar analysis tool" refers to software that preprocesses text into a form that is easier to analyze.

[0550] "Preprocessing" refers to the process of formatting the original data into a form that is easier to analyze.

[0551] "Server" refers to a computer system that performs processes such as data analysis and alert message generation.

[0552] A "generative artificial intelligence model" refers to an artificial intelligence model that learns from large amounts of data and analyzes input data.

[0553] "Analysis" refers to the process of evaluating input data and understanding and classifying its contents.

[0554] "Power harassment" refers to the act of using one's power or position to inflict psychological or physical pain on others.

[0555] "Alert Message" means a notification message sent to warn of a particular event or problem.

[0556] "Notification" refers to the act of informing a user of specific information.

[0557] "Real-time" refers to the process from receiving data to providing results occurring instantly.

[0558] The present invention is a system that uses a generative artificial intelligence model to detect statements or sentences that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[0559] Voice and text input and preprocessing

[0560] The terminals are devices such as smartphones and personal computers. They accept voice input during meetings and everyday communication, as well as text input for emails and chats, in real time. Voice data is converted into text data using speech recognition technology within the terminal (for example, Google Recognition API). Text data is preprocessed into a form that is easy to analyze using a grammar analysis tool (for example, SpaCy or NLTK).

[0561] Data transmission and analysis

[0562] The preprocessed data is sent over the internet to a server, which is equipped with a generative artificial intelligence model (e.g., GPT-4) that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[0563] Matching and alert message generation

[0564] If the received data is judged to be a possible example of power harassment, the server will generate an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[0565] Alerting and Notifications

[0566] The generated alert message is sent back from the server to the device. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0567] User response

[0568] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[0569] Specific examples

[0570] Example 1: Audio alert during a meeting

[0571] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[0572] 2. The preprocessed text data is sent to the server.

[0573] 3. The server analyzes the data and detects the statement, "You're slow at work."

[0574] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is non-constructive and constitutes power harassment."

[0575] 5. An alert message is sent to the terminal to notify the user.

[0576] 6. The user checks the alert and reviews their comments.

[0577] Example 2: Email alert

[0578] 1. The device retrieves the text of the email being composed in real time.

[0579] 2. The preprocessed text data is sent to the server.

[0580] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[0581] 4. The server determines that this sentence is power harassment and generates an alert message such as, "This sentence has been recognized as a personal attack and constitutes power harassment."

[0582] 5. An alert message is sent to the terminal to notify the user.

[0583] 6. The user checks the alert and corrects the email content.

[0584] Prompt Sentence Examples

[0585] "Evaluate whether the following statement constitutes power harassment and explain why: 'Your work is completely worthless.'"

[0586] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[0587] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0588] Step 1:

[0589] The device accepts voice and text input in real time. Specifically, the device's microphone is used to record what is said during a meeting. Text data is also obtained via email or chat applications. The input data can be voice or text data.

[0590] Step 2:

[0591] The device converts voice data into text data using speech recognition technology. Specifically, it analyzes the recorded voice using the Google Recognition API and outputs it as text data. The input here is voice data, and the output is the converted text data.

[0592] Step 3:

[0593] The text data received by the terminal is preprocessed using a grammar analysis tool. Specifically, grammar analysis tools such as SpaCy and NLTK are used to tokenize and parse the text, which then formats it into a format suitable for analysis. The input is raw text data, and the output is text data that has undergone preprocessing.

[0594] Step 4:

[0595] The terminal sends the preprocessed text data to the server. Specifically, it uses an HTTP POST request to send data to the server in JSON format. The input is the preprocessed text data, and the output is the result sent to the server.

[0596] Step 5:

[0597] The server analyzes the received data using a generative artificial intelligence model. Specifically, it uses GPT-4 installed on the server to evaluate whether words and phrases in the data constitute power harassment. The input is preprocessed text data, and the output is the analysis results.

[0598] Step 6:

[0599] The server generates an alert message based on the analysis results. Specifically, for detected cases of possible power harassment, it creates a message that explains in detail why the case constitutes power harassment. The input is the analysis results, and the output is the alert message.

[0600] Step 7:

[0601] The server sends the generated alert message to the terminal again. Specifically, it sends the message to the terminal using an HTTP POST request. The input is the alert message, and the output is the result sent to the terminal.

[0602] Step 8:

[0603] The device notifies the user of the alert message received. Specifically, it notifies the user using a push notification, a popup display, or an email. The input is the alert message, and the output is the notification sent to the user.

[0604] Step 9:

[0605] Users receive an alert message and reevaluate their statements or writing. Specifically, if they make an overbearing remark during a meeting, they reflect on their remarks and pay more attention to what they say in the future. In the case of emails or chats, they correct the content before sending. The input is the alert message, and the output is the corrected remarks or writing.

[0606] (Application example 1)

[0607] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0608] In physical stores, there is a need to detect power harassment that may occur during communication between store staff and customers in real time and deal with it on the spot.An effective system is needed to reduce customer dissatisfaction and store staff stress and maintain a healthy working environment.

[0609] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0610] In this invention, the server includes: means for a terminal equipped with a generative artificial intelligence model to accept voice and text input in real time; means for sending preprocessed data to the server, which analyzes and compares the data using the generative artificial intelligence model; means for generating an alert message and sending it to the terminal to notify the user when possible power harassment is detected; and means for collecting voice data using a device worn by store clerks and analyzing it in real time based on the generative artificial intelligence model. This allows conversations between store clerks and customers in a physical store to be monitored in real time, making it possible to detect and address signs of power harassment early on.

[0611] A "generative artificial intelligence model" is an artificial intelligence algorithm that can learn from huge data sets and understand and generate new input data.

[0612] A "terminal" is a device that a user operates to input or output data, and includes smartphones, computers, and the like.

[0613] "Server" means a computer system that receives and processes data over a network and transmits the analysis results.

[0614] "Power harassment" refers to the act of using one's position in the workplace to inflict mental or physical pain on another.

[0615] An "alert message" is notification information that alerts the user when a specific condition is met.

[0616] The "device" refers to the hardware worn by the store clerk for collecting voice data.

[0617] The present invention is a system that monitors voice communications between store clerks and customers in real time, and quickly detects and alerts users when statements that may constitute power harassment. The specific system configuration and processing procedures for implementing the present invention are described below.

[0618] Hardware configuration

[0619] This system uses the following hardware:

[0620] Device: Refers to smart glasses or smartphones for voice input and real-time notifications.

[0621] Server: A high-performance computer system that runs generative artificial intelligence models.

[0622] Device: Smart glasses worn by store staff and a microphone for collecting audio data.

[0623] Software configuration

[0624] This system consists of the following software:

[0625] Speech Recognition Software: Software for converting speech to text, such as the Google Cloud Speech-to-Text API.

[0626] Generative artificial intelligence model: An AI model for detecting power harassment using Transformer-based models such as BERT.

[0627] Communication Protocol: SSL / TLS protocol for secure data communication.

[0628] Data processing and calculation procedures

[0629] The device records conversations between store staff and customers in real time, and the recorded audio data is converted into text data by voice recognition software within the smart glasses, which is then pre-processed and sent to the server via a secure communication protocol.

[0630] The server analyzes the received text data using a generative artificial intelligence model (such as BERT). This analysis evaluates whether the phrases and words in the data constitute power harassment. If the evaluation results indicate the possibility of power harassment, the server generates an alert message pointing out the specific problem.

[0631] The generated alert message is sent to the terminal and notified to the user (store clerk) as a push notification or a pop-up display on the smart glasses or smartphone, allowing the store clerk to recognize the problematic part of their own statement in real time and correct their response appropriately.

[0632] Specific examples

[0633] For example, if a store clerk at a brick-and-mortar convenience store says to a customer, "This is simple, hurry up, the next person is waiting," the system will analyze the remark in real time and display a warning on the smart glasses saying, "This remark is overbearing and constitutes power harassment." This will enable the store clerk to immediately review their remark and correct it to respond appropriately to the customer.

[0634] Prompt Sentence Examples

[0635] TXT

[0636] <audio_file.wav>

[0637] -store_interaction

[0638] -convert_audio_to_text

[0639] -check_for_power_harassment

[0640] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0641] Step 1:

[0642] The device records conversations between store staff and customers in real time, using the microphone built into the smart glasses to collect audio data.

[0643] Input: Voice conversation between store clerk and customer

[0644] Output: Audio data file (.wav, etc.)

[0645] Step 2:

[0646] The device converts the collected voice data into text data using the Google Cloud Speech-to-Text API, and the converted text is preprocessed.

[0647] Input: Audio data file

[0648] Output: Text data

[0649] Step 3:

[0650] The terminal transmits the preprocessed text data to the server using a secure communication protocol (SSL / TLS).

[0651] Input: Preprocessed text data

[0652] Output: Sending text data

[0653] Step 4:

[0654] The server analyzes the received text data using a generative AI model (such as BERT), which analyzes words and phrases in the input text data and performs data calculations to evaluate whether they constitute power harassment.

[0655] Input: Transmitted text data

[0656] Output: Analysis results (power harassment judgment)

[0657] Step 5:

[0658] If the server detects possible power harassment, it generates an alert message explaining why a particular statement or sentence constitutes power harassment.

[0659] Input: Analysis result (power harassment judgment)

[0660] Output: Alert message

[0661] Step 6:

[0662] The server then sends the generated alert message back to the device, again using a secure protocol.

[0663] Input: Alert message

[0664] Output: Sending an alert message

[0665] Step 7:

[0666] The device notifies the user (store clerk) of the received alert message via a pop-up display on the smart glasses display, a push notification, or other means.

[0667] Input: Alert message

[0668] Output: User notification

[0669] Step 8:

[0670] The user checks the alert message, reviews their comments, and, if necessary, corrects the content of their comments and takes appropriate action.

[0671] Input: Alert message

[0672] Output: User behavior modification

[0673] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0674] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect the possibility of power harassment in real time and quickly issue an alert to the user. A specific embodiment of this system is described below.

[0675] Voice and text input and preprocessing

[0676] The device (smartphone or personal computer) accepts comments made during meetings, everyday communication (voice input), and text input (text input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and text data is preprocessed using grammar analysis tools to make it easier to analyze. An emotion engine is also used, and user emotion data is simultaneously generated from the voice and text.

[0677] Data transmission and analysis

[0678] The preprocessed text data and generated emotion data are sent to a server via the internet. The server's generative AI model analyzes the received data. This AI model has previously studied a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[0679] Matching and alert message generation

[0680] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[0681] Alerting and Notifications

[0682] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0683] User response

[0684] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[0685] Specific examples

[0686] Example 1: Voice alerts and emotion recognition during meetings

[0687] 1. The device records what is said during the meeting, converts it into text using voice recognition, and generates emotional data using an emotion engine.

[0688] 2. The preprocessed text data and emotion data are sent to the server.

[0689] 3. The server analyzes the data and detects the statement, "You're slow at work."

[0690] 4. The server determines that this comment constitutes power harassment and, taking into account the emotional data, generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotion has been recognized as irritation."

[0691] 5. An alert message is sent to the terminal to notify the user.

[0692] 6. The user checks the alert and reviews their comments.

[0693] Example 2: Email alerts and emotion recognition

[0694] 1. The device obtains the text of the email being composed in real time and generates emotion data using the emotion engine.

[0695] 2. The preprocessed text data and emotion data are sent to the server.

[0696] 3. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes the emotional data as dissatisfaction.

[0697] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[0698] 5. An alert message is sent to the terminal to notify the user.

[0699] 6. The user checks the alert and corrects the email content.

[0700] This invention allows users to recognize problems with their own statements, writing, and emotions in real time and have the opportunity to correct them, thereby more effectively preventing power harassment and improving the working environment.

[0701] The processing flow will be explained below.

[0702] Step 1:

[0703] The terminal accepts voice and text input from the user.

[0704] In the case of voice input, speech during the meeting is recorded using a microphone, and the recorded audio is stored in the device.

[0705] In the case of text input, the text data entered by the user via a keyboard or touch screen is acquired.

[0706] The input data is temporarily stored in a buffer.

[0707] Step 2:

[0708] Preprocess the data acquired by the device.

[0709] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[0710] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[0711] The converted text data is formatted into a format that is easy to parse.

[0712] Step 3:

[0713] The device generates emotion data.

[0714] In the case of audio data, the emotion engine analyzes it to extract the tone, speed, and emotional expression of the voice.

[0715] For text data, the emotion engine analyzes the context and keywords of the text to generate an emotion classification (e.g., anger, frustration, joy, etc.).

[0716] Step 4:

[0717] The terminal transmits the preprocessed text data and emotion data to the server.

[0718] The endpoints use SSL / TLS encryption to ensure secure data transfer.

[0719] Executes the appropriate API calls to send the data to the server.

[0720] Step 5:

[0721] The server parses the received data.

[0722] It receives text data and emotion data and passes it to its own data analysis engine.

[0723] Using a generative artificial intelligence model, words and phrases within text are identified and analyzed along with sentiment data.

[0724] Past cases of power harassment are referenced from the database and cross-checked.

[0725] Step 6:

[0726] The server evaluates the possibility of power harassment.

[0727] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[0728] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[0729] Step 7:

[0730] The server generates an alert message.

[0731] If possible power harassment is detected, an alert message is generated.

[0732] It describes specific problems in the text and the reasons for them. It also includes feedback based on emotional data, such as, "This statement is overbearing and may cause stress to the recipient. Furthermore, your emotion is recognized as anger."

[0733] Step 8:

[0734] The server generates an alert message and sends it to the terminal.

[0735] An alert message is sent to the terminal using secure communications means.

[0736] Send data to the device's receiving API.

[0737] Step 9:

[0738] The device displays an alert notification to the user.

[0739] Received alert messages are displayed as push notifications or pop-up notifications.

[0740] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[0741] Step 10:

[0742] The user checks the alert message and takes action.

[0743] Check the alert message and review your statements, writing, and emotions.

[0744] For example, take action such as revising the content of the email or improving the content of the statement.

[0745] As a result of this trend, the system not only detects potential power harassment in real time and provides users with an opportunity to respond quickly, but also takes into account the user's emotions, promoting more effective communication improvements.

[0746] Example 2

[0747] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0748] In today's work environment, mental stress and reduced motivation due to power harassment have become serious problems. Real-time monitoring and specific alerts are necessary to prevent this from happening, but conventional systems have difficulty in detecting power harassment in real time and generating appropriate alerts. Therefore, there is a need to establish a system that can detect power harassment in real time, including recognizing emotions, and provide appropriate feedback.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0750] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting voice data into text data using voice recognition technology, means for generating user emotion data using an emotion engine, means for transmitting the preprocessed text data and emotion data to the server, means for the server to analyze the preprocessed data using a generative artificial intelligence model and evaluate whether it constitutes power harassment, and means for generating an alert message and transmitting it to the terminal to notify the user if the possibility of power harassment is detected. This allows users to review their own comments and text in real time and receive feedback based on the emotion data.

[0751] A "terminal" is a device (such as a smartphone or personal computer) that allows a user to input voice or text.

[0752] "Speech recognition technology" is a technology that converts voice data into text data.

[0753] An "emotion engine" is a software technology that analyzes and classifies a user's emotions from input voice or text.

[0754] "Preprocessing" is a series of operations to prepare input data in a form that is easy to analyze.

[0755] A "generative artificial intelligence model" is an artificial intelligence system that learns from huge amounts of data and analyzes and evaluates input data.

[0756] "Power harassment" is the act of using one's position or status in the workplace to inflict mental or physical pain on others.

[0757] An "alert message" is a warning message that is generated and notified to the user when a specific condition (in this case, the possibility of power harassment) is detected.

[0758] A "server" is a central processing unit that receives and analyzes data, and generates and sends alert messages.

[0759] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect potential power harassment in real time and quickly issue an alert to the user.

[0760] Voice and text input and preprocessing

[0761] Devices (smartphones or personal computers) accept comments made during meetings and everyday communication (voice input), as well as text input (text input) for emails and chats in real time. Voice data is converted into text data using voice recognition technology (such as Google Speech-to-Text API). Text data is preprocessed into an easy-to-analyze form using a grammar analysis tool (such as SpaCy). In addition, an emotion engine (such as Amazon Comprehend) is used in combination to generate user emotion data from the voice and text.

[0762] Data transmission and analysis

[0763] The preprocessed text data and generated emotion data are sent to a server via the internet. The server runs a generative AI model (such as GPT-4) to analyze the received data. The generative AI model has learned from a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[0764] Matching and alert message generation

[0765] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[0766] Alerting and Notifications

[0767] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0768] User response

[0769] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[0770] Specific examples

[0771] Example 1: Voice alerts and emotion recognition during meetings

[0772] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[0773] 2. The device generates emotion data using an emotion engine.

[0774] 3. The preprocessed text data and emotion data are sent to the server.

[0775] 4. The server analyzes the data and detects the statement, "You're slow at work."

[0776] 5. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotions have been recognized as irritation."

[0777] 6. An alert message is sent to the terminal to notify the user.

[0778] 7. The user checks the alert and reviews their comments.

[0779] Example 2: Email alerts and emotion recognition

[0780] 1. The device retrieves the text of the email being composed in real time.

[0781] 2. The device generates emotion data using an emotion engine.

[0782] 3. The preprocessed text data and emotion data are sent to the server.

[0783] 4. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes it as dissatisfaction.

[0784] 5. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[0785] 6. An alert message is sent to the terminal to notify the user.

[0786] 7. The user checks the alert and corrects the email content.

[0787] Prompt Sentence Examples

[0788] The following prompt sentences are input to the generative AI model to analyze the data and generate an alert message:

[0789] Please rate whether the following statements constitute power harassment:

[0790] Saying: "You're slow at work"

[0791] Emotion Data: {"User Emotion": "Irritated"}

[0792] Additionally, generate an alert message that includes the reasons and feedback on why this statement is problematic.

[0793] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0794] Step 1:

[0795] The device collects user conversations and everyday communication as voice data. Specifically, it records voice using the microphone of a smartphone or personal computer. The input is voice data, which is collected in real time. The output is the voice data before conversion.

[0796] Step 2:

[0797] The device converts the collected voice data into text data using voice recognition technology. The technology used here is a voice recognition API (e.g., Google Speech-to-Text). The input is voice data, and the output is converted text data. Specifically, the voice is converted sequentially into recognizable text.

[0798] Step 3:

[0799] The device collects text data as sentence data. For example, emails and chat messages entered by users in real time are input here. The output is text data before preprocessing. The input strings are captured sequentially and prepared as data for analysis.

[0800] Step 4:

[0801] The terminal preprocesses the text data. Using a grammar analysis tool (e.g., SpaCy), it appropriately processes punctuation and special characters and converts them into a format that is easy to analyze. The input is text data before preprocessing, and the output is preprocessed text data. Specifically, the analysis tool is used to remove unnecessary elements and organize the data grammatically.

[0802] Step 5:

[0803] The device generates emotion data using an emotion engine. The technology used here is an emotion recognition engine (e.g., Amazon Comprehend). The input is preprocessed text data, and the output is emotion data. Specifically, the emotion engine analyzes emotions in the text and assigns labels such as anger or joy.

[0804] Step 6:

[0805] The terminal transmits the preprocessed text data and generated emotion data to a server via the Internet. The input is the preprocessed text data and emotion data, and the output is the transmitted data. Specifically, the terminal transmits the data to the server via an Internet connection.

[0806] Step 7:

[0807] The server analyzes the received text data and emotion data. The model used is a generative artificial intelligence model (e.g., GPT-4). The input is the transmitted data, and the output is the analysis results. Specifically, the AI ​​model detects specific words and phrases in the data and evaluates them along with the emotion data.

[0808] Step 8:

[0809] The server detects power harassment based on the analysis results. The input is the analysis results, and the output is an assessment of the possibility of power harassment. Specifically, the detection algorithm compares the analysis results with the assessment criteria to identify harmful remarks and behavior.

[0810] Step 9:

[0811] The server generates an alert message when there is a possibility of power harassment. This message includes feedback based on the specific problem and emotional data. The input is the evaluation result of power harassment, and the output is the alert message. Specifically, the alert generation algorithm creates a warning message based on the evaluation result.

[0812] Step 10:

[0813] The server sends the generated alert message to the terminal. The input is the alert message, and the output is the sent message. As a specific operation, the alert is delivered to the user's terminal via the message sending function.

[0814] Step 11:

[0815] The device notifies the user of the received alert message. Possible notification methods include push notification, pop-up display, and email. The input is the received alert message, and the output is the notification to the user. Specifically, the notification system displays the alert.

[0816] Step 12:

[0817] The user checks the alert message and reevaluates their own statements and writing. For example, if they make an overbearing statement during a meeting, they can reflect on it and be more careful about what they say in the future. The input is the alert message, and the output is the corrected statement or writing. The specific operation is for the user to review the content and make corrections.

[0818] (Application example 2)

[0819] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0820] Power harassment is becoming a serious problem in modern workplaces, causing deterioration in employees' mental health and the work environment. Effective measures to prevent power harassment, particularly in communication between superiors and subordinates, are needed. However, a system that can detect power harassment in real time and respond quickly has not yet been established. In addition, advanced analysis that takes into account not only voice and text data analysis but also emotional changes is required. However, conventional methods are often inadequate.

[0821] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0822] In this invention, the server includes a means for a terminal equipped with a generative artificial intelligence model to receive voice and text input in real time, a means for transmitting preprocessed data to the server, which then analyzes and collates the data using the generative artificial intelligence model, a means for generating an alert message and transmitting it to the terminal to notify the user when possible power harassment is detected, a means for recording conversations in real time using smart glasses and generating emotion data, and a means for sending push notifications using a smartphone, thereby enabling the detection of power harassment and rapid notification.

[0823] A "generative artificial intelligence model" is an AI technology that learns from vast amounts of data and generates and analyzes natural, human-like sentences and speech.

[0824] A "terminal" is an electronic device capable of accepting, processing, and displaying voice and text input in real time.

[0825] "Preprocessing" is the process of converting input voice data or text data into a form that is easier to analyze.

[0826] A "server" is a computer system installed to process data over a network.

[0827] "Matching" is the process of comparing the analyzed data with existing databases or standards.

[0828] An "alert message" is a warning message that notifies the user when a possible case of power harassment is detected.

[0829] "Smart glasses" are wearable devices that have built-in AR-compatible displays, cameras, and microphones, and can display information and collect data in real time.

[0830] "Real-time" means that processing and data acquisition are immediate.

[0831] "Emotional data" is data that indicates the user's emotional state, generated from analyzed speech or text.

[0832] "Push notification" is a technology that sends instant notifications from a server to a device.

[0833] The present invention is an advanced system for real-time detection and notification of workplace power harassment, utilizing a generative artificial intelligence model and an emotion engine. The system accepts and analyzes voice and text input in real time, and alerts the user if a risk of power harassment is detected. A specific embodiment of the system is described in detail below.

[0834] Hardware and software used

[0835] Hardware

[0836] Smart glasses: AR-enabled, equipped with a camera and microphone

[0837] Smartphone: iOS / Android compatible

[0838] software

[0839] Speech recognition engine: Google Cloud Speech-to-Text

[0840] Emotion engine: IBM Watson Tone Analyzer

[0841] Generative AI model: OpenAI's GPT series

[0842] Communications technology: Firebase, AWS

[0843] Push notification system: Firebase Cloud Messaging

[0844] System Operation

[0845] 1. Data Collection:

[0846] The server acquires audio and video data from the smart glasses and smartphone in real time.

[0847] This allows meetings and daily communications to be recorded.

[0848] 2. Pretreatment:

[0849] The server converts the voice data into text data using Google Cloud Speech-to-Text.

[0850] The text data is pre-processed to make it easier to interpret and analyze.

[0851] 3. Emotion data generation:

[0852] The server uses IBM Watson Tone Analyzer to generate emotion data from text data.

[0853] Emotional data indicates the user's emotional state (eg, anger, frustration, happiness).

[0854] 4. Data Analysis:

[0855] The server performs analysis using a generative artificial intelligence model (GPT series) based on the preprocessed text data and emotion data.

[0856] The server detects specific words and phrases in the input data and compares them with emotional data to assess the risk of power harassment.

[0857] 5. Alert Generation:

[0858] The server generates an alert message if a risk of power harassment is detected.

[0859] The alert message includes feedback on specific utterances or sentence problems and corresponding emotional data.

[0860] For example, "This statement is overbearing and may cause stress to the other person. Furthermore, your emotion is being perceived as anger."

[0861] 6. Notice:

[0862] The server sends the generated alert message to the smart glasses and the smartphone.

[0863] Users can view alerts via a pop-up display on their smart glasses or a push notification on their smartphone.

[0864] Specific examples

[0865] Example 1: Real-time detection during a meeting

[0866] Say: "How could you make such a basic mistake?"

[0867] Emotion data: Anger

[0868] Alert message: "This statement puts pressure on the other person and poses a risk of power harassment. We have also confirmed that your emotion is anger."

[0869] Example 2: Real-time chat monitoring

[0870] Say: "This project has been delayed entirely because of you."

[0871] Emotional data: Dissatisfied

[0872] Alert message: "This comment has been perceived as a personal attack and constitutes power harassment. Furthermore, your feelings of dissatisfaction have been confirmed."

[0873] Prompt Sentence Examples

[0874] Analyze the following statements and determine whether they pose a risk of power harassment. Also, consider the emotional data associated with the statements. Statement: "You're really useless. How many times do I have to tell you?" Emotional data: Anger

[0875] In this way, the present invention has the ability to detect the risk of power harassment in real time and promptly notify the user, thereby promoting a healthy work environment and protecting the mental health of employees.

[0876] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0877] Step 1:

[0878] The terminal collects conversation and audio data in real time through smart glasses and smartphones. The input is audio data and video data. The output is raw audio and video data collected in real time.

[0879] Step 2:

[0880] The voice data collected by the device is converted into text data using Google Cloud Speech-to-Text. The input is voice data. Data processing involves converting it into text data using voice recognition technology. The output is the converted text data.

[0881] Step 3:

[0882] The server receives the preprocessed text data from the terminal. Secure communication technologies such as AWS and Firebase are used for secure data transfer. The input is the converted text data. The output is the received text data.

[0883] Step 4:

[0884] Emotion data is generated from text data using IBM Watson Tone Analyzer. The input is the received text data. Data processing involves extracting and classifying the user's emotional state using emotion analysis technology. The output is the generated emotion data.

[0885] Step 5:

[0886] The server integrates the preprocessed text data and emotion data and performs analysis using a generative artificial intelligence model (OpenAI's GPT series). The input is the integrated text data and emotion data. Data calculation involves comparing the data with existing cases of power harassment and assessing the risk. The output is the analysis results.

[0887] Step 6:

[0888] The server evaluates the possibility of power harassment based on the analysis results and generates an alert message. The input is the analysis results. Data processing involves creating an alert message based on specific remarks and problems with the sentences, as well as feedback on emotional data. The output is the generated alert message.

[0889] Step 7:

[0890] The server uses Firebase Cloud Messaging to send the generated alert message to the devices (smart glasses and smartphones). The input is the generated alert message. The output is the sent alert message.

[0891] Step 8:

[0892] The user confirms the alert message through the device (a pop-up display on the smart glasses or a push notification on the smartphone). The input is the received alert message. The output is the user's confirmed alert message.

[0893] Step 9:

[0894] The user reevaluates their own statements and attitudes based on the alert content and makes corrections if necessary. The input is the confirmed alert message. The output is the corrected statements and attitudes.

[0895] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0896] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0897] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0898] [Third embodiment]

[0899] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0900] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0901] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0902] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0903] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0904] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0905] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0906] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0907] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0908] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0909] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0910] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0911] The present invention is a system that uses a generative artificial intelligence model to detect statements or text that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[0912] Voice and text input and preprocessing

[0913] The device (smartphone or personal computer) accepts speech during meetings, everyday communication (voice input), and text input (sentence input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and the sentence data is preprocessed using a grammar analysis tool to make it easier to analyze.

[0914] Data transmission and analysis

[0915] The preprocessed data is sent to a server via the internet. The server is equipped with a generative artificial intelligence model that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[0916] Matching and alert message generation

[0917] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[0918] Alerting and Notifications

[0919] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[0920] User response

[0921] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[0922] Specific examples

[0923] Example 1: Audio alert during a meeting

[0924] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[0925] 2. The preprocessed text data is sent to the server.

[0926] 3. The server analyzes the data and detects the statement, "You're slow at work."

[0927] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment."

[0928] 5. An alert message is sent to the terminal to notify the user.

[0929] 6. The user checks the alert and reviews their comments.

[0930] Example 2: Email alert

[0931] 1. The device retrieves the text of the email being composed in real time.

[0932] 2. The preprocessed text data is sent to the server.

[0933] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[0934] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment."

[0935] 5. An alert message is sent to the terminal to notify the user.

[0936] 6. The user checks the alert and corrects the email content.

[0937] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[0938] The processing flow will be explained below.

[0939] Step 1:

[0940] The terminal accepts voice and text input from the user.

[0941] In the case of voice input, speeches made during the meeting are recorded using a microphone.

[0942] When entering text, text data is obtained through a keyboard or touch screen.

[0943] The input data is temporarily stored in a buffer.

[0944] Step 2:

[0945] Preprocess the data acquired by the device.

[0946] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[0947] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[0948] The converted text data is formatted into a format that is easy to parse.

[0949] Step 3:

[0950] The terminal transmits the preprocessed text data to the server.

[0951] To ensure secure data transfer through the endpoints, encryption is provided using the SSL / TLS protocol.

[0952] Executes the appropriate API calls to send the data to the server.

[0953] Step 4:

[0954] The server parses the received data.

[0955] It receives input data and passes it to its own data analysis engine.

[0956] It uses generative artificial intelligence models to analyze and identify words and phrases within the text.

[0957] Past cases of power harassment are referenced from the database and cross-checked.

[0958] Step 5:

[0959] The server evaluates the possibility of power harassment.

[0960] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[0961] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[0962] Step 6:

[0963] The server generates an alert message.

[0964] If possible power harassment is detected, an alert message is generated.

[0965] It identifies specific problems in the text and the reasons for them. It generates content such as, "This statement is overbearing and may cause stress to the recipient, so it constitutes power harassment."

[0966] Step 7:

[0967] The server generates an alert message and sends it to the terminal.

[0968] An alert message is sent to the terminal using secure communications means.

[0969] Send data to the device's receiving API.

[0970] Step 8:

[0971] The device displays an alert notification to the user.

[0972] Received alert messages are displayed as push notifications or pop-up notifications.

[0973] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[0974] Step 9:

[0975] The user checks the alert message and takes action.

[0976] Check the alert message and review your comments and writings.

[0977] For example, take action such as revising the content of the email or improving the content of the statement.

[0978] This allows the system to detect potential power harassment in real time, providing users with the opportunity to respond quickly.

[0979] Example 1

[0980] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0981] Power harassment is a serious problem in modern workplaces, often having a negative impact on employees' mental health and the work environment. Therefore, it is important to quickly identify potential power harassment in statements or texts and respond appropriately. However, current approaches make it difficult to monitor and notify in real time, and it is also difficult to identify specific problems. Therefore, there is a need for a system that can detect power harassment in real time and promptly notify users.

[0982] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0983] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting the accepted voice into text data using voice recognition technology, means for preprocessing the accepted text data using a grammar analysis tool, means for transmitting the preprocessed data to the server, means for the server to analyze the received data using a generative artificial intelligence model and evaluate the possibility of power harassment, means for generating an alert message if the possibility of power harassment is detected, and means for transmitting the generated alert message to the terminal to notify the user. This makes it possible to detect the possibility of power harassment in real time and quickly notify the user of an alert message explaining the specific problem.

[0984] "Terminal" refers to any device that accepts voice and text input in real time.

[0985] "Speech recognition technology" refers to the technology that converts voice into text data.

[0986] A "grammar analysis tool" refers to software that preprocesses text into a form that is easier to analyze.

[0987] "Preprocessing" refers to the process of formatting the original data into a form that is easier to analyze.

[0988] "Server" refers to a computer system that performs processes such as data analysis and alert message generation.

[0989] A "generative artificial intelligence model" refers to an artificial intelligence model that learns from large amounts of data and analyzes input data.

[0990] "Analysis" refers to the process of evaluating input data and understanding and classifying its contents.

[0991] "Power harassment" refers to the act of using one's power or position to inflict psychological or physical pain on others.

[0992] "Alert Message" means a notification message sent to warn of a particular event or problem.

[0993] "Notification" refers to the act of informing a user of specific information.

[0994] "Real-time" refers to the process from receiving data to providing results occurring instantly.

[0995] The present invention is a system that uses a generative artificial intelligence model to detect statements or sentences that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[0996] Voice and text input and preprocessing

[0997] The terminals are devices such as smartphones and personal computers. They accept voice input during meetings and everyday communication, as well as text input for emails and chats, in real time. Voice data is converted into text data using speech recognition technology within the terminal (for example, Google Recognition API). Text data is preprocessed into a form that is easy to analyze using a grammar analysis tool (for example, SpaCy or NLTK).

[0998] Data transmission and analysis

[0999] The preprocessed data is sent over the internet to a server, which is equipped with a generative artificial intelligence model (e.g., GPT-4) that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[1000] Matching and alert message generation

[1001] If the received data is judged to be a possible example of power harassment, the server will generate an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[1002] Alerting and Notifications

[1003] The generated alert message is sent back from the server to the device. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[1004] User response

[1005] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[1006] Specific examples

[1007] Example 1: Audio alert during a meeting

[1008] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[1009] 2. The preprocessed text data is sent to the server.

[1010] 3. The server analyzes the data and detects the statement, "You're slow at work."

[1011] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is non-constructive and constitutes power harassment."

[1012] 5. An alert message is sent to the terminal to notify the user.

[1013] 6. The user checks the alert and reviews their comments.

[1014] Example 2: Email alert

[1015] 1. The device retrieves the text of the email being composed in real time.

[1016] 2. The preprocessed text data is sent to the server.

[1017] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[1018] 4. The server determines that this sentence is power harassment and generates an alert message such as, "This sentence has been recognized as a personal attack and constitutes power harassment."

[1019] 5. An alert message is sent to the terminal to notify the user.

[1020] 6. The user checks the alert and corrects the email content.

[1021] Prompt Sentence Examples

[1022] "Evaluate whether the following statement constitutes power harassment and explain why: 'Your work is completely worthless.'"

[1023] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[1024] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1025] Step 1:

[1026] The device accepts voice and text input in real time. Specifically, the device's microphone is used to record what is said during a meeting. Text data is also obtained via email or chat applications. The input data can be voice or text data.

[1027] Step 2:

[1028] The device converts voice data into text data using speech recognition technology. Specifically, it analyzes the recorded voice using the Google Recognition API and outputs it as text data. The input here is voice data, and the output is the converted text data.

[1029] Step 3:

[1030] The text data received by the terminal is preprocessed using a grammar analysis tool. Specifically, grammar analysis tools such as SpaCy and NLTK are used to tokenize and parse the text, which then formats it into a format suitable for analysis. The input is raw text data, and the output is text data that has undergone preprocessing.

[1031] Step 4:

[1032] The terminal sends the preprocessed text data to the server. Specifically, it uses an HTTP POST request to send data to the server in JSON format. The input is the preprocessed text data, and the output is the result sent to the server.

[1033] Step 5:

[1034] The server analyzes the received data using a generative artificial intelligence model. Specifically, it uses GPT-4 installed on the server to evaluate whether words and phrases in the data constitute power harassment. The input is preprocessed text data, and the output is the analysis results.

[1035] Step 6:

[1036] The server generates an alert message based on the analysis results. Specifically, for detected cases of possible power harassment, it creates a message that explains in detail why the case constitutes power harassment. The input is the analysis results, and the output is the alert message.

[1037] Step 7:

[1038] The server sends the generated alert message to the terminal again. Specifically, it sends the message to the terminal using an HTTP POST request. The input is the alert message, and the output is the result sent to the terminal.

[1039] Step 8:

[1040] The device notifies the user of the alert message received. Specifically, it notifies the user using a push notification, a popup display, or an email. The input is the alert message, and the output is the notification sent to the user.

[1041] Step 9:

[1042] Users receive an alert message and reevaluate their statements or writing. Specifically, if they make an overbearing remark during a meeting, they reflect on their remarks and pay more attention to what they say in the future. In the case of emails or chats, they correct the content before sending. The input is the alert message, and the output is the corrected remarks or writing.

[1043] (Application example 1)

[1044] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1045] In physical stores, there is a need to detect power harassment that may occur during communication between store staff and customers in real time and deal with it on the spot.An effective system is needed to reduce customer dissatisfaction and store staff stress and maintain a healthy working environment.

[1046] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1047] In this invention, the server includes: means for a terminal equipped with a generative artificial intelligence model to accept voice and text input in real time; means for sending preprocessed data to the server, which analyzes and compares the data using the generative artificial intelligence model; means for generating an alert message and sending it to the terminal to notify the user when possible power harassment is detected; and means for collecting voice data using a device worn by store clerks and analyzing it in real time based on the generative artificial intelligence model. This allows conversations between store clerks and customers in a physical store to be monitored in real time, making it possible to detect and address signs of power harassment early on.

[1048] A "generative artificial intelligence model" is an artificial intelligence algorithm that can learn from huge data sets and understand and generate new input data.

[1049] A "terminal" is a device that a user operates to input or output data, and includes smartphones, computers, and the like.

[1050] "Server" means a computer system that receives and processes data over a network and transmits the analysis results.

[1051] "Power harassment" refers to the act of using one's position in the workplace to inflict mental or physical pain on another.

[1052] An "alert message" is notification information that alerts the user when a specific condition is met.

[1053] The "device" refers to the hardware worn by the store clerk for collecting voice data.

[1054] The present invention is a system that monitors voice communications between store clerks and customers in real time, and quickly detects and alerts users when statements that may constitute power harassment. The specific system configuration and processing procedures for implementing the present invention are described below.

[1055] Hardware configuration

[1056] This system uses the following hardware:

[1057] Device: Refers to smart glasses or smartphones for voice input and real-time notifications.

[1058] Server: A high-performance computer system that runs generative artificial intelligence models.

[1059] Device: Smart glasses worn by store staff and a microphone for collecting audio data.

[1060] Software configuration

[1061] This system consists of the following software:

[1062] Speech Recognition Software: Software for converting speech to text, such as the Google Cloud Speech-to-Text API.

[1063] Generative artificial intelligence model: An AI model for detecting power harassment using Transformer-based models such as BERT.

[1064] Communication Protocol: SSL / TLS protocol for secure data communication.

[1065] Data processing and calculation procedures

[1066] The device records conversations between store staff and customers in real time, and the recorded audio data is converted into text data by voice recognition software within the smart glasses, which is then pre-processed and sent to the server via a secure communication protocol.

[1067] The server analyzes the received text data using a generative artificial intelligence model (such as BERT). This analysis evaluates whether the phrases and words in the data constitute power harassment. If the evaluation results indicate the possibility of power harassment, the server generates an alert message pointing out the specific problem.

[1068] The generated alert message is sent to the terminal and notified to the user (store clerk) as a push notification or a pop-up display on the smart glasses or smartphone, allowing the store clerk to recognize the problematic part of their own statement in real time and correct their response appropriately.

[1069] Specific examples

[1070] For example, if a store clerk at a brick-and-mortar convenience store says to a customer, "This is simple, hurry up, the next person is waiting," the system will analyze the remark in real time and display a warning on the smart glasses saying, "This remark is overbearing and constitutes power harassment." This will enable the store clerk to immediately review their remark and correct it to respond appropriately to the customer.

[1071] Prompt Sentence Examples

[1072] TXT

[1073] <audio_file.wav>

[1074] -store_interaction

[1075] -convert_audio_to_text

[1076] -check_for_power_harassment

[1077] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1078] Step 1:

[1079] The device records conversations between store staff and customers in real time, using the microphone built into the smart glasses to collect audio data.

[1080] Input: Voice conversation between store clerk and customer

[1081] Output: Audio data file (.wav, etc.)

[1082] Step 2:

[1083] The device converts the collected voice data into text data using the Google Cloud Speech-to-Text API, and the converted text is preprocessed.

[1084] Input: Audio data file

[1085] Output: Text data

[1086] Step 3:

[1087] The terminal transmits the preprocessed text data to the server using a secure communication protocol (SSL / TLS).

[1088] Input: Preprocessed text data

[1089] Output: Sending text data

[1090] Step 4:

[1091] The server analyzes the received text data using a generative AI model (such as BERT), which analyzes words and phrases in the input text data and performs data calculations to evaluate whether they constitute power harassment.

[1092] Input: Transmitted text data

[1093] Output: Analysis results (power harassment judgment)

[1094] Step 5:

[1095] If the server detects possible power harassment, it generates an alert message explaining why a particular statement or sentence constitutes power harassment.

[1096] Input: Analysis result (power harassment judgment)

[1097] Output: Alert message

[1098] Step 6:

[1099] The server then sends the generated alert message back to the device, again using a secure protocol.

[1100] Input: Alert message

[1101] Output: Sending an alert message

[1102] Step 7:

[1103] The device notifies the user (store clerk) of the received alert message via a pop-up display on the smart glasses display, a push notification, or other means.

[1104] Input: Alert message

[1105] Output: User notification

[1106] Step 8:

[1107] The user checks the alert message, reviews their comments, and, if necessary, corrects the content of their comments and takes appropriate action.

[1108] Input: Alert message

[1109] Output: User behavior modification

[1110] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1111] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect the possibility of power harassment in real time and quickly issue an alert to the user. A specific embodiment of this system is described below.

[1112] Voice and text input and preprocessing

[1113] The device (smartphone or personal computer) accepts comments made during meetings, everyday communication (voice input), and text input (text input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and text data is preprocessed using grammar analysis tools to make it easier to analyze. An emotion engine is also used, and user emotion data is simultaneously generated from the voice and text.

[1114] Data transmission and analysis

[1115] The preprocessed text data and generated emotion data are sent to a server via the internet. The server's generative AI model analyzes the received data. This AI model has previously studied a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[1116] Matching and alert message generation

[1117] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[1118] Alerting and Notifications

[1119] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[1120] User response

[1121] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[1122] Specific examples

[1123] Example 1: Voice alerts and emotion recognition during meetings

[1124] 1. The device records what is said during the meeting, converts it into text using voice recognition, and generates emotional data using an emotion engine.

[1125] 2. The preprocessed text data and emotion data are sent to the server.

[1126] 3. The server analyzes the data and detects the statement, "You're slow at work."

[1127] 4. The server determines that this comment constitutes power harassment and, taking into account the emotional data, generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotion has been recognized as irritation."

[1128] 5. An alert message is sent to the terminal to notify the user.

[1129] 6. The user checks the alert and reviews their comments.

[1130] Example 2: Email alerts and emotion recognition

[1131] 1. The device obtains the text of the email being composed in real time and generates emotion data using the emotion engine.

[1132] 2. The preprocessed text data and emotion data are sent to the server.

[1133] 3. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes the emotional data as dissatisfaction.

[1134] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[1135] 5. An alert message is sent to the terminal to notify the user.

[1136] 6. The user checks the alert and corrects the email content.

[1137] This invention allows users to recognize problems with their own statements, writing, and emotions in real time and have the opportunity to correct them, thereby more effectively preventing power harassment and improving the working environment.

[1138] The processing flow will be explained below.

[1139] Step 1:

[1140] The terminal accepts voice and text input from the user.

[1141] In the case of voice input, speech during the meeting is recorded using a microphone, and the recorded audio is stored in the device.

[1142] In the case of text input, the text data entered by the user via a keyboard or touch screen is acquired.

[1143] The input data is temporarily stored in a buffer.

[1144] Step 2:

[1145] Preprocess the data acquired by the device.

[1146] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[1147] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[1148] The converted text data is formatted into a format that is easy to parse.

[1149] Step 3:

[1150] The device generates emotion data.

[1151] In the case of audio data, the emotion engine analyzes it to extract the tone, speed, and emotional expression of the voice.

[1152] For text data, the emotion engine analyzes the context and keywords of the text to generate an emotion classification (e.g., anger, frustration, joy, etc.).

[1153] Step 4:

[1154] The terminal transmits the preprocessed text data and emotion data to the server.

[1155] The endpoints use SSL / TLS encryption to ensure secure data transfer.

[1156] Executes the appropriate API calls to send the data to the server.

[1157] Step 5:

[1158] The server parses the received data.

[1159] It receives text data and emotion data and passes it to its own data analysis engine.

[1160] Using a generative artificial intelligence model, words and phrases within text are identified and analyzed along with sentiment data.

[1161] Past cases of power harassment are referenced from the database and cross-checked.

[1162] Step 6:

[1163] The server evaluates the possibility of power harassment.

[1164] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[1165] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[1166] Step 7:

[1167] The server generates an alert message.

[1168] If possible power harassment is detected, an alert message is generated.

[1169] It describes specific problems in the text and the reasons for them. It also includes feedback based on emotional data, such as, "This statement is overbearing and may cause stress to the recipient. Furthermore, your emotion is recognized as anger."

[1170] Step 8:

[1171] The server generates an alert message and sends it to the terminal.

[1172] An alert message is sent to the terminal using secure communications means.

[1173] Send data to the device's receiving API.

[1174] Step 9:

[1175] The device displays an alert notification to the user.

[1176] Received alert messages are displayed as push notifications or pop-up notifications.

[1177] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[1178] Step 10:

[1179] The user checks the alert message and takes action.

[1180] Check the alert message and review your statements, writing, and emotions.

[1181] For example, take action such as revising the content of the email or improving the content of the statement.

[1182] As a result of this trend, the system not only detects potential power harassment in real time and provides users with an opportunity to respond quickly, but also takes into account the user's emotions, promoting more effective communication improvements.

[1183] Example 2

[1184] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1185] In today's work environment, mental stress and reduced motivation due to power harassment have become serious problems. Real-time monitoring and specific alerts are necessary to prevent this from happening, but conventional systems have difficulty in detecting power harassment in real time and generating appropriate alerts. Therefore, there is a need to establish a system that can detect power harassment in real time, including recognizing emotions, and provide appropriate feedback.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1187] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting voice data into text data using voice recognition technology, means for generating user emotion data using an emotion engine, means for transmitting the preprocessed text data and emotion data to the server, means for the server to analyze the preprocessed data using a generative artificial intelligence model and evaluate whether it constitutes power harassment, and means for generating an alert message and transmitting it to the terminal to notify the user if the possibility of power harassment is detected. This allows users to review their own comments and text in real time and receive feedback based on the emotion data.

[1188] A "terminal" is a device (such as a smartphone or personal computer) that allows a user to input voice or text.

[1189] "Speech recognition technology" is a technology that converts voice data into text data.

[1190] An "emotion engine" is a software technology that analyzes and classifies a user's emotions from input voice or text.

[1191] "Preprocessing" is a series of operations to prepare input data in a form that is easy to analyze.

[1192] A "generative artificial intelligence model" is an artificial intelligence system that learns from huge amounts of data and analyzes and evaluates input data.

[1193] "Power harassment" is the act of using one's position or status in the workplace to inflict mental or physical pain on others.

[1194] An "alert message" is a warning message that is generated and notified to the user when a specific condition (in this case, the possibility of power harassment) is detected.

[1195] A "server" is a central processing unit that receives and analyzes data, and generates and sends alert messages.

[1196] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect potential power harassment in real time and quickly issue an alert to the user.

[1197] Voice and text input and preprocessing

[1198] Devices (smartphones or personal computers) accept comments made during meetings and everyday communication (voice input), as well as text input (text input) for emails and chats in real time. Voice data is converted into text data using voice recognition technology (such as Google Speech-to-Text API). Text data is preprocessed into an easy-to-analyze form using a grammar analysis tool (such as SpaCy). In addition, an emotion engine (such as Amazon Comprehend) is used in combination to generate user emotion data from the voice and text.

[1199] Data transmission and analysis

[1200] The preprocessed text data and generated emotion data are sent to a server via the internet. The server runs a generative AI model (such as GPT-4) to analyze the received data. The generative AI model has learned from a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[1201] Matching and alert message generation

[1202] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[1203] Alerting and Notifications

[1204] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[1205] User response

[1206] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[1207] Specific examples

[1208] Example 1: Voice alerts and emotion recognition during meetings

[1209] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[1210] 2. The device generates emotion data using an emotion engine.

[1211] 3. The preprocessed text data and emotion data are sent to the server.

[1212] 4. The server analyzes the data and detects the statement, "You're slow at work."

[1213] 5. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotions have been recognized as irritation."

[1214] 6. An alert message is sent to the terminal to notify the user.

[1215] 7. The user checks the alert and reviews their comments.

[1216] Example 2: Email alerts and emotion recognition

[1217] 1. The device retrieves the text of the email being composed in real time.

[1218] 2. The device generates emotion data using an emotion engine.

[1219] 3. The preprocessed text data and emotion data are sent to the server.

[1220] 4. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes it as dissatisfaction.

[1221] 5. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[1222] 6. An alert message is sent to the terminal to notify the user.

[1223] 7. The user checks the alert and corrects the email content.

[1224] Prompt Sentence Examples

[1225] The following prompt sentences are input to the generative AI model to analyze the data and generate an alert message:

[1226] Please rate whether the following statements constitute power harassment:

[1227] Saying: "You're slow at work"

[1228] Emotion Data: {"User Emotion": "Irritated"}

[1229] Additionally, generate an alert message that includes the reasons and feedback on why this statement is problematic.

[1230] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1231] Step 1:

[1232] The device collects user conversations and everyday communication as voice data. Specifically, it records voice using the microphone of a smartphone or personal computer. The input is voice data, which is collected in real time. The output is the voice data before conversion.

[1233] Step 2:

[1234] The device converts the collected voice data into text data using voice recognition technology. The technology used here is a voice recognition API (e.g., Google Speech-to-Text). The input is voice data, and the output is converted text data. Specifically, the voice is converted sequentially into recognizable text.

[1235] Step 3:

[1236] The device collects text data as sentence data. For example, emails and chat messages entered by users in real time are input here. The output is text data before preprocessing. The input strings are captured sequentially and prepared as data for analysis.

[1237] Step 4:

[1238] The terminal preprocesses the text data. Using a grammar analysis tool (e.g., SpaCy), it appropriately processes punctuation and special characters and converts them into a format that is easy to analyze. The input is text data before preprocessing, and the output is preprocessed text data. Specifically, the analysis tool is used to remove unnecessary elements and organize the data grammatically.

[1239] Step 5:

[1240] The device generates emotion data using an emotion engine. The technology used here is an emotion recognition engine (e.g., Amazon Comprehend). The input is preprocessed text data, and the output is emotion data. Specifically, the emotion engine analyzes emotions in the text and assigns labels such as anger or joy.

[1241] Step 6:

[1242] The terminal transmits the preprocessed text data and generated emotion data to a server via the Internet. The input is the preprocessed text data and emotion data, and the output is the transmitted data. Specifically, the terminal transmits the data to the server via an Internet connection.

[1243] Step 7:

[1244] The server analyzes the received text data and emotion data. The model used is a generative artificial intelligence model (e.g., GPT-4). The input is the transmitted data, and the output is the analysis results. Specifically, the AI ​​model detects specific words and phrases in the data and evaluates them along with the emotion data.

[1245] Step 8:

[1246] The server detects power harassment based on the analysis results. The input is the analysis results, and the output is an assessment of the possibility of power harassment. Specifically, the detection algorithm compares the analysis results with the assessment criteria to identify harmful remarks and behavior.

[1247] Step 9:

[1248] The server generates an alert message when there is a possibility of power harassment. This message includes feedback based on the specific problem and emotional data. The input is the evaluation result of power harassment, and the output is the alert message. Specifically, the alert generation algorithm creates a warning message based on the evaluation result.

[1249] Step 10:

[1250] The server sends the generated alert message to the terminal. The input is the alert message, and the output is the sent message. As a specific operation, the alert is delivered to the user's terminal via the message sending function.

[1251] Step 11:

[1252] The device notifies the user of the received alert message. Possible notification methods include push notification, pop-up display, and email. The input is the received alert message, and the output is the notification to the user. Specifically, the notification system displays the alert.

[1253] Step 12:

[1254] The user checks the alert message and reevaluates their own statements and writing. For example, if they make an overbearing statement during a meeting, they can reflect on it and be more careful about what they say in the future. The input is the alert message, and the output is the corrected statement or writing. The specific operation is for the user to review the content and make corrections.

[1255] (Application example 2)

[1256] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1257] Power harassment is becoming a serious problem in modern workplaces, causing deterioration in employees' mental health and the work environment. Effective measures to prevent power harassment, particularly in communication between superiors and subordinates, are needed. However, a system that can detect power harassment in real time and respond quickly has not yet been established. In addition, advanced analysis that takes into account not only voice and text data analysis but also emotional changes is required. However, conventional methods are often inadequate.

[1258] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1259] In this invention, the server includes a means for a terminal equipped with a generative artificial intelligence model to receive voice and text input in real time, a means for transmitting preprocessed data to the server, which then analyzes and collates the data using the generative artificial intelligence model, a means for generating an alert message and transmitting it to the terminal to notify the user when possible power harassment is detected, a means for recording conversations in real time using smart glasses and generating emotion data, and a means for sending push notifications using a smartphone, thereby enabling the detection of power harassment and rapid notification.

[1260] A "generative artificial intelligence model" is an AI technology that learns from vast amounts of data and generates and analyzes natural, human-like sentences and speech.

[1261] A "terminal" is an electronic device capable of accepting, processing, and displaying voice and text input in real time.

[1262] "Preprocessing" is the process of converting input voice data or text data into a form that is easier to analyze.

[1263] A "server" is a computer system installed to process data over a network.

[1264] "Matching" is the process of comparing the analyzed data with existing databases or standards.

[1265] An "alert message" is a warning message that notifies the user when a possible case of power harassment is detected.

[1266] "Smart glasses" are wearable devices that have built-in AR-compatible displays, cameras, and microphones, and can display information and collect data in real time.

[1267] "Real-time" means that processing and data acquisition are immediate.

[1268] "Emotional data" is data that indicates the user's emotional state, generated from analyzed speech or text.

[1269] "Push notification" is a technology that sends instant notifications from a server to a device.

[1270] The present invention is an advanced system for real-time detection and notification of workplace power harassment, utilizing a generative artificial intelligence model and an emotion engine. The system accepts and analyzes voice and text input in real time, and alerts the user if a risk of power harassment is detected. A specific embodiment of the system is described in detail below.

[1271] Hardware and software used

[1272] Hardware

[1273] Smart glasses: AR-enabled, equipped with a camera and microphone

[1274] Smartphone: iOS / Android compatible

[1275] software

[1276] Speech recognition engine: Google Cloud Speech-to-Text

[1277] Emotion engine: IBM Watson Tone Analyzer

[1278] Generative AI model: OpenAI's GPT series

[1279] Communications technology: Firebase, AWS

[1280] Push notification system: Firebase Cloud Messaging

[1281] System Operation

[1282] 1. Data Collection:

[1283] The server acquires audio and video data from the smart glasses and smartphone in real time.

[1284] This allows meetings and daily communications to be recorded.

[1285] 2. Pretreatment:

[1286] The server converts the voice data into text data using Google Cloud Speech-to-Text.

[1287] The text data is pre-processed to make it easier to interpret and analyze.

[1288] 3. Emotion data generation:

[1289] The server uses IBM Watson Tone Analyzer to generate emotion data from text data.

[1290] Emotional data indicates the user's emotional state (eg, anger, frustration, happiness).

[1291] 4. Data Analysis:

[1292] The server performs analysis using a generative artificial intelligence model (GPT series) based on the preprocessed text data and emotion data.

[1293] The server detects specific words and phrases in the input data and compares them with emotional data to assess the risk of power harassment.

[1294] 5. Alert Generation:

[1295] The server generates an alert message if a risk of power harassment is detected.

[1296] The alert message includes feedback on specific utterances or sentence problems and corresponding emotional data.

[1297] For example, "This statement is overbearing and may cause stress to the other person. Furthermore, your emotion is being perceived as anger."

[1298] 6. Notice:

[1299] The server sends the generated alert message to the smart glasses and the smartphone.

[1300] Users can view alerts via a pop-up display on their smart glasses or a push notification on their smartphone.

[1301] Specific examples

[1302] Example 1: Real-time detection during a meeting

[1303] Say: "How could you make such a basic mistake?"

[1304] Emotion data: Anger

[1305] Alert message: "This statement puts pressure on the other person and poses a risk of power harassment. We have also confirmed that your emotion is anger."

[1306] Example 2: Real-time chat monitoring

[1307] Say: "This project has been delayed entirely because of you."

[1308] Emotional data: Dissatisfied

[1309] Alert message: "This comment has been perceived as a personal attack and constitutes power harassment. Furthermore, your feelings of dissatisfaction have been confirmed."

[1310] Prompt Sentence Examples

[1311] Analyze the following statements and determine whether they pose a risk of power harassment. Also, consider the emotional data associated with the statements. Statement: "You're really useless. How many times do I have to tell you?" Emotional data: Anger

[1312] In this way, the present invention has the ability to detect the risk of power harassment in real time and promptly notify the user, thereby promoting a healthy work environment and protecting the mental health of employees.

[1313] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1314] Step 1:

[1315] The terminal collects conversation and audio data in real time through smart glasses and smartphones. The input is audio data and video data. The output is raw audio and video data collected in real time.

[1316] Step 2:

[1317] The voice data collected by the device is converted into text data using Google Cloud Speech-to-Text. The input is voice data. Data processing involves converting it into text data using voice recognition technology. The output is the converted text data.

[1318] Step 3:

[1319] The server receives the preprocessed text data from the terminal. Secure communication technologies such as AWS and Firebase are used for secure data transfer. The input is the converted text data. The output is the received text data.

[1320] Step 4:

[1321] Emotion data is generated from text data using IBM Watson Tone Analyzer. The input is the received text data. Data processing involves extracting and classifying the user's emotional state using emotion analysis technology. The output is the generated emotion data.

[1322] Step 5:

[1323] The server integrates the preprocessed text data and emotion data and performs analysis using a generative artificial intelligence model (OpenAI's GPT series). The input is the integrated text data and emotion data. Data calculation involves comparing the data with existing cases of power harassment and assessing the risk. The output is the analysis results.

[1324] Step 6:

[1325] The server evaluates the possibility of power harassment based on the analysis results and generates an alert message. The input is the analysis results. Data processing involves creating an alert message based on specific remarks and problems with the sentences, as well as feedback on emotional data. The output is the generated alert message.

[1326] Step 7:

[1327] The server uses Firebase Cloud Messaging to send the generated alert message to the devices (smart glasses and smartphones). The input is the generated alert message. The output is the sent alert message.

[1328] Step 8:

[1329] The user confirms the alert message through the device (a pop-up display on the smart glasses or a push notification on the smartphone). The input is the received alert message. The output is the user's confirmed alert message.

[1330] Step 9:

[1331] The user reevaluates their own statements and attitudes based on the alert content and makes corrections if necessary. The input is the confirmed alert message. The output is the corrected statements and attitudes.

[1332] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1333] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1334] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1335] [Fourth embodiment]

[1336] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1337] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1338] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1339] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1340] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1341] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1342] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1343] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1344] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1347] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1348] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1349] The present invention is a system that uses a generative artificial intelligence model to detect statements or text that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[1350] Voice and text input and preprocessing

[1351] The device (smartphone or personal computer) accepts speech during meetings, everyday communication (voice input), and text input (sentence input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and the sentence data is preprocessed using a grammar analysis tool to make it easier to analyze.

[1352] Data transmission and analysis

[1353] The preprocessed data is sent to a server via the internet. The server is equipped with a generative artificial intelligence model that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[1354] Matching and alert message generation

[1355] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[1356] Alerting and Notifications

[1357] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[1358] User response

[1359] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[1360] Specific examples

[1361] Example 1: Audio alert during a meeting

[1362] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[1363] 2. The preprocessed text data is sent to the server.

[1364] 3. The server analyzes the data and detects the statement, "You're slow at work."

[1365] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment."

[1366] 5. An alert message is sent to the terminal to notify the user.

[1367] 6. The user checks the alert and reviews their comments.

[1368] Example 2: Email alert

[1369] 1. The device retrieves the text of the email being composed in real time.

[1370] 2. The preprocessed text data is sent to the server.

[1371] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[1372] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment."

[1373] 5. An alert message is sent to the terminal to notify the user.

[1374] 6. The user checks the alert and corrects the email content.

[1375] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[1376] The processing flow will be explained below.

[1377] Step 1:

[1378] The terminal accepts voice and text input from the user.

[1379] In the case of voice input, speeches made during the meeting are recorded using a microphone.

[1380] When entering text, text data is obtained through a keyboard or touch screen.

[1381] The input data is temporarily stored in a buffer.

[1382] Step 2:

[1383] Preprocess the data acquired by the device.

[1384] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[1385] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[1386] The converted text data is formatted into a format that is easy to parse.

[1387] Step 3:

[1388] The terminal transmits the preprocessed text data to the server.

[1389] To ensure secure data transfer through the endpoints, encryption is provided using the SSL / TLS protocol.

[1390] Executes the appropriate API calls to send the data to the server.

[1391] Step 4:

[1392] The server parses the received data.

[1393] It receives input data and passes it to its own data analysis engine.

[1394] It uses generative artificial intelligence models to analyze and identify words and phrases within the text.

[1395] Past cases of power harassment are referenced from the database and cross-checked.

[1396] Step 5:

[1397] The server evaluates the possibility of power harassment.

[1398] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[1399] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[1400] Step 6:

[1401] The server generates an alert message.

[1402] If possible power harassment is detected, an alert message is generated.

[1403] It identifies specific problems in the text and the reasons for them. It generates content such as, "This statement is overbearing and may cause stress to the recipient, so it constitutes power harassment."

[1404] Step 7:

[1405] The server generates an alert message and sends it to the terminal.

[1406] An alert message is sent to the terminal using secure communications means.

[1407] Send data to the device's receiving API.

[1408] Step 8:

[1409] The device displays an alert notification to the user.

[1410] Received alert messages are displayed as push notifications or pop-up notifications.

[1411] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[1412] Step 9:

[1413] The user checks the alert message and takes action.

[1414] Check the alert message and review your comments and writings.

[1415] For example, take action such as revising the content of the email or improving the content of the statement.

[1416] This allows the system to detect potential power harassment in real time, providing users with the opportunity to respond quickly.

[1417] Example 1

[1418] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1419] Power harassment is a serious problem in modern workplaces, often having a negative impact on employees' mental health and the work environment. Therefore, it is important to quickly identify potential power harassment in statements or texts and respond appropriately. However, current approaches make it difficult to monitor and notify in real time, and it is also difficult to identify specific problems. Therefore, there is a need for a system that can detect power harassment in real time and promptly notify users.

[1420] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1421] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting the accepted voice into text data using voice recognition technology, means for preprocessing the accepted text data using a grammar analysis tool, means for transmitting the preprocessed data to the server, means for the server to analyze the received data using a generative artificial intelligence model and evaluate the possibility of power harassment, means for generating an alert message if the possibility of power harassment is detected, and means for transmitting the generated alert message to the terminal to notify the user. This makes it possible to detect the possibility of power harassment in real time and quickly notify the user of an alert message explaining the specific problem.

[1422] "Terminal" refers to any device that accepts voice and text input in real time.

[1423] "Speech recognition technology" refers to the technology that converts voice into text data.

[1424] A "grammar analysis tool" refers to software that preprocesses text into a form that is easier to analyze.

[1425] "Preprocessing" refers to the process of formatting the original data into a form that is easier to analyze.

[1426] "Server" refers to a computer system that performs processes such as data analysis and alert message generation.

[1427] A "generative artificial intelligence model" refers to an artificial intelligence model that learns from large amounts of data and analyzes input data.

[1428] "Analysis" refers to the process of evaluating input data and understanding and classifying its contents.

[1429] "Power harassment" refers to the act of using one's power or position to inflict psychological or physical pain on others.

[1430] "Alert Message" means a notification message sent to warn of a particular event or problem.

[1431] "Notification" refers to the act of informing a user of specific information.

[1432] "Real-time" refers to the process from receiving data to providing results occurring instantly.

[1433] The present invention is a system that uses a generative artificial intelligence model to detect statements or sentences that may be power harassment in real time and quickly issue an alert. Specific embodiments of this system are described below.

[1434] Voice and text input and preprocessing

[1435] The terminals are devices such as smartphones and personal computers. They accept voice input during meetings and everyday communication, as well as text input for emails and chats, in real time. Voice data is converted into text data using speech recognition technology within the terminal (for example, Google Recognition API). Text data is preprocessed into a form that is easy to analyze using a grammar analysis tool (for example, SpaCy or NLTK).

[1436] Data transmission and analysis

[1437] The preprocessed data is sent over the internet to a server, which is equipped with a generative artificial intelligence model (e.g., GPT-4) that analyzes the received data. This AI model has previously studied a large number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment.

[1438] Matching and alert message generation

[1439] If the received data is judged to be a possible example of power harassment, the server will generate an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. For example, if a statement such as "There are plenty of people who can replace you" is detected, a message will be generated stating, "This statement is oppressive and may cause great stress to the recipient, so it constitutes power harassment."

[1440] Alerting and Notifications

[1441] The generated alert message is sent back from the server to the device. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[1442] User response

[1443] Users receive an alert message and can reevaluate their own comments and writings. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails and chats, they can edit the content before sending it. This helps prevent power harassment at an early stage and maintain a healthy working environment.

[1444] Specific examples

[1445] Example 1: Audio alert during a meeting

[1446] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[1447] 2. The preprocessed text data is sent to the server.

[1448] 3. The server analyzes the data and detects the statement, "You're slow at work."

[1449] 4. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is non-constructive and constitutes power harassment."

[1450] 5. An alert message is sent to the terminal to notify the user.

[1451] 6. The user checks the alert and reviews their comments.

[1452] Example 2: Email alert

[1453] 1. The device retrieves the text of the email being composed in real time.

[1454] 2. The preprocessed text data is sent to the server.

[1455] 3. The server analyzes the data and detects a statement such as, "Making mistakes like this is just your personality, so it can't be helped."

[1456] 4. The server determines that this sentence is power harassment and generates an alert message such as, "This sentence has been recognized as a personal attack and constitutes power harassment."

[1457] 5. An alert message is sent to the terminal to notify the user.

[1458] 6. The user checks the alert and corrects the email content.

[1459] Prompt Sentence Examples

[1460] "Evaluate whether the following statement constitutes power harassment and explain why: 'Your work is completely worthless.'"

[1461] This invention allows users to recognize problems in their own comments and writings in real time and have the opportunity to correct them, which is expected to prevent power harassment and improve the working environment.

[1462] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1463] Step 1:

[1464] The device accepts voice and text input in real time. Specifically, the device's microphone is used to record what is said during a meeting. Text data is also obtained via email or chat applications. The input data can be voice or text data.

[1465] Step 2:

[1466] The device converts voice data into text data using speech recognition technology. Specifically, it analyzes the recorded voice using the Google Recognition API and outputs it as text data. The input here is voice data, and the output is the converted text data.

[1467] Step 3:

[1468] The text data received by the terminal is preprocessed using a grammar analysis tool. Specifically, grammar analysis tools such as SpaCy and NLTK are used to tokenize and parse the text, which then formats it into a format suitable for analysis. The input is raw text data, and the output is text data that has undergone preprocessing.

[1469] Step 4:

[1470] The terminal sends the preprocessed text data to the server. Specifically, it uses an HTTP POST request to send data to the server in JSON format. The input is the preprocessed text data, and the output is the result sent to the server.

[1471] Step 5:

[1472] The server analyzes the received data using a generative artificial intelligence model. Specifically, it uses GPT-4 installed on the server to evaluate whether words and phrases in the data constitute power harassment. The input is preprocessed text data, and the output is the analysis results.

[1473] Step 6:

[1474] The server generates an alert message based on the analysis results. Specifically, for detected cases of possible power harassment, it creates a message that explains in detail why the case constitutes power harassment. The input is the analysis results, and the output is the alert message.

[1475] Step 7:

[1476] The server sends the generated alert message to the terminal again. Specifically, it sends the message to the terminal using an HTTP POST request. The input is the alert message, and the output is the result sent to the terminal.

[1477] Step 8:

[1478] The device notifies the user of the alert message received. Specifically, it notifies the user using a push notification, a popup display, or an email. The input is the alert message, and the output is the notification sent to the user.

[1479] Step 9:

[1480] Users receive an alert message and reevaluate their statements or writing. Specifically, if they make an overbearing remark during a meeting, they reflect on their remarks and pay more attention to what they say in the future. In the case of emails or chats, they correct the content before sending. The input is the alert message, and the output is the corrected remarks or writing.

[1481] (Application example 1)

[1482] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1483] In physical stores, there is a need to detect power harassment that may occur during communication between store staff and customers in real time and deal with it on the spot.An effective system is needed to reduce customer dissatisfaction and store staff stress and maintain a healthy working environment.

[1484] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1485] In this invention, the server includes: means for a terminal equipped with a generative artificial intelligence model to accept voice and text input in real time; means for sending preprocessed data to the server, which analyzes and compares the data using the generative artificial intelligence model; means for generating an alert message and sending it to the terminal to notify the user when possible power harassment is detected; and means for collecting voice data using a device worn by store clerks and analyzing it in real time based on the generative artificial intelligence model. This allows conversations between store clerks and customers in a physical store to be monitored in real time, making it possible to detect and address signs of power harassment early on.

[1486] A "generative artificial intelligence model" is an artificial intelligence algorithm that can learn from huge data sets and understand and generate new input data.

[1487] A "terminal" is a device that a user operates to input or output data, and includes smartphones, computers, and the like.

[1488] "Server" means a computer system that receives and processes data over a network and transmits the analysis results.

[1489] "Power harassment" refers to the act of using one's position in the workplace to inflict mental or physical pain on another.

[1490] An "alert message" is notification information that alerts the user when a specific condition is met.

[1491] The "device" refers to the hardware worn by the store clerk for collecting voice data.

[1492] The present invention is a system that monitors voice communications between store clerks and customers in real time, and quickly detects and alerts users when statements that may constitute power harassment. The specific system configuration and processing procedures for implementing the present invention are described below.

[1493] Hardware configuration

[1494] This system uses the following hardware:

[1495] Device: Refers to smart glasses or smartphones for voice input and real-time notifications.

[1496] Server: A high-performance computer system that runs generative artificial intelligence models.

[1497] Device: Smart glasses worn by store staff and a microphone for collecting audio data.

[1498] Software configuration

[1499] This system consists of the following software:

[1500] Speech Recognition Software: Software for converting speech to text, such as the Google Cloud Speech-to-Text API.

[1501] Generative artificial intelligence model: An AI model for detecting power harassment using Transformer-based models such as BERT.

[1502] Communication Protocol: SSL / TLS protocol for secure data communication.

[1503] Data processing and calculation procedures

[1504] The device records conversations between store staff and customers in real time, and the recorded audio data is converted into text data by voice recognition software within the smart glasses, which is then pre-processed and sent to the server via a secure communication protocol.

[1505] The server analyzes the received text data using a generative artificial intelligence model (such as BERT). This analysis evaluates whether the phrases and words in the data constitute power harassment. If the evaluation results indicate the possibility of power harassment, the server generates an alert message pointing out the specific problem.

[1506] The generated alert message is sent to the terminal and notified to the user (store clerk) as a push notification or a pop-up display on the smart glasses or smartphone, allowing the store clerk to recognize the problematic part of their own statement in real time and correct their response appropriately.

[1507] Specific examples

[1508] For example, if a store clerk at a brick-and-mortar convenience store says to a customer, "This is simple, hurry up, the next person is waiting," the system will analyze the remark in real time and display a warning on the smart glasses saying, "This remark is overbearing and constitutes power harassment." This will enable the store clerk to immediately review their remark and correct it to respond appropriately to the customer.

[1509] Prompt Sentence Examples

[1510] TXT

[1511] <audio_file.wav>

[1512] -store_interaction

[1513] -convert_audio_to_text

[1514] -check_for_power_harassment

[1515] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1516] Step 1:

[1517] The device records conversations between store staff and customers in real time, using the microphone built into the smart glasses to collect audio data.

[1518] Input: Voice conversation between store clerk and customer

[1519] Output: Audio data file (.wav, etc.)

[1520] Step 2:

[1521] The device converts the collected voice data into text data using the Google Cloud Speech-to-Text API, and the converted text is preprocessed.

[1522] Input: Audio data file

[1523] Output: Text data

[1524] Step 3:

[1525] The terminal transmits the preprocessed text data to the server using a secure communication protocol (SSL / TLS).

[1526] Input: Preprocessed text data

[1527] Output: Sending text data

[1528] Step 4:

[1529] The server analyzes the received text data using a generative AI model (such as BERT), which analyzes words and phrases in the input text data and performs data calculations to evaluate whether they constitute power harassment.

[1530] Input: Transmitted text data

[1531] Output: Analysis results (power harassment judgment)

[1532] Step 5:

[1533] If the server detects possible power harassment, it generates an alert message explaining why a particular statement or sentence constitutes power harassment.

[1534] Input: Analysis result (power harassment judgment)

[1535] Output: Alert message

[1536] Step 6:

[1537] The server then sends the generated alert message back to the device, again using a secure protocol.

[1538] Input: Alert message

[1539] Output: Sending an alert message

[1540] Step 7:

[1541] The device notifies the user (store clerk) of the received alert message via a pop-up display on the smart glasses display, a push notification, or other means.

[1542] Input: Alert message

[1543] Output: User notification

[1544] Step 8:

[1545] The user checks the alert message, reviews their comments, and, if necessary, corrects the content of their comments and takes appropriate action.

[1546] Input: Alert message

[1547] Output: User behavior modification

[1548] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1549] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect the possibility of power harassment in real time and quickly issue an alert to the user. A specific embodiment of this system is described below.

[1550] Voice and text input and preprocessing

[1551] The device (smartphone or personal computer) accepts comments made during meetings, everyday communication (voice input), and text input (text input) for emails and chats in real time. Voice data is converted into text data using the device's voice recognition technology, and text data is preprocessed using grammar analysis tools to make it easier to analyze. An emotion engine is also used, and user emotion data is simultaneously generated from the voice and text.

[1552] Data transmission and analysis

[1553] The preprocessed text data and generated emotion data are sent to a server via the internet. The server's generative AI model analyzes the received data. This AI model has previously studied a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[1554] Matching and alert message generation

[1555] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[1556] Alerting and Notifications

[1557] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[1558] User response

[1559] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[1560] Specific examples

[1561] Example 1: Voice alerts and emotion recognition during meetings

[1562] 1. The device records what is said during the meeting, converts it into text using voice recognition, and generates emotional data using an emotion engine.

[1563] 2. The preprocessed text data and emotion data are sent to the server.

[1564] 3. The server analyzes the data and detects the statement, "You're slow at work."

[1565] 4. The server determines that this comment constitutes power harassment and, taking into account the emotional data, generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotion has been recognized as irritation."

[1566] 5. An alert message is sent to the terminal to notify the user.

[1567] 6. The user checks the alert and reviews their comments.

[1568] Example 2: Email alerts and emotion recognition

[1569] 1. The device obtains the text of the email being composed in real time and generates emotion data using the emotion engine.

[1570] 2. The preprocessed text data and emotion data are sent to the server.

[1571] 3. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes the emotional data as dissatisfaction.

[1572] 4. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[1573] 5. An alert message is sent to the terminal to notify the user.

[1574] 6. The user checks the alert and corrects the email content.

[1575] This invention allows users to recognize problems with their own statements, writing, and emotions in real time and have the opportunity to correct them, thereby more effectively preventing power harassment and improving the working environment.

[1576] The processing flow will be explained below.

[1577] Step 1:

[1578] The terminal accepts voice and text input from the user.

[1579] In the case of voice input, speech during the meeting is recorded using a microphone, and the recorded audio is stored in the device.

[1580] In the case of text input, the text data entered by the user via a keyboard or touch screen is acquired.

[1581] The input data is temporarily stored in a buffer.

[1582] Step 2:

[1583] Preprocess the data acquired by the device.

[1584] In the case of voice data, it is converted into text in real time using the device's voice recognition engine.

[1585] For text data, natural language processing tools are used to analyze the structure of the text and complete punctuation as needed.

[1586] The converted text data is formatted into a format that is easy to parse.

[1587] Step 3:

[1588] The device generates emotion data.

[1589] In the case of audio data, the emotion engine analyzes it to extract the tone, speed, and emotional expression of the voice.

[1590] For text data, the emotion engine analyzes the context and keywords of the text to generate an emotion classification (e.g., anger, frustration, joy, etc.).

[1591] Step 4:

[1592] The terminal transmits the preprocessed text data and emotion data to the server.

[1593] The endpoints use SSL / TLS encryption to ensure secure data transfer.

[1594] Executes the appropriate API calls to send the data to the server.

[1595] Step 5:

[1596] The server parses the received data.

[1597] It receives text data and emotion data and passes it to its own data analysis engine.

[1598] Using a generative artificial intelligence model, words and phrases within text are identified and analyzed along with sentiment data.

[1599] Past cases of power harassment are referenced from the database and cross-checked.

[1600] Step 6:

[1601] The server evaluates the possibility of power harassment.

[1602] Based on the matching results, the entered remarks or sentences are scored to determine whether they constitute power harassment.

[1603] If the score is above the set threshold, it is determined to be a possible case of power harassment.

[1604] Step 7:

[1605] The server generates an alert message.

[1606] If possible power harassment is detected, an alert message is generated.

[1607] It describes specific problems in the text and the reasons for them. It also includes feedback based on emotional data, such as, "This statement is overbearing and may cause stress to the recipient. Furthermore, your emotion is recognized as anger."

[1608] Step 8:

[1609] The server generates an alert message and sends it to the terminal.

[1610] An alert message is sent to the terminal using secure communications means.

[1611] Send data to the device's receiving API.

[1612] Step 9:

[1613] The device displays an alert notification to the user.

[1614] Received alert messages are displayed as push notifications or pop-up notifications.

[1615] Place notifications in appropriate locations on the interface so that they are immediately accessible to the user.

[1616] Step 10:

[1617] The user checks the alert message and takes action.

[1618] Check the alert message and review your statements, writing, and emotions.

[1619] For example, take action such as revising the content of the email or improving the content of the statement.

[1620] As a result of this trend, the system not only detects potential power harassment in real time and provides users with an opportunity to respond quickly, but also takes into account the user's emotions, promoting more effective communication improvements.

[1621] Example 2

[1622] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1623] In today's work environment, mental stress and reduced motivation due to power harassment have become serious problems. Real-time monitoring and specific alerts are necessary to prevent this from happening, but conventional systems have difficulty in detecting power harassment in real time and generating appropriate alerts. Therefore, there is a need to establish a system that can detect power harassment in real time, including recognizing emotions, and provide appropriate feedback.

[1624] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1625] In this invention, the server includes means for a terminal to accept voice and text input in real time, means for converting voice data into text data using voice recognition technology, means for generating user emotion data using an emotion engine, means for transmitting the preprocessed text data and emotion data to the server, means for the server to analyze the preprocessed data using a generative artificial intelligence model and evaluate whether it constitutes power harassment, and means for generating an alert message and transmitting it to the terminal to notify the user if the possibility of power harassment is detected. This allows users to review their own comments and text in real time and receive feedback based on the emotion data.

[1626] A "terminal" is a device (such as a smartphone or personal computer) that allows a user to input voice or text.

[1627] "Speech recognition technology" is a technology that converts voice data into text data.

[1628] An "emotion engine" is a software technology that analyzes and classifies a user's emotions from input voice or text.

[1629] "Preprocessing" is a series of operations to prepare input data in a form that is easy to analyze.

[1630] A "generative artificial intelligence model" is an artificial intelligence system that learns from huge amounts of data and analyzes and evaluates input data.

[1631] "Power harassment" is the act of using one's position or status in the workplace to inflict mental or physical pain on others.

[1632] An "alert message" is a warning message that is generated and notified to the user when a specific condition (in this case, the possibility of power harassment) is detected.

[1633] A "server" is a central processing unit that receives and analyzes data, and generates and sends alert messages.

[1634] The present invention is a system that uses a generative artificial intelligence model and an emotion engine to detect potential power harassment in real time and quickly issue an alert to the user.

[1635] Voice and text input and preprocessing

[1636] Devices (smartphones or personal computers) accept comments made during meetings and everyday communication (voice input), as well as text input (text input) for emails and chats in real time. Voice data is converted into text data using voice recognition technology (such as Google Speech-to-Text API). Text data is preprocessed into an easy-to-analyze form using a grammar analysis tool (such as SpaCy). In addition, an emotion engine (such as Amazon Comprehend) is used in combination to generate user emotion data from the voice and text.

[1637] Data transmission and analysis

[1638] The preprocessed text data and generated emotion data are sent to a server via the internet. The server runs a generative AI model (such as GPT-4) to analyze the received data. The generative AI model has learned from a vast number of cases of power harassment, and detects words and phrases in the input data and evaluates whether they constitute power harassment, along with the emotion data.

[1639] Matching and alert message generation

[1640] If the received data is judged to be potentially power harassment, the server generates an alert message. This alert message will contain specific reasons why a particular statement or sentence constitutes power harassment. It will also include feedback based on emotional data. For example, if a statement such as "There are plenty of people who can replace you" is detected and the user's emotion is classified as anger or dissatisfaction, a message will be generated stating, "This statement is oppressive and may cause stress to the recipient. Furthermore, your emotion has been recognized as anger."

[1641] Alerting and Notifications

[1642] The generated alert message is sent back to the device from the server. The device receives it and notifies the user appropriately. Possible notification methods include push notification, pop-up display, and email. The user can check this notification and review their own comments and writings.

[1643] User response

[1644] Users receive an alert message and can reevaluate their own comments, writing, and even their emotions. For example, if they make an overbearing remark during a meeting, they can reflect on it and be more careful about what they say in the future. Furthermore, in the case of emails or chats, they can edit the content before sending it. This helps prevent power harassment early on and maintain a healthy working environment.

[1645] Specific examples

[1646] Example 1: Voice alerts and emotion recognition during meetings

[1647] 1. The device records what is said during the meeting and converts it into text using voice recognition.

[1648] 2. The device generates emotion data using an emotion engine.

[1649] 3. The preprocessed text data and emotion data are sent to the server.

[1650] 4. The server analyzes the data and detects the statement, "You're slow at work."

[1651] 5. The server determines that this comment constitutes power harassment and generates an alert message stating, "This comment is unconstructive and constitutes power harassment. Furthermore, your emotions have been recognized as irritation."

[1652] 6. An alert message is sent to the terminal to notify the user.

[1653] 7. The user checks the alert and reviews their comments.

[1654] Example 2: Email alerts and emotion recognition

[1655] 1. The device retrieves the text of the email being composed in real time.

[1656] 2. The device generates emotion data using an emotion engine.

[1657] 3. The preprocessed text data and emotion data are sent to the server.

[1658] 4. The server analyzes the data and detects the statement, "Making mistakes like this is just your personality, so it can't be helped," and recognizes it as dissatisfaction.

[1659] 5. The server determines that this sentence constitutes power harassment and generates an alert message stating, "This sentence has been recognized as a personal attack and constitutes power harassment. Furthermore, your feelings have been recognized as dissatisfaction."

[1660] 6. An alert message is sent to the terminal to notify the user.

[1661] 7. The user checks the alert and corrects the email content.

[1662] Prompt Sentence Examples

[1663] The following prompt sentences are input to the generative AI model to analyze the data and generate an alert message:

[1664] Please rate whether the following statements constitute power harassment:

[1665] Saying: "You're slow at work"

[1666] Emotion Data: {"User Emotion": "Irritated"}

[1667] Additionally, generate an alert message that includes the reasons and feedback on why this statement is problematic.

[1668] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1669] Step 1:

[1670] The device collects user conversations and everyday communication as voice data. Specifically, it records voice using the microphone of a smartphone or personal computer. The input is voice data, which is collected in real time. The output is the voice data before conversion.

[1671] Step 2:

[1672] The device converts the collected voice data into text data using voice recognition technology. The technology used here is a voice recognition API (e.g., Google Speech-to-Text). The input is voice data, and the output is converted text data. Specifically, the voice is converted sequentially into recognizable text.

[1673] Step 3:

[1674] The device collects text data as sentence data. For example, emails and chat messages entered by users in real time are input here. The output is text data before preprocessing. The input strings are captured sequentially and prepared as data for analysis.

[1675] Step 4:

[1676] The terminal preprocesses the text data. Using a grammar analysis tool (e.g., SpaCy), it appropriately processes punctuation and special characters and converts them into a format that is easy to analyze. The input is text data before preprocessing, and the output is preprocessed text data. Specifically, the analysis tool is used to remove unnecessary elements and organize the data grammatically.

[1677] Step 5:

[1678] The device generates emotion data using an emotion engine. The technology used here is an emotion recognition engine (e.g., Amazon Comprehend). The input is preprocessed text data, and the output is emotion data. Specifically, the emotion engine analyzes emotions in the text and assigns labels such as anger or joy.

[1679] Step 6:

[1680] The terminal transmits the preprocessed text data and generated emotion data to a server via the Internet. The input is the preprocessed text data and emotion data, and the output is the transmitted data. Specifically, the terminal transmits the data to the server via an Internet connection.

[1681] Step 7:

[1682] The server analyzes the received text data and emotion data. The model used is a generative artificial intelligence model (e.g., GPT-4). The input is the transmitted data, and the output is the analysis results. Specifically, the AI ​​model detects specific words and phrases in the data and evaluates them along with the emotion data.

[1683] Step 8:

[1684] The server detects power harassment based on the analysis results. The input is the analysis results, and the output is an assessment of the possibility of power harassment. Specifically, the detection algorithm compares the analysis results with the assessment criteria to identify harmful remarks and behavior.

[1685] Step 9:

[1686] The server generates an alert message when there is a possibility of power harassment. This message includes feedback based on the specific problem and emotional data. The input is the evaluation result of power harassment, and the output is the alert message. Specifically, the alert generation algorithm creates a warning message based on the evaluation result.

[1687] Step 10:

[1688] The server sends the generated alert message to the terminal. The input is the alert message, and the output is the sent message. As a specific operation, the alert is delivered to the user's terminal via the message sending function.

[1689] Step 11:

[1690] The device notifies the user of the received alert message. Possible notification methods include push notification, pop-up display, and email. The input is the received alert message, and the output is the notification to the user. Specifically, the notification system displays the alert.

[1691] Step 12:

[1692] The user checks the alert message and reevaluates their own statements and writing. For example, if they make an overbearing statement during a meeting, they can reflect on it and be more careful about what they say in the future. The input is the alert message, and the output is the corrected statement or writing. The specific operation is for the user to review the content and make corrections.

[1693] (Application example 2)

[1694] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1695] Power harassment is becoming a serious problem in modern workplaces, causing deterioration in employees' mental health and the work environment. Effective measures to prevent power harassment, particularly in communication between superiors and subordinates, are needed. However, a system that can detect power harassment in real time and respond quickly has not yet been established. In addition, advanced analysis that takes into account not only voice and text data analysis but also emotional changes is required. However, conventional methods are often inadequate.

[1696] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1697] In this invention, the server includes a means for a terminal equipped with a generative artificial intelligence model to receive voice and text input in real time, a means for transmitting preprocessed data to the server, which then analyzes and collates the data using the generative artificial intelligence model, a means for generating an alert message and transmitting it to the terminal to notify the user when possible power harassment is detected, a means for recording conversations in real time using smart glasses and generating emotion data, and a means for sending push notifications using a smartphone, thereby enabling the detection of power harassment and rapid notification.

[1698] A "generative artificial intelligence model" is an AI technology that learns from vast amounts of data and generates and analyzes natural, human-like sentences and speech.

[1699] A "terminal" is an electronic device capable of accepting, processing, and displaying voice and text input in real time.

[1700] "Preprocessing" is the process of converting input voice data or text data into a form that is easier to analyze.

[1701] A "server" is a computer system installed to process data over a network.

[1702] "Matching" is the process of comparing the analyzed data with existing databases or standards.

[1703] An "alert message" is a warning message that notifies the user when a possible case of power harassment is detected.

[1704] "Smart glasses" are wearable devices that have built-in AR-compatible displays, cameras, and microphones, and can display information and collect data in real time.

[1705] "Real-time" means that processing and data acquisition are immediate.

[1706] "Emotional data" is data that indicates the user's emotional state, generated from analyzed speech or text.

[1707] "Push notification" is a technology that sends instant notifications from a server to a device.

[1708] The present invention is an advanced system for real-time detection and notification of workplace power harassment, utilizing a generative artificial intelligence model and an emotion engine. The system accepts and analyzes voice and text input in real time, and alerts the user if a risk of power harassment is detected. A specific embodiment of the system is described in detail below.

[1709] Hardware and software used

[1710] Hardware

[1711] Smart glasses: AR-enabled, equipped with a camera and microphone

[1712] Smartphone: iOS / Android compatible

[1713] software

[1714] Speech recognition engine: Google Cloud Speech-to-Text

[1715] Emotion engine: IBM Watson Tone Analyzer

[1716] Generative AI model: OpenAI's GPT series

[1717] Communications technology: Firebase, AWS

[1718] Push notification system: Firebase Cloud Messaging

[1719] System Operation

[1720] 1. Data Collection:

[1721] The server acquires audio and video data from the smart glasses and smartphone in real time.

[1722] This allows meetings and daily communications to be recorded.

[1723] 2. Pretreatment:

[1724] The server converts the voice data into text data using Google Cloud Speech-to-Text.

[1725] The text data is pre-processed to make it easier to interpret and analyze.

[1726] 3. Emotion data generation:

[1727] The server uses IBM Watson Tone Analyzer to generate emotion data from text data.

[1728] Emotional data indicates the user's emotional state (eg, anger, frustration, happiness).

[1729] 4. Data Analysis:

[1730] The server performs analysis using a generative artificial intelligence model (GPT series) based on the preprocessed text data and emotion data.

[1731] The server detects specific words and phrases in the input data and compares them with emotional data to assess the risk of power harassment.

[1732] 5. Alert Generation:

[1733] The server generates an alert message if a risk of power harassment is detected.

[1734] The alert message includes feedback on specific utterances or sentence problems and corresponding emotional data.

[1735] For example, "This statement is overbearing and may cause stress to the other person. Furthermore, your emotion is being perceived as anger."

[1736] 6. Notice:

[1737] The server sends the generated alert message to the smart glasses and the smartphone.

[1738] Users can view alerts via a pop-up display on their smart glasses or a push notification on their smartphone.

[1739] Specific examples

[1740] Example 1: Real-time detection during a meeting

[1741] Say: "How could you make such a basic mistake?"

[1742] Emotion data: Anger

[1743] Alert message: "This statement puts pressure on the other person and poses a risk of power harassment. We have also confirmed that your emotion is anger."

[1744] Example 2: Real-time chat monitoring

[1745] Say: "This project has been delayed entirely because of you."

[1746] Emotional data: Dissatisfied

[1747] Alert message: "This comment has been perceived as a personal attack and constitutes power harassment. Furthermore, your feelings of dissatisfaction have been confirmed."

[1748] Prompt Sentence Examples

[1749] Analyze the following statements and determine whether they pose a risk of power harassment. Also, consider the emotional data associated with the statements. Statement: "You're really useless. How many times do I have to tell you?" Emotional data: Anger

[1750] In this way, the present invention has the ability to detect the risk of power harassment in real time and promptly notify the user, thereby promoting a healthy work environment and protecting the mental health of employees.

[1751] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1752] Step 1:

[1753] The terminal collects conversation and audio data in real time through smart glasses and smartphones. The input is audio data and video data. The output is raw audio and video data collected in real time.

[1754] Step 2:

[1755] The voice data collected by the device is converted into text data using Google Cloud Speech-to-Text. The input is voice data. Data processing involves converting it into text data using voice recognition technology. The output is the converted text data.

[1756] Step 3:

[1757] The server receives the preprocessed text data from the terminal. Secure communication technologies such as AWS and Firebase are used for secure data transfer. The input is the converted text data. The output is the received text data.

[1758] Step 4:

[1759] Emotion data is generated from text data using IBM Watson Tone Analyzer. The input is the received text data. Data processing involves extracting and classifying the user's emotional state using emotion analysis technology. The output is the generated emotion data.

[1760] Step 5:

[1761] The server integrates the preprocessed text data and emotion data and performs analysis using a generative artificial intelligence model (OpenAI's GPT series). The input is the integrated text data and emotion data. Data calculation involves comparing the data with existing cases of power harassment and assessing the risk. The output is the analysis results.

[1762] Step 6:

[1763] The server evaluates the possibility of power harassment based on the analysis results and generates an alert message. The input is the analysis results. Data processing involves creating an alert message based on specific remarks and problems with the sentences, as well as feedback on emotional data. The output is the generated alert message.

[1764] Step 7:

[1765] The server uses Firebase Cloud Messaging to send the generated alert message to the devices (smart glasses and smartphones). The input is the generated alert message. The output is the sent alert message.

[1766] Step 8:

[1767] The user confirms the alert message through the device (a pop-up display on the smart glasses or a push notification on the smartphone). The input is the received alert message. The output is the user's confirmed alert message.

[1768] Step 9:

[1769] The user reevaluates their own statements and attitudes based on the alert content and makes corrections if necessary. The input is the confirmed alert message. The output is the corrected statements and attitudes.

[1770] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1771] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1772] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1773] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1774] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1775] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1776] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1777] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1778] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1779] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1780] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1781] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1782] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1783] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1784] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1785] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1786] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1787] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1788] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1789] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1790] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1791] The following is further disclosed regarding the above embodiment.

[1792] (Claim 1)

[1793] A device equipped with a generative artificial intelligence model has a means for accepting voice and text input in real time;

[1794] A means for transmitting the preprocessed data to a server, and for the server to analyze and compare the data using a generative artificial intelligence model;

[1795] means for generating an alert message and transmitting it to the terminal to notify the user when a possibility of power harassment is detected;

[1796] A system including:

[1797] (Claim 2)

[1798] 10. The system of claim 1, wherein the preprocessed data is transmitted using secure communication means.

[1799] (Claim 3)

[1800] 2. The system according to claim 1, wherein the generated alert message is a message that explains a specific problem with the statement or text.

[1801] "Example 1"

[1802] (Claim 1)

[1803] a means for the terminal to accept voice and text input in real time;

[1804] A means for converting the received voice into text data using voice recognition technology;

[1805] means for preprocessing the received text data using a grammar analysis tool;

[1806] means for transmitting the preprocessed data to a server;

[1807] A means for the server to analyze the received data using a generative artificial intelligence model and evaluate the possibility of power harassment;

[1808] means for generating an alert message when a potential power harassment is detected;

[1809] means for transmitting the generated alert message to a terminal to notify a user;

[1810] A system including:

[1811] (Claim 2)

[1812] 10. The system of claim 1, wherein the preprocessed data is transmitted using secure communication means.

[1813] (Claim 3)

[1814] 2. The system according to claim 1, wherein the generated alert message is a message that explains a specific problem with the statement or text.

[1815] "Application Example 1"

[1816] (Claim 1)

[1817] A device equipped with a generative artificial intelligence model has a means for accepting voice and text input in real time;

[1818] A means for transmitting the preprocessed data to a server, and for the server to analyze and compare the data using a generative artificial intelligence model;

[1819] means for generating an alert message and transmitting it to the terminal to notify the user when a possibility of power harassment is detected;

[1820] A means for collecting voice data using a device worn by a store clerk and analyzing the data in real time based on a generative artificial intelligence model;

[1821] A system including:

[1822] (Claim 2)

[1823] 10. The system of claim 1, wherein the preprocessed data is transmitted using secure communication means.

[1824] (Claim 3)

[1825] The system according to claim 1, wherein the generated alert message is a message explaining a specific problem with the statement or text and is displayed on the device.

[1826] "Example 2: Combining Emotion Engines"

[1827] (Claim 1)

[1828] a means for the terminal to accept voice and text input in real time;

[1829] A means for converting voice data into text data using voice recognition technology;

[1830] means for generating user emotion data using an emotion engine;

[1831] means for transmitting the preprocessed text data and emotion data to a server;

[1832] A means for the server to analyze the preprocessed data using a generative artificial intelligence model and evaluate whether it constitutes power harassment;

[1833] means for generating an alert message and transmitting it to the terminal to notify the user when a possibility of power harassment is detected;

[1834] A system including:

[1835] (Claim 2)

[1836] 10. The system of claim 1, wherein the preprocessed data is transmitted using secure communication means.

[1837] (Claim 3)

[1838] 2. The system of claim 1, wherein the generated alert message is a message that explains a specific problem with the statement or text and includes feedback based on the user's emotional state.

[1839] "Application example 2 when combining emotion engines"

[1840] (Claim 1)

[1841] A device equipped with a generative artificial intelligence model has a means for accepting voice and text input in real time;

[1842] A means for transmitting the preprocessed data to a server, and for the server to analyze and compare the data using a generative artificial intelligence model;

[1843] means for generating an alert message and transmitting it to the terminal to notify the user when a possibility of power harassment is detected;

[1844] a means for recording conversations in real time using smart glasses to generate emotion data;

[1845] A means of sending push notifications using a smartphone;

[1846] A system including:

[1847] (Claim 2)

[1848] 10. The system of claim 1, wherein the preprocessed data is transmitted using secure communication means.

[1849] (Claim 3)

[1850] 2. The system according to claim 1, wherein the generated alert message is a message that explains specific problems with the utterance or sentence and provides feedback on emotional data based on the problems. [Explanation of symbols]

[1851] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A device equipped with a generative artificial intelligence model has a means for accepting voice and text input in real time; A means for transmitting the preprocessed data to a server, and for the server to analyze and compare the data using a generative artificial intelligence model; means for generating an alert message and transmitting it to the terminal to notify the user when a possibility of power harassment is detected; A system including:

2. 10. The system of claim 1, wherein the preprocessed data is transmitted using secure communication means.

3. 2. The system according to claim 1, wherein the generated alert message is a message that explains a specific problem with the statement or text.

Images