System

Abstract

A system is provided.SOLUTION: A system comprising: means for storing inputs received from user terminals; means for transmitting the inputs received from the user terminals to a server; means for analyzing the inputs at the server and automatically generating a failure report by a AI; means for transmitting the generated failure report to the user terminals; and means for sharing the failure report confirmed by the user with a technical department.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem in that it is difficult to respond quickly and accurately to complaints about communication outages from corporate customers. In particular, sales representatives and inquiry desk staff often lack technical expertise, which makes it difficult to smoothly share information with the technical department, which can delay problem resolution. In addition, there is no established method for accurately collecting and sharing information about malfunctions and reports from customers, which often results in a decline in customer satisfaction. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. First, it provides a means for storing input data received from a user terminal and then transmitting that data to a server. Next, the server has a means for analyzing the received data and automatically generating a problem report using AI. The generated report is sent to the user terminal, confirmed by the user, and shared with the technical department. In addition, it also includes a means for comparing the report with past problem data and a case database and adding any necessary supplementary information to the report. Finally, it realizes a system that allows efficient and accurate problem reporting by receiving input data in the form of checkboxes on the user terminal, without the user having to perform tedious input work.

[0006] "User terminal" refers to an input device such as a computer or smartphone used by sales representatives or inquiry desk staff.

[0007] "Server" refers to the central processing unit that receives and analyzes data sent from user terminals and generates reports using AI.

[0008] "Input data" refers to information about communication failures and the contents of selected check boxes provided by users via their terminals.

[0009] A "check box" refers to a component on a graphical user interface (GUI) that allows a user to select a particular item.

[0010] "Outage Report" refers to a document generated by AI that contains details of a communications outage and information needed to resolve it.

[0011] "Technical department" refers to the department of technical professionals responsible for analyzing and resolving communication problems.

[0012] "AI" refers to software systems that use artificial intelligence technology to perform data analysis and natural language generation.

[0013] "Past outage data" refers to records and database information regarding past communication outages.

[0014] A "case database" refers to a database that records past incidents of problems and how they were dealt with.

[0015] "Supplementary information" refers to past case data and related technical information that is added to the AI-generated report. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that responds quickly and accurately to complaints about communication failures from corporate customers, and it allows sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with a server, the system of this invention automatically generates a failure report from simple input in the form of check boxes, and makes it possible to quickly share the report with the technical department.

[0038] A natural language description of the program's processing

[0039] 1. Input from the user's device

[0040] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, specifically by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods."

[0041] 2. Send and save input data

[0042] The information of the checkboxes selected by the user is temporarily stored on the device and then sent to the server in real time. This transmission uses a secure communication protocol, so the safety of the data is ensured.

[0043] 3. Data reception and analysis on the server

[0044] The server receives the data sent from the device. The received data is first stored in temporary storage and then converted into a format that can be analyzed by AI. In this analysis process, the data is classified by the contents of the checkboxes and appropriate keywords and phrases are generated.

[0045] 4. AI-based fault report generation

[0046] The converted data is provided to an AI model installed on the server. The AI ​​generates an outage report by referencing the received data, past outage data, and a case database. For example, the generated text might read, "According to user reports, communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0047] 5. Send the generated report to the user's device

[0048] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[0049] 6. User confirmation of report

[0050] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[0051] 7. Information sharing with the technical department

[0052] Once the user has confirmed the report, it is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the problem.

[0053] Specific examples of operation

[0054] For example, if a user (sales representative) receives information that communication was disrupted in a certain area during a specific time period, they can select check boxes such as "communication disrupted," "specific area only," or "specific time period" from the GUI on their device. This input data is sent to the server in real time, which analyzes the data and generates a specific failure report using AI. The report is then sent to the user's device, where the user checks the contents and, if there are no problems, shares it with the technical department.

[0055] In this way, the system of the present invention allows users to easily generate accurate trouble reports without having to write technical documents, and enables quick and accurate information sharing with the technical department, thereby aiming to improve customer satisfaction with regard to communication troubles.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, for example, by selecting check boxes such as "communication was interrupted," "only in a specific area," and "specific time period."

[0059] Step 2:

[0060] The device temporarily stores the information of the checkboxes selected by the user, and this data is stored in temporary memory or local storage for subsequent processing.

[0061] Step 3:

[0062] The device sends the stored data to the server in real time using secure communication protocols such as HTTPS to prevent data leakage and tampering.

[0063] Step 4:

[0064] The server receives the data sent from the device and stores it in temporary storage.

[0065] Step 5:

[0066] The server parses the incoming data and converts it into a format that the AI ​​can understand. This parsing process involves converting checkbox selections into keywords and phrases.

[0067] Step 6:

[0068] The server provides the analyzed data to the AI ​​model, which then compares the received data with past fault data and a case database.

[0069] Step 7:

[0070] The server's AI generates a problem report based on the provided data, such as "Users have reported that communication has been interrupted in a specific area and time period. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0071] Step 8:

[0072] The server temporarily stores the generated failure report and prepares it for transmission to the user terminal.

[0073] Step 9:

[0074] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[0075] Step 10:

[0076] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[0077] Step 11:

[0078] The user retransmits the confirmed problem report from the terminal to the server and shares it with the technical department, and the server notifies the technical department of the shared report.

[0079] Step 12:

[0080] The technical department will promptly analyze and resolve the problem based on the received report, and will then provide feedback on the results of the response to the user.

[0081] Example 1

[0082] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0083] In conventional communication outage response systems, it is difficult for non-technical sales representatives and customer service representatives to properly convey information about communication outages to the technical department, resulting in delays and mistakes in the response. Furthermore, to reduce the time and effort required to manually create outage reports, automation is required, but creating professional-looking reports is not easy. Therefore, a system is needed that allows users to easily enter information about communication outages, automatically and quickly generate outage reports based on that information, and reliably share them with the technical department.

[0084] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0085] In this invention, the server includes means for saving input data received from a user terminal, means for analyzing the input data received from the user terminal and automatically generating a problem report based on a prompt sentence using a generative AI model, means for sending the generated problem report to the user terminal, and means for comparing past problem data and a case database and adding necessary supplemental information to the generated report. This allows even non-technical personnel to easily input information about communication problems, automatically and quickly create problem reports that meet specifications, and share the information with the technical department.

[0086] A "user terminal" is a device that provides an interface for a user to use, and is used to input information about a communication failure and transmit it to a server.

[0087] The "server" is a central processing system that receives, analyzes, and stores data sent from the user terminal, generates a fault report using a generative AI model, and sends it back to the user terminal.

[0088] "Input data" is information sent from a user terminal to a server, including details about a communication failure.

[0089] A "generative AI model" is an artificial intelligence model that automatically generates a fault report based on data received from the user's device, and constructs the text of the report using prompt sentences.

[0090] A "prompt sentence" is the input sentence that serves as the basis for the generative AI model to generate a fault report, and indicates the details and conditions of the fault.

[0091] A "trouble report" is a document that compiles detailed information about a communication failure, and is created to accurately and promptly communicate the details of the failure to the technical department.

[0092] A "check box" is a GUI element that allows the user to input data in a format that allows the user to select, and indicates options that represent specific conditions of a communication failure.

[0093] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers. In particular, it enables sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with a server, the system of the present invention automatically generates outage reports from simple checkbox input, enabling them to be quickly shared with the technical department.

[0094] The user terminal provides a graphical user interface (GUI) that allows users to input information about communication disruptions in the form of check boxes. This check box-style input interface is designed to allow users to easily select information visually. For example, the user can select check boxes such as "communication disrupted," "specific areas only," and "specific time periods."

[0095] When a user selects a checkbox, the information is temporarily stored on the device and then sent to the server in real time using a secure communication protocol (e.g., HTTPS). The server receives the data sent from the device and first stores it in temporary storage. The stored data is then analyzed using Python scripts and machine learning libraries (e.g., scikit-learn, TENSORFLOW®).

[0096] The analyzed data is provided to a generative AI model installed on the server, which then automatically generates an outage report based on the prompt. The generative AI model used here is, for example, OpenAI's GPT-3 (registered trademark). An example of a prompt might be, "Users have reported that communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0097] The generated problem report is then sent back from the server to the user's device. This communication is also carried out using a secure protocol. The user then checks the problem report received on their device, inspects its contents, and makes any necessary corrections. Finally, the user sends the checked report back to the server and shares it with the technical department. The technical department then uses the received report to quickly analyze and resolve the problem.

[0098] This system allows users to generate simple and accurate fault reports without having to write technical documents, enabling quick and accurate information sharing with the technical department, which will improve customer satisfaction with regard to communication faults.

[0099] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0100] Step 1:

[0101] Input from the user terminal

[0102] The user uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, the user selects check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." This input data is temporarily stored in the device.

[0103] Input: User selected checkbox information

[0104] Output: Input data temporarily saved in the device

[0105] Specific operation: The user selects a checkbox in the GUI and enters conditions such as "communication has been lost," "specific areas only," and "specific time periods."

[0106] Step 2:

[0107] Sending and saving input data

[0108] The information of the checkboxes selected by the user is sent to the server in real time. The data is sent using a secure communication protocol such as HTTPS to ensure data safety. The server then stores the received data in temporary storage.

[0109] Input: Checkbox information sent from the device

[0110] Output: Data stored in temporary storage on the server

[0111] What it does: When a user selects a checkbox, the information is sent in real time to the server using an HTTPS request, and the server stores the data in temporary storage.

[0112] Step 3:

[0113] Data reception and analysis on the server

[0114] The server receives the data sent from the device and then converts it into a format that can be analyzed by AI. This process uses Python scripts and machine learning libraries (e.g., scikit-learn, TensorFlow). The data is classified by the contents of the checkboxes and generates appropriate keywords and phrases.

[0115] Input: Checkbox information stored on the server

[0116] Output: Data converted into a format that can be analyzed by AI

[0117] What it does: The Python script reads the data in JSON format, categorizes it based on the contents of the checkboxes, and extracts the required keywords and phrases.

[0118] Step 4:

[0119] AI-powered fault report generation

[0120] The converted data is provided to a generative AI model installed on the server. The AI ​​generates a fault report based on the prompt text, referring to the received data, past fault data, and a case database. OpenAI's GPT-3 is used as the generative AI model.

[0121] Input: Data converted into a format that can be analyzed by AI

[0122] Output: Generated problem report

[0123] Specific operation: The server sends a prompt to GPT-3 saying, "According to user reports..." and outputs a detailed error report.

[0124] Step 5:

[0125] Sending the generated report to the user's terminal

[0126] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[0127] Input: Generated problem report

[0128] Output: The problem report sent to the user's terminal

[0129] Specific operation: The generated report is sent from the server to the user terminal via a POST request, and the data received on the terminal is displayed in the GUI.

[0130] Step 6:

[0131] User review of report

[0132] The user then checks the received error report on the terminal, checks the contents, and makes any necessary corrections. This checking process is also performed through the GUI of the user terminal.

[0133] Input: A trouble report sent to the user's terminal.

[0134] Output: User-confirmed and corrected fault report

[0135] Specific operation: The user views the generated report on the terminal screen, edits the text if necessary, and clicks the confirmation button.

[0136] Step 7:

[0137] Sharing information with the technical department

[0138] Once the user has confirmed the issue, the report is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the issue. This information sharing is done using internal communication protocols and custom tools.

[0139] Input: User-confirmed and corrected fault report

[0140] Output: Fault report sent and shared with technical department

[0141] Specific operation: When the user presses the confirmation button, the report is sent to the server and a notification is automatically sent to the technical department's system. The technical department can then view the report on a dedicated dashboard and begin analysis.

[0142] (Application example 1)

[0143] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0144] In current security services, the procedures for security guards and security personnel to report emergencies are extremely complicated, making it difficult to respond in real time. In particular, personnel without technical expertise are required to communicate information accurately and quickly, but the systems to achieve this are not well-developed. As a result, false reports and delayed responses are likely to occur, hindering the rapid resolution of emergencies.

[0145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0146] In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for analyzing the input data in the server and automatically generating a report using AI, means for transmitting the generated report to the user terminal, means for sharing the report confirmed by the user with the response department, means for enabling security guards and security personnel to use the system as an emergency reporting system, means for providing the terminal with a means for inputting information about the emergency in the form of check boxes, and means for referring to a database of past cases and adding supplemental information when generating a report using AI. This enables security guards and security personnel to quickly and accurately report emergencies and share information with the response team in real time.

[0147] "User terminals" are input / display devices such as smartphones and head-mounted displays used by security guards and other security personnel.

[0148] "Input data" is information about an emergency that is entered by a security guard or security officer using a user terminal. It includes checkbox items such as "fire," "suspicious person found," and "intrusion."

[0149] "Server" means a system that includes data processing equipment and software for receiving and analyzing input data sent from user terminals, and generating and transmitting reports.

[0150] "Analysis" is the process in which the server classifies the input data it receives and prepares it for the AI ​​to generate a report.

[0151] A "report" is an AI-generated document containing details of an emergency that is sent to a user's device for final review and sharing.

[0152] The "checkbox format" is a graphical user interface format that allows a user to select specific conditions or items.

[0153] An "AI model" is an artificial intelligence that uses technologies such as machine learning and deep learning to automatically generate reports based on input data.

[0154] A "past case database" is a system that stores data on previously recorded emergency situations, which the AI ​​model references when generating its reports.

[0155] An "emergency reporting system" is a system that allows security guards and other security personnel to easily report emergencies and share information with response teams in real time.

[0156] A "prompt" is a guided sentence used by an AI model to generate a detailed report in natural language.

[0157] This invention is a system that allows security guards and other security personnel to quickly report emergencies and share information with response teams in real time. The system includes a user terminal, a server, an AI model, and a database of past cases.

[0158] First, the user terminal is a device that allows security guards and other security personnel to input information about emergencies. The terminal's graphical user interface (GUI) displays check boxes indicating emergency situations such as "fire," "suspicious person detected," and "intrusion." Security guards select the appropriate check box and enter the necessary information.

[0159] The server then receives and stores the input data sent from the user device in real time. The stored data is protected using a secure communication protocol (SSL / TLS). The received data is first temporarily stored in storage, and then converted into a format that the AI ​​model can analyze.

[0160] The AI ​​model in the server references the received data and a database of past cases, and then uses the generative AI model to automatically generate a report. A detailed natural language report is created using prompts. For example, a prompt such as "A fire has broken out. Please enter the detailed location and time."

[0161] The generated report is then sent back to the user's device from the server, where it is reviewed by security guards and other security personnel. Once reviewed, the report is sent back to the server and shared with the appropriate department in real time, enabling a prompt and accurate response.

[0162] As a concrete example, if a security guard discovers a fire in a certain area, the user selects the "fire" checkbox in the GUI on their device and enters the detailed location and time. This information is sent to the server, and the AI ​​model generates a specific report stating, "A fire broke out in the office area at 2:00 PM." The generated report is sent to the security guard's device, where it is reviewed and shared with the relevant department.

[0163] As components of this system, the server is deployed on an AWS® EC2 instance, the AI ​​model uses Google® Cloud AutoML, the database is managed by Amazon RDS, and the user devices include an ANDROID® smartphone and an Oculus Quest 2 (HMD). These elements work together to create an emergency reporting system that achieves the objective of the invention.

[0164] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0165] Step 1:

[0166] Input from the user terminal

[0167] The user (security guard or security officer) uses the terminal's graphical user interface (GUI) to input information about the emergency. They select specific check boxes (e.g., "fire," "suspicious person detected," "intrusion") and enter additional information such as detailed location and time. The input data is temporarily stored within the terminal.

[0168] Input: Emergency information (checkbox selection, detailed location, time)

[0169] Output: Temporarily saved data

[0170] Specific behavior:

[0171] The security guard selects the "Fire" checkbox and enters "Office Area" and "14:00".

[0172] Step 2:

[0173] Sending input data

[0174] The terminal transmits the temporarily stored input data to the server in real time using a secure communication protocol (SSL / TLS). The transmitted data reaches the server through a secure connection.

[0175] Input:Temporarily saved data

[0176] Output: Data sent to the server

[0177] Specific behavior:

[0178] The terminal sends the input data (selection of "fire", "office area", "14:00") to the server.

[0179] Step 3:

[0180] Receiving and temporarily storing data on the server

[0181] The server receives input data sent from the terminal and temporarily stores it in storage. It checks the format and content of the received data and temporarily stores it in preparation for converting it into the correct format.

[0182] Input: Data sent from the terminal

[0183] Output: Temporarily saved data

[0184] Specific behavior:

[0185] The server receives the data for "fire," "office area," and "14:00" and temporarily stores it.

[0186] Step 4:

[0187] Data analysis and preparation for input to AI models

[0188] The server then analyzes the temporarily stored data and converts it into a format that can be analyzed by the generative AI model. During this analysis process, the data is classified according to the contents of the checkboxes and appropriate keywords and phrases are generated.

[0189] Input:Temporarily saved data

[0190] Output: Parsed data

[0191] Specific behavior:

[0192] The server converts the data "fire", "office area", and "14:00" into "emergency: fire", "location: office area", and "time: 14:00".

[0193] Step 5:

[0194] AI-powered report generation

[0195] The AI ​​model installed on the server generates a detailed natural language report using prompt sentences based on the analyzed data and a database of past cases. The AI ​​model applies a generative AI model.

[0196] Input: Parsed data

[0197] Output: Generated report

[0198] Specific behavior:

[0199] The AI ​​generates a report stating, "A fire broke out in the office area at 14:00."

[0200] Step 6:

[0201] Sending generated reports

[0202] The report generated by the AI ​​is then sent from the server to the user's device, again using a secure communication protocol (SSL / TLS).

[0203] Input: Generated report

[0204] Output: Report sent to user terminal

[0205] Specific behavior:

[0206] A report "A fire broke out in the office area at 14:00" is sent from the server to the terminal.

[0207] Step 7:

[0208] User review of report

[0209] The user (security guard or security officer) checks the report received on the terminal, checks the contents, and makes corrections as necessary.

[0210] Input: Report sent to user terminal

[0211] Output: Confirmed and corrected report

[0212] Specific behavior:

[0213] The security guard will review the report and make corrections if necessary.

[0214] Step 8:

[0215] Share the report

[0216] The confirmed report is sent back to the server and shared in real time with the department in charge, who then promptly analyzes and responds to the problem based on the received report.

[0217] Input: Confirmed and corrected report

[0218] Output: Report shared with corresponding departments

[0219] Specific behavior:

[0220] The confirmed report is sent from the user terminal to the server, and the server shares it with the relevant department.

[0221] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0222] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers, and by combining it with an emotion engine that recognizes the user's emotions, it enables a more humane and appropriate response. By linking the user terminal with a server and emotion engine, the system of this invention automatically generates outage reports based on simple input in the form of checkboxes and an analysis of the user's emotions, making it possible to quickly share these reports with the technical department.

[0223] A natural language description of the program's processing

[0224] 1. Input from the user's device

[0225] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is also collected.

[0226] 2. Sending and saving input data and emotion data

[0227] The information on the checkboxes selected by the user and emotion data are temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol to ensure data safety.

[0228] 3. Data reception and analysis on the server

[0229] The server receives the data sent from the device. The received data is stored in temporary storage. Next, the input data and emotion data are converted into appropriate formats for data analysis. In this analysis process, the checkbox selections are converted into a format that can be input to the AI, and the emotion data is analyzed by the emotion engine.

[0230] 4. AI-based fault report generation

[0231] The AI ​​model on the server generates an outage report based on the analyzed data. The emotion engine's analysis results are used to adjust the tone and content of the report. For example, if the user is in a high stress state, a more detailed message urging a prompt response is generated. An example of a generated message is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0232] 5. Send the generated report to the user's device

[0233] The AI-generated fault report is sent from the server to the user's device, and this transmission also uses a secure communication protocol.

[0234] 6. User confirmation of report

[0235] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[0236] 7. Information sharing with the technical department

[0237] Once the user has confirmed the report, it is sent back to the server and shared with the technical department. The server then notifies the technical department of the report, including the analysis results of the emotion engine. This allows the technical department to analyze and resolve the problem while taking into account priorities based on the user's emotional state.

[0238] Specific examples of operation

[0239] For example, if a user (sales representative) receives information that communication was interrupted in a certain area during a specific time period, they can select checkboxes on their device's GUI, such as "Communication interrupted," "Only in a specific area," or "Specific time period." At this time, the user's emotional state is also recognized. The data is sent to the server in real time, where it is analyzed, and a specific problem report is generated using AI and an emotion engine. The report is then sent to the user's device, where the user confirms it and shares it with the technical department. In this way, users can easily generate accurate and appropriate problem reports based on their emotions, without having to write technical sentences. Furthermore, the technical department can quickly and appropriately resolve the problem by taking the user's emotional state into account.

[0240] As described above, the system of the present invention combines checkbox entry and emotion analysis to realize efficient and accurate fault reporting and information sharing, and aims to improve customer satisfaction with communication faults.

[0241] The processing flow will be explained below.

[0242] Step 1:

[0243] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, they select check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." The device also analyzes the user's facial expressions and tone of voice using an emotion engine, simultaneously acquiring emotional data.

[0244] Step 2:

[0245] The device temporarily stores the information about the checkboxes selected by the user and their emotional data, and transmits it to a server in real time using a secure communication protocol such as HTTPS to prevent data leakage or tampering.

[0246] Step 3:

[0247] The server receives the data sent from the device and stores it in temporary storage, where it is converted into an appropriate format that can be understood by the AI ​​and emotion engine in preparation for the analysis process.

[0248] Step 4:

[0249] As part of the analysis process, the server passes the input data to the AI ​​model and also passes the emotion data to the emotion engine. The AI ​​analyzes the input data and extracts the necessary information, while also referencing past failure data and case databases.

[0250] Step 5:

[0251] The emotion engine analyzes the user's emotional data and determines their emotional state (e.g., stress, anxiety, irritation, etc.). Based on this determination, the server adjusts the tone and content of the AI-generated report.

[0252] Step 6:

[0253] The server's AI model generates an outage report based on the analyzed data and the emotion engine's judgment. For example, it could generate a message like, "Users have reported that communication has been interrupted in a specific area and time period. Users are extremely dissatisfied. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0254] Step 7:

[0255] The server temporarily stores the generated problem report and prepares to send it to the user terminal. At this time, the generated report also includes the emotion analysis results.

[0256] Step 8:

[0257] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[0258] Step 9:

[0259] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[0260] Step 10:

[0261] The user resubmits the confirmed problem report from their device to the server and shares it with the engineering department. The server then notifies the engineering department of the report, including the analysis results of the emotion engine. This allows the engineering department to take action based on the user's emotional state.

[0262] Step 11:

[0263] The technical department will then promptly analyze and resolve the problem based on the received report. The results of the technical department's response will be fed back to the user's device via the server, and the final status will be shared with the user.

[0264] Example 2

[0265] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0266] Conventional communication outage reporting systems require users to manually create detailed reports, which requires a lot of time and effort. Furthermore, the content of the reports is often inconsistent, making it difficult to share information with the technical department. Furthermore, reports are generated in a fixed format without taking into account the user's emotional state, which can make it difficult to take prompt and appropriate action.

[0267] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0268] In this invention, the server includes means for storing input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for analyzing the input data and converting the data into an appropriate format in the server, means for analyzing the user's emotional data using an emotion analysis engine in the server, means for automatically generating a trouble report based on the analyzed data and the emotional data using AI, means for transmitting the generated trouble report to the user terminal, and means for sharing the trouble report confirmed by the user with the technical department. This allows the user to simply provide input data in a simple checkbox format, and accurate and consistent trouble reports generated in real time enable efficient information sharing with the technical department. Furthermore, emotion analysis allows for prompt and appropriate responses to be taken according to the user's emotional state.

[0269] A "user terminal" is a computer device that can be operated by a user and has an interface for inputting information about a communication failure and transmitting it to a server.

[0270] "Input data" refers to information relating to a communication failure that is input by a user via a terminal, and is data that includes selection items such as check boxes.

[0271] "Storage means" refers to a storage device or storage for temporarily or permanently storing input data and analyzed data.

[0272] "Transmission means" refers to a communication protocol or communication device for securely transmitting input data and generated reports to a user terminal or server.

[0273] A "server" is a computer system that analyzes data received from a user terminal and performs the necessary processing.

[0274] "Means of analysis" refers to software or algorithms that convert received input data into an appropriate format and analyze it using an AI model or sentiment analysis engine.

[0275] An "emotion analysis engine" is software or algorithms that analyze a user's emotional data and identify their state.

[0276] "Means for generating" refers to software or algorithms that use AI models to automatically generate incident reports based on analytical data and emotional data.

[0277] A "trouble report" is a document that describes detailed information about a communication failure, and includes analysis results and information that reflects the user's emotional state.

[0278] "Technical department" refers to the organization or team responsible for analyzing and resolving communication problems.

[0279] "Supplementary information" refers to relevant information added to reports generated based on past failure data and case databases.

[0280] The "checkbox format" is a GUI-style input format that allows users to easily enter options.

[0281] The present invention is a system that links a user terminal with a server and an emotion analysis engine to quickly and accurately generate reports about communication failures and share information with the technical department. This system is realized using the following hardware and software.

[0282] 1. Input via user terminal

[0283] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication outage. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is acquired. To acquire this emotional data, the device's sensors and camera are used to analyze facial expressions through facial recognition. This analysis uses common facial recognition algorithms and emotion analysis software.

[0284] 2. Sending input data and emotion data

[0285] The information about the checkboxes selected by the user and the emotion data are temporarily stored on the device. A secure communication protocol (e.g., HTTPS) is used to transmit the data to the server in real time. When the user enters information about the communication failure and presses the send button, the data is sent to the server in encrypted form.

[0286] 3. Data reception and analysis on the server

[0287] The server receives the data sent from the device and stores it in temporary storage. For data analysis, the input data is converted into a format that can be input to the AI, and the emotional data is analyzed by a sentiment analysis engine. This analysis process uses Python libraries (such as Pandas and SciPy). The sentiment analysis engine uses sentiment analysis APIs from IBM Watson (registered trademark) and Microsoft (registered trademark) Azure (registered trademark).

[0288] 4. AI-based fault report generation

[0289] The AI ​​model on the server generates an outage report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. Specifically, a machine learning model using TensorFlow and PyTorch is used. If the user is in a high stress state, the report will include more detailed text encouraging a prompt response. An example of the generated text is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0290] 5. Send the generated report to the user's device

[0291] The AI-generated fault report is sent from the server to the user's device via a secure communication protocol (e.g., HTTPS).

[0292] 6. User confirmation of report

[0293] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary. After corrections are completed, the report is sent back to the server.

[0294] 7. Information sharing with the technical department

[0295] Once the user has confirmed and completed the corrections, the report is sent to the server and shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[0296] A specific example of a prompt might be, "We have received information that communication was disrupted in Shibuya Ward between 14:00 and 16:00. The user is feeling extremely stressed. Based on this information, please generate an outage report that reflects the user's emotional state." When this prompt is input into a generative AI model (e.g., OpenAI's GPT-4 (registered trademark)), an appropriate outage report is output.

[0297] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0298] Step 1:

[0299] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." In addition, the device's sensors and camera collect the user's emotional data (e.g., facial expression analysis).

[0300] Input: User selected checkbox data and emotion data

[0301] Output: Temporarily saved checkbox data and emotion data

[0302] Step 2:

[0303] The device sends the temporarily stored data to the server in real time using a secure communication protocol (e.g., HTTPS). When the user completes entering the information and presses the send button, the data is sent in encrypted form.

[0304] Input: Temporarily stored checkbox data and emotion data

[0305] Output: Checkbox data and emotion data sent to the server

[0306] Step 3:

[0307] The server receives the data sent from the device and stores it in temporary storage (e.g., a database). Next, for data analysis, the input data (checkbox data) is converted into a format that can be input to AI (e.g., JSON format), and the emotion data is analyzed by an emotion analysis engine.

[0308] Input: Checkbox data and emotion data sent to the server

[0309] Output: Data converted into a format that can be fed into AI and sentiment analysis engines

[0310] Step 4:

[0311] The emotion analysis engine analyzes the received emotion data. Specifically, it analyzes the user's facial expression data to identify their stress level and emotional state. The emotion analysis engine uses common facial recognition algorithms and emotion analysis APIs.

[0312] Input: Emotion data input to the emotion analysis engine

[0313] Output: Emotion analysis results (e.g., stress level, high or low, etc.)

[0314] Step 5:

[0315] The AI ​​model in the server generates an incident report based on the analyzed checkbox data and emotion data, and adjusts the content and tone of the report based on the results of the emotion analysis engine. For example, a machine learning model using TensorFlow or PyTorch is used.

[0316] Input: Parsed checkbox data and sentiment data

[0317] Output: Generated problem report

[0318] Step 6:

[0319] The generated problem report is sent from the server to the user terminal, and this transmission is also performed via a secure communication protocol (e.g., HTTPS).

[0320] Input: Generated problem report

[0321] Output: The problem report sent to the user's terminal

[0322] Step 7:

[0323] The user checks the received failure report on the terminal, checks the contents, corrects the report if necessary, and then sends the corrected report back to the server.

[0324] Input: A trouble report sent to the user's terminal.

[0325] Output: A confirmed and corrected problem report

[0326] Step 8:

[0327] Once the user has confirmed and completed the corrections, the report is sent to the server and ultimately shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[0328] Input: Verified and corrected problem report

[0329] Output: A fault report shared with the technical department

[0330] (Application example 2)

[0331] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0332] Conventional systems have the drawback of being time-consuming to report communication failures or security issues, and are unable to respond to them taking into account the user's emotional state. Furthermore, it is difficult to respond quickly because users may not be able to express the content of their reports accurately, or it may take a long time to report. Therefore, a system that combines emotion analysis is needed to provide appropriate responses that reflect the user's emotions.

[0333] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for the generative AI model to automatically generate a problem report, an emotion analysis engine that analyzes the user's emotional state and includes means for adjusting the tone and content of the generated report, means for transmitting the generated problem report to the user terminal, and means for sharing the problem report confirmed by the user with related departments. This enables quick and appropriate problem reporting and response according to the user's emotions.

[0334] A "user terminal" is a device used by a user to input and manipulate information.

[0335] "Input data" refers to information about a problem that is provided by a user via a user terminal.

[0336] A "storage means" is a mechanism for temporarily or permanently storing input data.

[0337] The "means for transmitting" is a mechanism for transferring input data from the user terminal to the server.

[0338] A "generative AI model" is an artificial intelligence algorithm that automatically generates problem reports based on input data.

[0339] A "problem report" is a document that details a problem generated by a generative AI model.

[0340] An "emotion analysis engine" is a software or hardware mechanism for analyzing a user's emotional state.

[0341] "Means to adjust tone and content" refers to a function that adjusts the wording of problem reports based on the results of the sentiment analysis engine.

[0342] "Sharing means" refers to a mechanism for sharing generated problem reports with related departments.

[0343] "Historical Problem Data" is information about previously recorded problems.

[0344] A "case database" is a database that collects and stores data on past problems and cases.

[0345] A "checkbox format" is a type of GUI that allows users to easily select and enter information.

[0346] "Emotion data" is data relating to the user's emotional state.

[0347] This invention provides an efficient and accurate problem reporting system by linking a user terminal, a server, and an emotion analysis engine. In particular, the use of the emotion analysis engine enables more human-like and appropriate problem reporting and responses according to the user's emotions.

[0348] Hardware and Software

[0349] The hardware and software used in this system will now be described in detail.

[0350] Hardware

[0351] Smartphones: iOS and Android devices

[0352] Smart glasses: One example is Google Glass (registered trademark)

[0353] Head-mounted display: One example is Microsoft HoloLens (registered trademark)

[0354] software

[0355] Graphical User Interface (GUI) framework: Examples include React Native and Flutter (registered trademark)

[0356] Secure communication protocols: e.g. TLS / SSL

[0357] Implementation of AI models: Examples include TensorFlow and PyTorch

[0358] Emotion engine: Examples include Microsoft Azure Emotion API and IBM Watson Tone Analyzer

[0359] Overall system flow

[0360] 1. Input from the user terminal

[0361] Users use the device's graphical user interface (GUI) to report security issues. Specifically, they input information by selecting checkboxes such as "door lock is broken," "surveillance camera footage has been lost," and "this only occurs at certain times of the day." The user's emotional data is also collected via the camera and microphone and analyzed by an emotion analysis engine.

[0362] 2. Sending and saving input data and emotion data

[0363] Data sent from the device is temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol (e.g., TLS / SSL).

[0364] 3. Data reception and analysis on the server

[0365] The server stores the data sent from the device in temporary storage and converts it into an appropriate format for analysis: checkbox selections are converted into a format understandable by the generative AI model, and emotional data is analyzed by the emotion analysis engine.

[0366] 4. AI-powered problem report generation

[0367] The generative AI model in the server generates a problem report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. For example, if the user is in a high stress state, a message urging a quick response is generated.

[0368] 5. Sending the generated report to the user's terminal

[0369] The generated problem report is transmitted to the user terminal, again using a secure communications protocol.

[0370] 6. User review of report

[0371] The user can check the received problem report on the terminal, check the contents, and make corrections if necessary.

[0372] 7. Information sharing with the Security Center

[0373] After verification, the report is sent back to the server and shared with the security center and related departments. The report also includes the results of the emotion analysis engine, allowing the center to analyze and resolve the problem while taking into account the user's emotional state.

[0374] Specific examples of operation

[0375] Consider a case where a facility manager reports a problem where "surveillance camera footage has been lost." From the terminal's GUI, the manager selects "surveillance camera footage has been lost," "specific area only," and "specific time period." At this time, the manager's emotional state is also recognized, and this data is sent to the server. The server analyzes the data and uses a generative AI model to generate a specific problem report, reflecting the emotional tone.

[0376] Examples of generated prompts include:

[0377] A user reported an issue where the surveillance camera footage was cut off. The issue occurred in a specific area and at a specific time of day. This caused great anxiety for the user. The details are as follows: Area - Building B1, Time - 18:00-20:00.

[0378] As described above, the system of the present invention takes into consideration the user's feelings, generates a prompt and appropriate problem report, and shares it with related departments, thereby enabling a quick response and high customer satisfaction.

[0379] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0380] Step 1:

[0381] Question input on user device

[0382] The user inputs information about the problem through the device's graphical user interface (GUI). Specifically, they select options in the form of check boxes, such as "The door lock is broken," "The surveillance camera footage has been lost," or "This only occurs at certain times of the day." At the same time, the user's emotional data is acquired from the device's camera and microphone. The input data and emotional data are temporarily stored on the device.

[0383] Input: Problem selection and user sentiment data

[0384] Output: Temporarily saved input data and emotion data

[0385] Step 2:

[0386] Sending data

[0387] The device sends the saved input data and emotion data to the server using a secure communication protocol (e.g., TLS / SSL). The device notifies the user that the data has been sent.

[0388] Input: Temporarily saved input data and emotion data

[0389] Output: Input data and emotion data sent to the server

[0390] Step 3:

[0391] Receiving and storing data on the server

[0392] The server receives the transmitted data and stores it in temporary storage. The received data is converted into an appropriate format for analysis, and checkbox selections are converted into a format that can be input into a generative AI model. Emotion data is sent to an emotion analysis engine to analyze the emotional state.

[0393] Input: Input data and emotion data sent to the server

[0394] Output: Input data and emotion data converted into a parsable format

[0395] Step 4:

[0396] Generate a problem report

[0397] The server-based generative AI model generates a problem report based on the analyzed input data and emotional data. The tone and content of the report are adjusted using the results of the emotion analysis engine. For example, if the user is in a high-stress state, the report may include a prompt to take action.

[0398] Input: Input data and emotion data converted into a parsable format

[0399] Output: Problem reports with tailored tone and content

[0400] Step 5:

[0401] Submitting and reviewing reports

[0402] The generated problem report is then sent from the server to the terminal using a secure communication protocol, where the user can review the report and make corrections as necessary.

[0403] Input: Coordinated problem report

[0404] Output: Problem report sent to user's terminal, including whether it has been fixed or not

[0405] Step 6:

[0406] Sharing the final report

[0407] After the user has checked and made any necessary corrections, the final report is sent back to the server and shared with the Security Center and relevant departments. The analysis results from the sentiment analysis engine are also shared. Based on this, the Security Center can take prompt and appropriate action.

[0408] Input: Final Problem Report Corrected

[0409] Output: Final problem report shared with security center and relevant departments

[0410] The above are the processing steps of the system program that realizes the application example.

[0411] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0412] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0413] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0414] [Second embodiment]

[0415] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0416] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0417] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0418] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0419] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0420] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0421] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0422] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0423] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0424] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0425] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0426] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0427] This invention is a system that responds quickly and accurately to complaints about communication failures from corporate customers, and it allows sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with a server, the system of this invention automatically generates a failure report from simple input in the form of check boxes, and makes it possible to quickly share the report with the technical department.

[0428] A natural language description of the program's processing

[0429] 1. Input from the user's device

[0430] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, specifically by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods."

[0431] 2. Send and save input data

[0432] The information of the checkboxes selected by the user is temporarily stored on the device and then sent to the server in real time. This transmission uses a secure communication protocol, so the safety of the data is ensured.

[0433] 3. Data reception and analysis on the server

[0434] The server receives the data sent from the device. The received data is first stored in temporary storage and then converted into a format that can be analyzed by AI. In this analysis process, the data is classified by the contents of the checkboxes and appropriate keywords and phrases are generated.

[0435] 4. AI-based fault report generation

[0436] The converted data is provided to an AI model installed on the server. The AI ​​generates an outage report by referencing the received data, past outage data, and a case database. For example, the generated text might read, "According to user reports, communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0437] 5. Send the generated report to the user's device

[0438] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[0439] 6. User confirmation of report

[0440] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[0441] 7. Information sharing with the technical department

[0442] Once the user has confirmed the report, it is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the problem.

[0443] Specific examples of operation

[0444] For example, if a user (sales representative) receives information that communication was disrupted in a certain area during a specific time period, they can select check boxes such as "communication disrupted," "specific area only," or "specific time period" from the GUI on their device. This input data is sent to the server in real time, which analyzes the data and generates a specific failure report using AI. The report is then sent to the user's device, where the user checks the contents and, if there are no problems, shares it with the technical department.

[0445] In this way, the system of the present invention allows users to easily generate accurate trouble reports without having to write technical documents, and enables quick and accurate information sharing with the technical department, thereby aiming to improve customer satisfaction with regard to communication troubles.

[0446] The processing flow will be explained below.

[0447] Step 1:

[0448] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, for example, by selecting check boxes such as "communication was interrupted," "only in a specific area," and "specific time period."

[0449] Step 2:

[0450] The device temporarily stores the information of the checkboxes selected by the user, and this data is stored in temporary memory or local storage for subsequent processing.

[0451] Step 3:

[0452] The device sends the stored data to the server in real time using secure communication protocols such as HTTPS to prevent data leakage and tampering.

[0453] Step 4:

[0454] The server receives the data sent from the device and stores it in temporary storage.

[0455] Step 5:

[0456] The server parses the incoming data and converts it into a format that the AI ​​can understand. This parsing process involves converting checkbox selections into keywords and phrases.

[0457] Step 6:

[0458] The server provides the analyzed data to the AI ​​model, which then compares the received data with past fault data and a case database.

[0459] Step 7:

[0460] The server's AI generates a problem report based on the provided data, such as "Users have reported that communication has been interrupted in a specific area and time period. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0461] Step 8:

[0462] The server temporarily stores the generated failure report and prepares it for transmission to the user terminal.

[0463] Step 9:

[0464] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[0465] Step 10:

[0466] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[0467] Step 11:

[0468] The user retransmits the confirmed problem report from the terminal to the server and shares it with the technical department, and the server notifies the technical department of the shared report.

[0469] Step 12:

[0470] The technical department will promptly analyze and resolve the problem based on the received report, and will then provide feedback on the results of the response to the user.

[0471] Example 1

[0472] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0473] In conventional communication outage response systems, it is difficult for non-technical sales representatives and customer service representatives to properly convey information about communication outages to the technical department, resulting in delays and mistakes in the response. Furthermore, to reduce the time and effort required to manually create outage reports, automation is required, but creating professional-looking reports is not easy. Therefore, a system is needed that allows users to easily enter information about communication outages, automatically and quickly generate outage reports based on that information, and reliably share them with the technical department.

[0474] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0475] In this invention, the server includes means for saving input data received from a user terminal, means for analyzing the input data received from the user terminal and automatically generating a problem report based on a prompt sentence using a generative AI model, means for sending the generated problem report to the user terminal, and means for comparing past problem data and a case database and adding necessary supplemental information to the generated report. This allows even non-technical personnel to easily input information about communication problems, automatically and quickly create problem reports that meet specifications, and share the information with the technical department.

[0476] A "user terminal" is a device that provides an interface for a user to use, and is used to input information about a communication failure and transmit it to a server.

[0477] The "server" is a central processing system that receives, analyzes, and stores data sent from the user terminal, generates a fault report using a generative AI model, and sends it back to the user terminal.

[0478] "Input data" is information sent from a user terminal to a server, including details about a communication failure.

[0479] A "generative AI model" is an artificial intelligence model that automatically generates a fault report based on data received from the user's device, and constructs the text of the report using prompt sentences.

[0480] A "prompt sentence" is the input sentence that serves as the basis for the generative AI model to generate a fault report, and indicates the details and conditions of the fault.

[0481] A "trouble report" is a document that compiles detailed information about a communication failure, and is created to accurately and promptly communicate the details of the failure to the technical department.

[0482] A "check box" is a GUI element that allows the user to input data in a format that allows the user to select, and indicates options that represent specific conditions of a communication failure.

[0483] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers. In particular, it enables sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with a server, the system of the present invention automatically generates outage reports from simple checkbox input, enabling them to be quickly shared with the technical department.

[0484] The user terminal provides a graphical user interface (GUI) that allows users to input information about communication disruptions in the form of check boxes. This check box-style input interface is designed to allow users to easily select information visually. For example, the user can select check boxes such as "communication disrupted," "specific areas only," and "specific time periods."

[0485] When a user selects a checkbox, the information is temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol (e.g., HTTPS). The server receives the data sent from the device and first stores it in temporary storage. The stored data is then analyzed using Python scripts or machine learning libraries (e.g., scikit-learn, TensorFlow).

[0486] The analyzed data is provided to a generative AI model installed on the server, which then automatically generates an outage report based on the prompt text. The generative AI model used here is, for example, OpenAI's GPT-3. An example of a prompt text might be, "Users have reported that communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0487] The generated problem report is then sent back from the server to the user's device. This communication is also carried out using a secure protocol. The user then checks the problem report received on their device, inspects its contents, and makes any necessary corrections. Finally, the user sends the checked report back to the server and shares it with the technical department. The technical department then uses the received report to quickly analyze and resolve the problem.

[0488] This system allows users to generate simple and accurate fault reports without having to write technical documents, enabling quick and accurate information sharing with the technical department, which will improve customer satisfaction with regard to communication faults.

[0489] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0490] Step 1:

[0491] Input from the user terminal

[0492] The user uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, the user selects check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." This input data is temporarily stored in the device.

[0493] Input: User selected checkbox information

[0494] Output: Input data temporarily saved in the device

[0495] Specific operation: The user selects a checkbox in the GUI and enters conditions such as "communication has been lost," "specific areas only," and "specific time periods."

[0496] Step 2:

[0497] Sending and saving input data

[0498] The information of the checkboxes selected by the user is sent to the server in real time. The data is sent using a secure communication protocol such as HTTPS to ensure data safety. The server then stores the received data in temporary storage.

[0499] Input: Checkbox information sent from the device

[0500] Output: Data stored in temporary storage on the server

[0501] What it does: When a user selects a checkbox, the information is sent in real time to the server using an HTTPS request, and the server stores the data in temporary storage.

[0502] Step 3:

[0503] Data reception and analysis on the server

[0504] The server receives the data sent from the device and then converts it into a format that can be analyzed by AI. This process uses Python scripts and machine learning libraries (e.g., scikit-learn, TensorFlow). The data is classified by the contents of the checkboxes and generates appropriate keywords and phrases.

[0505] Input: Checkbox information stored on the server

[0506] Output: Data converted into a format that can be analyzed by AI

[0507] What it does: The Python script reads the data in JSON format, categorizes it based on the contents of the checkboxes, and extracts the required keywords and phrases.

[0508] Step 4:

[0509] AI-powered fault report generation

[0510] The converted data is provided to a generative AI model installed on the server. The AI ​​generates a fault report based on the prompt text, referring to the received data, past fault data, and a case database. OpenAI's GPT-3 is used as the generative AI model.

[0511] Input: Data converted into a format that can be analyzed by AI

[0512] Output: Generated problem report

[0513] Specific operation: The server sends a prompt to GPT-3 saying, "According to user reports..." and outputs a detailed error report.

[0514] Step 5:

[0515] Sending the generated report to the user's terminal

[0516] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[0517] Input: Generated problem report

[0518] Output: The problem report sent to the user's terminal

[0519] Specific operation: The generated report is sent from the server to the user terminal via a POST request, and the data received on the terminal is displayed in the GUI.

[0520] Step 6:

[0521] User review of report

[0522] The user then checks the received error report on the terminal, checks the contents, and makes any necessary corrections. This checking process is also performed through the GUI of the user terminal.

[0523] Input: A trouble report sent to the user's terminal.

[0524] Output: User-confirmed and corrected fault report

[0525] Specific operation: The user views the generated report on the terminal screen, edits the text if necessary, and clicks the confirmation button.

[0526] Step 7:

[0527] Sharing information with the technical department

[0528] Once the user has confirmed the issue, the report is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the issue. This information sharing is done using internal communication protocols and custom tools.

[0529] Input: User-confirmed and corrected fault report

[0530] Output: Fault report sent and shared with technical department

[0531] Specific operation: When the user presses the confirmation button, the report is sent to the server and a notification is automatically sent to the technical department's system. The technical department can then view the report on a dedicated dashboard and begin analysis.

[0532] (Application example 1)

[0533] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0534] In current security services, the procedures for security guards and security personnel to report emergencies are extremely complicated, making it difficult to respond in real time. In particular, personnel without technical expertise are required to communicate information accurately and quickly, but the systems to achieve this are not well-developed. As a result, false reports and delayed responses are likely to occur, hindering the rapid resolution of emergencies.

[0535] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0536] In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for analyzing the input data in the server and automatically generating a report using AI, means for transmitting the generated report to the user terminal, means for sharing the report confirmed by the user with the response department, means for enabling security guards and security personnel to use the system as an emergency reporting system, means for providing the terminal with a means for inputting information about the emergency in the form of check boxes, and means for referring to a database of past cases and adding supplemental information when generating a report using AI. This enables security guards and security personnel to quickly and accurately report emergencies and share information with the response team in real time.

[0537] "User terminals" are input / display devices such as smartphones and head-mounted displays used by security guards and other security personnel.

[0538] "Input data" is information about an emergency that is entered by a security guard or security officer using a user terminal. It includes checkbox items such as "fire," "suspicious person found," and "intrusion."

[0539] "Server" means a system that includes data processing equipment and software for receiving and analyzing input data sent from user terminals, and generating and transmitting reports.

[0540] "Analysis" is the process in which the server classifies the input data it receives and prepares it for the AI ​​to generate a report.

[0541] A "report" is an AI-generated document containing details of an emergency that is sent to a user's device for final review and sharing.

[0542] The "checkbox format" is a graphical user interface format that allows a user to select specific conditions or items.

[0543] An "AI model" is an artificial intelligence that uses technologies such as machine learning and deep learning to automatically generate reports based on input data.

[0544] A "past case database" is a system that stores data on previously recorded emergency situations, which the AI ​​model references when generating its reports.

[0545] An "emergency reporting system" is a system that allows security guards and other security personnel to easily report emergencies and share information with response teams in real time.

[0546] A "prompt" is a guided sentence used by an AI model to generate a detailed report in natural language.

[0547] This invention is a system that allows security guards and other security personnel to quickly report emergencies and share information with response teams in real time. The system includes a user terminal, a server, an AI model, and a database of past cases.

[0548] First, the user terminal is a device that allows security guards and other security personnel to input information about emergencies. The terminal's graphical user interface (GUI) displays check boxes indicating emergency situations such as "fire," "suspicious person detected," and "intrusion." Security guards select the appropriate check box and enter the necessary information.

[0549] The server then receives and stores the input data sent from the user device in real time. The stored data is protected using a secure communication protocol (SSL / TLS). The received data is first temporarily stored in storage, and then converted into a format that the AI ​​model can analyze.

[0550] The AI ​​model in the server references the received data and a database of past cases, and then uses the generative AI model to automatically generate a report. A detailed natural language report is created using prompts. For example, a prompt such as "A fire has broken out. Please enter the detailed location and time."

[0551] The generated report is then sent back to the user's device from the server, where it is reviewed by security guards and other security personnel. Once reviewed, the report is sent back to the server and shared with the appropriate department in real time, enabling a prompt and accurate response.

[0552] As a concrete example, if a security guard discovers a fire in a certain area, the user selects the "fire" checkbox in the GUI on their device and enters the detailed location and time. This information is sent to the server, and the AI ​​model generates a specific report stating, "A fire broke out in the office area at 2:00 PM." The generated report is sent to the security guard's device, where it is reviewed and shared with the relevant department.

[0553] The components of this system include a server deployed on an AWS EC2 instance, an AI model using Google Cloud AutoML, a database managed by Amazon RDS, and user devices including Android smartphones and Oculus Quest 2 (HMD). These elements work together to create an emergency reporting system that achieves the objective of the invention.

[0554] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0555] Step 1:

[0556] Input from the user terminal

[0557] The user (security guard or security officer) uses the terminal's graphical user interface (GUI) to input information about the emergency. They select specific check boxes (e.g., "fire," "suspicious person detected," "intrusion") and enter additional information such as detailed location and time. The input data is temporarily stored within the terminal.

[0558] Input: Emergency information (checkbox selection, detailed location, time)

[0559] Output: Temporarily saved data

[0560] Specific behavior:

[0561] The security guard selects the "Fire" checkbox and enters "Office Area" and "14:00".

[0562] Step 2:

[0563] Sending input data

[0564] The terminal transmits the temporarily stored input data to the server in real time using a secure communication protocol (SSL / TLS). The transmitted data reaches the server through a secure connection.

[0565] Input:Temporarily saved data

[0566] Output: Data sent to the server

[0567] Specific behavior:

[0568] The terminal sends the input data (selection of "fire", "office area", "14:00") to the server.

[0569] Step 3:

[0570] Receiving and temporarily storing data on the server

[0571] The server receives input data sent from the terminal and temporarily stores it in storage. It checks the format and content of the received data and temporarily stores it in preparation for converting it into the correct format.

[0572] Input: Data sent from the terminal

[0573] Output: Temporarily saved data

[0574] Specific behavior:

[0575] The server receives the data for "fire," "office area," and "14:00" and temporarily stores it.

[0576] Step 4:

[0577] Data analysis and preparation for input to AI models

[0578] The server then analyzes the temporarily stored data and converts it into a format that can be analyzed by the generative AI model. During this analysis process, the data is classified according to the contents of the checkboxes and appropriate keywords and phrases are generated.

[0579] Input:Temporarily saved data

[0580] Output: Parsed data

[0581] Specific behavior:

[0582] The server converts the data "fire", "office area", and "14:00" into "emergency: fire", "location: office area", and "time: 14:00".

[0583] Step 5:

[0584] AI-powered report generation

[0585] The AI ​​model installed on the server generates a detailed natural language report using prompt sentences based on the analyzed data and a database of past cases. The AI ​​model applies a generative AI model.

[0586] Input: Parsed data

[0587] Output: Generated report

[0588] Specific behavior:

[0589] The AI ​​generates a report stating, "A fire broke out in the office area at 14:00."

[0590] Step 6:

[0591] Sending generated reports

[0592] The report generated by the AI ​​is then sent from the server to the user's device, again using a secure communication protocol (SSL / TLS).

[0593] Input: Generated report

[0594] Output: Report sent to user terminal

[0595] Specific behavior:

[0596] A report "A fire broke out in the office area at 14:00" is sent from the server to the terminal.

[0597] Step 7:

[0598] User review of report

[0599] The user (security guard or security officer) checks the report received on the terminal, checks the contents, and makes corrections as necessary.

[0600] Input: Report sent to user terminal

[0601] Output: Confirmed and corrected report

[0602] Specific behavior:

[0603] The security guard will review the report and make corrections if necessary.

[0604] Step 8:

[0605] Share the report

[0606] The confirmed report is sent back to the server and shared in real time with the department in charge, who then promptly analyzes and responds to the problem based on the received report.

[0607] Input: Confirmed and corrected report

[0608] Output: Report shared with corresponding departments

[0609] Specific behavior:

[0610] The confirmed report is sent from the user terminal to the server, and the server shares it with the relevant department.

[0611] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0612] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers, and by combining it with an emotion engine that recognizes the user's emotions, it enables a more humane and appropriate response. By linking the user terminal with a server and emotion engine, the system of this invention automatically generates outage reports based on simple input in the form of checkboxes and an analysis of the user's emotions, making it possible to quickly share these reports with the technical department.

[0613] A natural language description of the program's processing

[0614] 1. Input from the user's device

[0615] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is also collected.

[0616] 2. Sending and saving input data and emotion data

[0617] The information on the checkboxes selected by the user and emotion data are temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol to ensure data safety.

[0618] 3. Data reception and analysis on the server

[0619] The server receives the data sent from the device. The received data is stored in temporary storage. Next, the input data and emotion data are converted into appropriate formats for data analysis. In this analysis process, the checkbox selections are converted into a format that can be input to the AI, and the emotion data is analyzed by the emotion engine.

[0620] 4. AI-based fault report generation

[0621] The AI ​​model on the server generates an outage report based on the analyzed data. The emotion engine's analysis results are used to adjust the tone and content of the report. For example, if the user is in a high stress state, a more detailed message urging a prompt response is generated. An example of a generated message is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0622] 5. Send the generated report to the user's device

[0623] The AI-generated fault report is sent from the server to the user's device, and this transmission also uses a secure communication protocol.

[0624] 6. User confirmation of report

[0625] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[0626] 7. Information sharing with the technical department

[0627] Once the user has confirmed the report, it is sent back to the server and shared with the technical department. The server then notifies the technical department of the report, including the analysis results of the emotion engine. This allows the technical department to analyze and resolve the problem while taking into account priorities based on the user's emotional state.

[0628] Specific examples of operation

[0629] For example, if a user (sales representative) receives information that communication was interrupted in a certain area during a specific time period, they can select checkboxes on their device's GUI, such as "Communication interrupted," "Only in a specific area," or "Specific time period." At this time, the user's emotional state is also recognized. The data is sent to the server in real time, where it is analyzed, and a specific problem report is generated using AI and an emotion engine. The report is then sent to the user's device, where the user confirms it and shares it with the technical department. In this way, users can easily generate accurate and appropriate problem reports based on their emotions, without having to write technical sentences. Furthermore, the technical department can quickly and appropriately resolve the problem by taking the user's emotional state into account.

[0630] As described above, the system of the present invention combines checkbox entry and emotion analysis to realize efficient and accurate fault reporting and information sharing, and aims to improve customer satisfaction with communication faults.

[0631] The processing flow will be explained below.

[0632] Step 1:

[0633] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, they select check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." The device also analyzes the user's facial expressions and tone of voice using an emotion engine, simultaneously acquiring emotional data.

[0634] Step 2:

[0635] The device temporarily stores the information about the checkboxes selected by the user and their emotional data, and transmits it to a server in real time using a secure communication protocol such as HTTPS to prevent data leakage or tampering.

[0636] Step 3:

[0637] The server receives the data sent from the device and stores it in temporary storage, where it is converted into an appropriate format that can be understood by the AI ​​and emotion engine in preparation for the analysis process.

[0638] Step 4:

[0639] As part of the analysis process, the server passes the input data to the AI ​​model and also passes the emotion data to the emotion engine. The AI ​​analyzes the input data and extracts the necessary information, while also referencing past failure data and case databases.

[0640] Step 5:

[0641] The emotion engine analyzes the user's emotional data and determines their emotional state (e.g., stress, anxiety, irritation, etc.). Based on this determination, the server adjusts the tone and content of the AI-generated report.

[0642] Step 6:

[0643] The server's AI model generates an outage report based on the analyzed data and the emotion engine's judgment. For example, it could generate a message like, "Users have reported that communication has been interrupted in a specific area and time period. Users are extremely dissatisfied. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0644] Step 7:

[0645] The server temporarily stores the generated problem report and prepares to send it to the user terminal. At this time, the generated report also includes the emotion analysis results.

[0646] Step 8:

[0647] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[0648] Step 9:

[0649] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[0650] Step 10:

[0651] The user resubmits the confirmed problem report from their device to the server and shares it with the engineering department. The server then notifies the engineering department of the report, including the analysis results of the emotion engine. This allows the engineering department to take action based on the user's emotional state.

[0652] Step 11:

[0653] The technical department will then promptly analyze and resolve the problem based on the received report. The results of the technical department's response will be fed back to the user's device via the server, and the final status will be shared with the user.

[0654] Example 2

[0655] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] Conventional communication outage reporting systems require users to manually create detailed reports, which requires a lot of time and effort. Furthermore, the content of the reports is often inconsistent, making it difficult to share information with the technical department. Furthermore, reports are generated in a fixed format without taking into account the user's emotional state, which can make it difficult to take prompt and appropriate action.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0658] In this invention, the server includes: means for storing input data received from a user terminal; means for transmitting the input data received from the user terminal to the server; means for analyzing the input data and converting the data into an appropriate format at the server; means for analyzing the user's emotional data using an emotion analysis engine at the server; means for automatically generating a trouble report based on the analyzed data and the emotional data using AI; means for transmitting the generated trouble report to the user terminal; and means for sharing the trouble report confirmed by the user with the technical department. This allows the user to simply provide input data in a simple checkbox format, and accurate and consistent trouble reports generated in real time enable efficient information sharing with the technical department. Furthermore, emotion analysis allows for prompt and appropriate responses to be taken according to the user's emotional state.

[0659] A "user terminal" is a computer device that can be operated by a user and has an interface for inputting information about a communication failure and transmitting it to a server.

[0660] "Input data" refers to information relating to a communication failure that is input by a user via a terminal, and is data that includes selection items such as check boxes.

[0661] "Storage means" refers to a storage device or storage for temporarily or permanently storing input data and analyzed data.

[0662] "Transmission means" refers to a communication protocol or communication device for securely transmitting input data and generated reports to a user terminal or server.

[0663] A "server" is a computer system that analyzes data received from a user terminal and performs the necessary processing.

[0664] "Means of analysis" refers to software or algorithms that convert received input data into an appropriate format and analyze it using an AI model or sentiment analysis engine.

[0665] An "emotion analysis engine" is software or algorithms that analyze a user's emotional data and identify their state.

[0666] "Means for generating" refers to software or algorithms that use AI models to automatically generate incident reports based on analytical data and emotional data.

[0667] A "trouble report" is a document that describes detailed information about a communication failure, and includes analysis results and information that reflects the user's emotional state.

[0668] "Technical department" refers to the organization or team responsible for analyzing and resolving communication problems.

[0669] "Supplementary information" refers to relevant information added to reports generated based on past failure data and case databases.

[0670] The "checkbox format" is a GUI-style input format that allows users to easily enter options.

[0671] The present invention is a system that links a user terminal with a server and an emotion analysis engine to quickly and accurately generate reports about communication failures and share information with the technical department. This system is realized using the following hardware and software.

[0672] 1. Input via user terminal

[0673] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication outage. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is acquired. To acquire this emotional data, the device's sensors and camera are used to analyze facial expressions through facial recognition. This analysis uses common facial recognition algorithms and emotion analysis software.

[0674] 2. Sending input data and emotion data

[0675] The information about the checkboxes selected by the user and the emotion data are temporarily stored on the device. A secure communication protocol (e.g., HTTPS) is used to transmit the data to the server in real time. When the user enters information about the communication failure and presses the send button, the data is sent to the server in encrypted form.

[0676] 3. Data reception and analysis on the server

[0677] The server receives the data sent from the device and stores it in temporary storage. For data analysis, the input data is converted into a format that can be input to the AI, and the emotional data is analyzed by a sentiment analysis engine. This analysis process uses Python libraries (such as Pandas and SciPy). The sentiment analysis engine also uses sentiment analysis APIs from IBM Watson and Microsoft Azure.

[0678] 4. AI-based fault report generation

[0679] The AI ​​model on the server generates an outage report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. Specifically, a machine learning model using TensorFlow and PyTorch is used. If the user is in a high stress state, the report will include more detailed text encouraging a prompt response. An example of the generated text is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0680] 5. Send the generated report to the user's device

[0681] The AI-generated fault report is sent from the server to the user's device via a secure communication protocol (e.g., HTTPS).

[0682] 6. User confirmation of report

[0683] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary. After corrections are completed, the report is sent back to the server.

[0684] 7. Information sharing with the technical department

[0685] Once the user has confirmed and completed the corrections, the report is sent to the server and shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[0686] A specific example of a prompt might be, "We have received information that communication was disrupted in Shibuya Ward between 14:00 and 16:00. The user is feeling very stressed. Based on this information, please generate an outage report that reflects the user's emotional state." When this prompt is input into a generative AI model (e.g., OpenAI's GPT-4), an appropriate outage report is output.

[0687] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0688] Step 1:

[0689] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." In addition, the device's sensors and camera collect the user's emotional data (e.g., facial expression analysis).

[0690] Input: User selected checkbox data and emotion data

[0691] Output: Temporarily saved checkbox data and emotion data

[0692] Step 2:

[0693] The device sends the temporarily stored data to the server in real time using a secure communication protocol (e.g., HTTPS). When the user completes entering the information and presses the send button, the data is sent in encrypted form.

[0694] Input: Temporarily stored checkbox data and emotion data

[0695] Output: Checkbox data and emotion data sent to the server

[0696] Step 3:

[0697] The server receives the data sent from the device and stores it in temporary storage (e.g., a database). Next, for data analysis, the input data (checkbox data) is converted into a format that can be input to AI (e.g., JSON format), and the emotion data is analyzed by an emotion analysis engine.

[0698] Input: Checkbox data and emotion data sent to the server

[0699] Output: Data converted into a format that can be fed into AI and sentiment analysis engines

[0700] Step 4:

[0701] The emotion analysis engine analyzes the received emotion data. Specifically, it analyzes the user's facial expression data to identify their stress level and emotional state. The emotion analysis engine uses common facial recognition algorithms and emotion analysis APIs.

[0702] Input: Emotion data input to the emotion analysis engine

[0703] Output: Emotion analysis results (e.g., stress level, high or low, etc.)

[0704] Step 5:

[0705] The AI ​​model in the server generates an incident report based on the analyzed checkbox data and emotion data, and adjusts the content and tone of the report based on the results of the emotion analysis engine. For example, a machine learning model using TensorFlow or PyTorch is used.

[0706] Input: Parsed checkbox data and sentiment data

[0707] Output: Generated problem report

[0708] Step 6:

[0709] The generated problem report is sent from the server to the user terminal, and this transmission is also performed via a secure communication protocol (e.g., HTTPS).

[0710] Input: Generated problem report

[0711] Output: The problem report sent to the user's terminal

[0712] Step 7:

[0713] The user checks the received failure report on the terminal, checks the contents, corrects the report if necessary, and then sends the corrected report back to the server.

[0714] Input: A trouble report sent to the user's terminal.

[0715] Output: A confirmed and corrected problem report

[0716] Step 8:

[0717] Once the user has confirmed and completed the corrections, the report is sent to the server and ultimately shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[0718] Input: Verified and corrected problem report

[0719] Output: A fault report shared with the technical department

[0720] (Application example 2)

[0721] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0722] Conventional systems have the drawback of being time-consuming to report communication failures or security issues, and are unable to respond to them taking into account the user's emotional state. Furthermore, it is difficult to respond quickly because users may not be able to express the content of their reports accurately, or it may take a long time to report. Therefore, a system that combines emotion analysis is needed to provide appropriate responses that reflect the user's emotions.

[0723] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for the generative AI model to automatically generate a problem report, an emotion analysis engine that analyzes the user's emotional state and includes means for adjusting the tone and content of the generated report, means for transmitting the generated problem report to the user terminal, and means for sharing the problem report confirmed by the user with related departments. This enables quick and appropriate problem reporting and response according to the user's emotions.

[0724] A "user terminal" is a device used by a user to input and manipulate information.

[0725] "Input data" refers to information about a problem that is provided by a user via a user terminal.

[0726] A "storage means" is a mechanism for temporarily or permanently storing input data.

[0727] The "means for transmitting" is a mechanism for transferring input data from the user terminal to the server.

[0728] A "generative AI model" is an artificial intelligence algorithm that automatically generates problem reports based on input data.

[0729] A "problem report" is a document that details a problem generated by a generative AI model.

[0730] An "emotion analysis engine" is a software or hardware mechanism for analyzing a user's emotional state.

[0731] "Means to adjust tone and content" refers to a function that adjusts the wording of problem reports based on the results of the sentiment analysis engine.

[0732] "Sharing means" refers to a mechanism for sharing generated problem reports with related departments.

[0733] "Historical Problem Data" is information about previously recorded problems.

[0734] A "case database" is a database that collects and stores data on past problems and cases.

[0735] A "checkbox format" is a type of GUI that allows users to easily select and enter information.

[0736] "Emotion data" is data relating to the user's emotional state.

[0737] This invention provides an efficient and accurate problem reporting system by linking a user terminal, a server, and an emotion analysis engine. In particular, the use of the emotion analysis engine enables more human-like and appropriate problem reporting and responses according to the user's emotions.

[0738] Hardware and Software

[0739] The hardware and software used in this system will now be described in detail.

[0740] Hardware

[0741] Smartphones: iOS and Android devices

[0742] Smart glasses: Google Glass as an example

[0743] Head-mounted displays: An example is Microsoft HoloLens

[0744] software

[0745] Graphical User Interface (GUI) frameworks: Examples include React Native and Flutter

[0746] Secure communication protocols: e.g. TLS / SSL

[0747] Implementation of AI models: Examples include TensorFlow and PyTorch

[0748] Emotion engine: Examples include Microsoft Azure Emotion API and IBM Watson Tone Analyzer

[0749] Overall system flow

[0750] 1. Input from the user terminal

[0751] Users use the device's graphical user interface (GUI) to report security issues. Specifically, they input information by selecting checkboxes such as "door lock is broken," "surveillance camera footage has been lost," and "this only occurs at certain times of the day." The user's emotional data is also collected via the camera and microphone and analyzed by an emotion analysis engine.

[0752] 2. Sending and saving input data and emotion data

[0753] Data sent from the device is temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol (e.g., TLS / SSL).

[0754] 3. Data reception and analysis on the server

[0755] The server stores the data sent from the device in temporary storage and converts it into an appropriate format for analysis: checkbox selections are converted into a format understandable by the generative AI model, and emotional data is analyzed by the emotion analysis engine.

[0756] 4. AI-powered problem report generation

[0757] The generative AI model in the server generates a problem report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. For example, if the user is in a high stress state, a message urging a quick response is generated.

[0758] 5. Sending the generated report to the user's terminal

[0759] The generated problem report is transmitted to the user terminal, again using a secure communications protocol.

[0760] 6. User review of report

[0761] The user can check the received problem report on the terminal, check the contents, and make corrections if necessary.

[0762] 7. Information sharing with the Security Center

[0763] After verification, the report is sent back to the server and shared with the security center and related departments. The report also includes the results of the emotion analysis engine, allowing the center to analyze and resolve the problem while taking into account the user's emotional state.

[0764] Specific examples of operation

[0765] Consider a case where a facility manager reports a problem where "surveillance camera footage has been lost." From the terminal's GUI, the manager selects "surveillance camera footage has been lost," "specific area only," and "specific time period." At this time, the manager's emotional state is also recognized, and this data is sent to the server. The server analyzes the data and uses a generative AI model to generate a specific problem report, reflecting the emotional tone.

[0766] Examples of generated prompts include:

[0767] A user reported an issue where the surveillance camera footage was cut off. The issue occurred in a specific area and at a specific time of day. This caused great anxiety for the user. The details are as follows: Area - Building B1, Time - 18:00-20:00.

[0768] As described above, the system of the present invention takes into consideration the user's feelings, generates a prompt and appropriate problem report, and shares it with related departments, thereby enabling a quick response and high customer satisfaction.

[0769] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0770] Step 1:

[0771] Question input on user device

[0772] The user inputs information about the problem through the device's graphical user interface (GUI). Specifically, they select options in the form of check boxes, such as "The door lock is broken," "The surveillance camera footage has been lost," or "This only occurs at certain times of the day." At the same time, the user's emotional data is acquired from the device's camera and microphone. The input data and emotional data are temporarily stored on the device.

[0773] Input: Problem selection and user sentiment data

[0774] Output: Temporarily saved input data and emotion data

[0775] Step 2:

[0776] Sending data

[0777] The device sends the saved input data and emotion data to the server using a secure communication protocol (e.g., TLS / SSL). The device notifies the user that the data has been sent.

[0778] Input: Temporarily saved input data and emotion data

[0779] Output: Input data and emotion data sent to the server

[0780] Step 3:

[0781] Receiving and storing data on the server

[0782] The server receives the transmitted data and stores it in temporary storage. The received data is converted into an appropriate format for analysis, and checkbox selections are converted into a format that can be input into a generative AI model. Emotion data is sent to an emotion analysis engine to analyze the emotional state.

[0783] Input: Input data and emotion data sent to the server

[0784] Output: Input data and emotion data converted into a parsable format

[0785] Step 4:

[0786] Generate a problem report

[0787] The server-based generative AI model generates a problem report based on the analyzed input data and emotional data. The tone and content of the report are adjusted using the results of the emotion analysis engine. For example, if the user is in a high-stress state, the report may include a prompt to take action.

[0788] Input: Input data and emotion data converted into a parsable format

[0789] Output: Problem reports with tailored tone and content

[0790] Step 5:

[0791] Submitting and reviewing reports

[0792] The generated problem report is then sent from the server to the terminal using a secure communication protocol, where the user can review the report and make corrections as necessary.

[0793] Input: Coordinated problem report

[0794] Output: Problem report sent to user's terminal, including whether it has been fixed or not

[0795] Step 6:

[0796] Sharing the final report

[0797] After the user has checked and made any necessary corrections, the final report is sent back to the server and shared with the Security Center and relevant departments. The analysis results from the sentiment analysis engine are also shared. Based on this, the Security Center can take prompt and appropriate action.

[0798] Input: Final Problem Report Corrected

[0799] Output: Final problem report shared with security center and relevant departments

[0800] The above are the processing steps of the system program that realizes the application example.

[0801] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0802] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0803] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0804] [Third embodiment]

[0805] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0806] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0807] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0808] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0809] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0810] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0811] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0812] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0813] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0814] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0815] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0816] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0817] This invention is a system that responds quickly and accurately to complaints about communication failures from corporate customers, and it allows sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with the server, the system of this invention automatically generates a failure report from simple input in the form of check boxes, and makes it possible to quickly share the report with the technical department.

[0818] A natural language description of the program's processing

[0819] 1. Input from the user's device

[0820] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, specifically by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods."

[0821] 2. Send and save input data

[0822] The information of the checkboxes selected by the user is temporarily stored on the device and then sent to the server in real time. This transmission uses a secure communication protocol, so the safety of the data is ensured.

[0823] 3. Data reception and analysis on the server

[0824] The server receives the data sent from the device. The received data is first stored in temporary storage and then converted into a format that can be analyzed by AI. In this analysis process, the data is classified by the contents of the checkboxes and appropriate keywords and phrases are generated.

[0825] 4. AI-based fault report generation

[0826] The converted data is provided to an AI model installed on the server. The AI ​​generates an outage report by referencing the received data, past outage data, and a case database. For example, the generated text might read, "According to user reports, communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0827] 5. Send the generated report to the user's device

[0828] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[0829] 6. User confirmation of report

[0830] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[0831] 7. Information sharing with the technical department

[0832] Once the user has confirmed the report, it is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the problem.

[0833] Specific examples of operation

[0834] For example, if a user (sales representative) receives information that communication was disrupted in a certain area during a specific time period, they can select check boxes such as "communication disrupted," "specific area only," or "specific time period" from the GUI on their device. This input data is sent to the server in real time, which analyzes the data and generates a specific failure report using AI. The report is then sent to the user's device, where the user checks the contents and, if there are no problems, shares it with the technical department.

[0835] In this way, the system of the present invention allows users to easily generate accurate trouble reports without having to write technical documents, and enables quick and accurate information sharing with the technical department, thereby aiming to improve customer satisfaction with regard to communication troubles.

[0836] The processing flow will be explained below.

[0837] Step 1:

[0838] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, for example, by selecting check boxes such as "communication was interrupted," "only in a specific area," and "specific time period."

[0839] Step 2:

[0840] The device temporarily stores the information of the checkboxes selected by the user, and this data is stored in temporary memory or local storage for subsequent processing.

[0841] Step 3:

[0842] The device sends the stored data to the server in real time using secure communication protocols such as HTTPS to prevent data leakage and tampering.

[0843] Step 4:

[0844] The server receives the data sent from the device and stores it in temporary storage.

[0845] Step 5:

[0846] The server parses the incoming data and converts it into a format that the AI ​​can understand. This parsing process involves converting checkbox selections into keywords and phrases.

[0847] Step 6:

[0848] The server provides the analyzed data to the AI ​​model, which then compares the received data with past fault data and a case database.

[0849] Step 7:

[0850] The server's AI generates a problem report based on the provided data, such as "Users have reported that communication has been interrupted in a specific area and time period. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0851] Step 8:

[0852] The server temporarily stores the generated failure report and prepares it for transmission to the user terminal.

[0853] Step 9:

[0854] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[0855] Step 10:

[0856] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[0857] Step 11:

[0858] The user retransmits the confirmed problem report from the terminal to the server and shares it with the technical department, and the server notifies the technical department of the shared report.

[0859] Step 12:

[0860] The technical department will promptly analyze and resolve the problem based on the received report, and will then provide feedback on the results of the response to the user.

[0861] Example 1

[0862] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0863] In conventional communication outage response systems, it is difficult for non-technical sales representatives and customer service representatives to properly convey information about communication outages to the technical department, resulting in delays and mistakes in the response. Furthermore, to reduce the time and effort required to manually create outage reports, automation is required, but creating professional-looking reports is not easy. Therefore, a system is needed that allows users to easily enter information about communication outages, automatically and quickly generate outage reports based on that information, and reliably share them with the technical department.

[0864] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0865] In this invention, the server includes means for saving input data received from a user terminal, means for analyzing the input data received from the user terminal and automatically generating a problem report based on a prompt sentence using a generative AI model, means for sending the generated problem report to the user terminal, and means for comparing past problem data and a case database and adding necessary supplemental information to the generated report. This allows even non-technical personnel to easily input information about communication problems, automatically and quickly create problem reports that meet specifications, and share the information with the technical department.

[0866] A "user terminal" is a device that provides an interface for a user to use, and is used to input information about a communication failure and transmit it to a server.

[0867] The "server" is a central processing system that receives, analyzes, and stores data sent from the user terminal, generates a fault report using a generative AI model, and sends it back to the user terminal.

[0868] "Input data" is information sent from a user terminal to a server, including details about a communication failure.

[0869] A "generative AI model" is an artificial intelligence model that automatically generates a fault report based on data received from the user's device, and constructs the text of the report using prompt sentences.

[0870] A "prompt sentence" is the input sentence that serves as the basis for the generative AI model to generate a fault report, and indicates the details and conditions of the fault.

[0871] A "trouble report" is a document that compiles detailed information about a communication failure, and is created to accurately and promptly communicate the details of the failure to the technical department.

[0872] A "check box" is a GUI element that allows the user to input data in a format that allows the user to select, and indicates options that represent specific conditions of a communication failure.

[0873] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers. In particular, it enables sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with a server, the system of the present invention automatically generates outage reports from simple checkbox input, enabling them to be quickly shared with the technical department.

[0874] The user terminal provides a graphical user interface (GUI) that allows users to input information about communication disruptions in the form of check boxes. This check box-style input interface is designed to allow users to easily select information visually. For example, the user can select check boxes such as "communication disrupted," "specific areas only," and "specific time periods."

[0875] When a user selects a checkbox, the information is temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol (e.g., HTTPS). The server receives the data sent from the device and first stores it in temporary storage. The stored data is then analyzed using Python scripts or machine learning libraries (e.g., scikit-learn, TensorFlow).

[0876] The analyzed data is provided to a generative AI model installed on the server, which then automatically generates an outage report based on the prompt text. The generative AI model used here is, for example, OpenAI's GPT-3. An example of a prompt text might be, "Users have reported that communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[0877] The generated problem report is then sent back from the server to the user's device. This communication is also carried out using a secure protocol. The user then checks the problem report received on their device, inspects its contents, and makes any necessary corrections. Finally, the user sends the checked report back to the server and shares it with the technical department. The technical department then uses the received report to quickly analyze and resolve the problem.

[0878] This system allows users to generate simple and accurate fault reports without having to write technical documents, enabling quick and accurate information sharing with the technical department, which will improve customer satisfaction with regard to communication faults.

[0879] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0880] Step 1:

[0881] Input from the user terminal

[0882] The user uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, the user selects check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." This input data is temporarily stored in the device.

[0883] Input: User selected checkbox information

[0884] Output: Input data temporarily saved in the device

[0885] Specific operation: The user selects a checkbox in the GUI and enters conditions such as "communication has been lost," "specific areas only," and "specific time periods."

[0886] Step 2:

[0887] Sending and saving input data

[0888] The information of the checkboxes selected by the user is sent to the server in real time. The data is sent using a secure communication protocol such as HTTPS to ensure data safety. The server then stores the received data in temporary storage.

[0889] Input: Checkbox information sent from the device

[0890] Output: Data stored in temporary storage on the server

[0891] What it does: When a user selects a checkbox, the information is sent in real time to the server using an HTTPS request, and the server stores the data in temporary storage.

[0892] Step 3:

[0893] Data reception and analysis on the server

[0894] The server receives the data sent from the device and then converts it into a format that can be analyzed by AI. This process uses Python scripts and machine learning libraries (e.g., scikit-learn, TensorFlow). The data is classified by the contents of the checkboxes and generates appropriate keywords and phrases.

[0895] Input: Checkbox information stored on the server

[0896] Output: Data converted into a format that can be analyzed by AI

[0897] What it does: The Python script reads the data in JSON format, categorizes it based on the contents of the checkboxes, and extracts the required keywords and phrases.

[0898] Step 4:

[0899] AI-powered fault report generation

[0900] The converted data is provided to a generative AI model installed on the server. The AI ​​generates a fault report based on the prompt text, referring to the received data, past fault data, and a case database. OpenAI's GPT-3 is used as the generative AI model.

[0901] Input: Data converted into a format that can be analyzed by AI

[0902] Output: Generated problem report

[0903] Specific operation: The server sends a prompt to GPT-3 saying, "According to user reports..." and outputs a detailed error report.

[0904] Step 5:

[0905] Sending the generated report to the user's terminal

[0906] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[0907] Input: Generated problem report

[0908] Output: The problem report sent to the user's terminal

[0909] Specific operation: The generated report is sent from the server to the user terminal via a POST request, and the data received on the terminal is displayed in the GUI.

[0910] Step 6:

[0911] User review of report

[0912] The user then checks the received error report on the terminal, checks the contents, and makes any necessary corrections. This checking process is also performed through the GUI of the user terminal.

[0913] Input: A trouble report sent to the user's terminal.

[0914] Output: User-confirmed and corrected fault report

[0915] Specific operation: The user views the generated report on the terminal screen, edits the text if necessary, and clicks the confirmation button.

[0916] Step 7:

[0917] Sharing information with the technical department

[0918] Once the user has confirmed the issue, the report is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the issue. This information sharing is done using internal communication protocols and custom tools.

[0919] Input: User-confirmed and corrected fault report

[0920] Output: Fault report sent and shared with technical department

[0921] Specific operation: When the user presses the confirmation button, the report is sent to the server and a notification is automatically sent to the technical department's system. The technical department can then view the report on a dedicated dashboard and begin analysis.

[0922] (Application example 1)

[0923] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0924] In current security services, the procedures for security guards and security personnel to report emergencies are extremely complicated, making it difficult to respond in real time. In particular, personnel without technical expertise are required to communicate information accurately and quickly, but the systems to achieve this are not well-developed. As a result, false reports and delayed responses are likely to occur, hindering the rapid resolution of emergencies.

[0925] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0926] In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for analyzing the input data in the server and automatically generating a report using AI, means for transmitting the generated report to the user terminal, means for sharing the report confirmed by the user with the response department, means for enabling security guards and security personnel to use the system as an emergency reporting system, means for providing the terminal with a means for inputting information about the emergency in the form of check boxes, and means for referring to a database of past cases and adding supplemental information when generating a report using AI. This enables security guards and security personnel to quickly and accurately report emergencies and share information with the response team in real time.

[0927] "User terminals" are input / display devices such as smartphones and head-mounted displays used by security guards and other security personnel.

[0928] "Input data" is information about an emergency that is entered by a security guard or security officer using a user terminal. It includes checkbox items such as "fire," "suspicious person found," and "intrusion."

[0929] "Server" means a system that includes data processing equipment and software for receiving and analyzing input data sent from user terminals, and generating and transmitting reports.

[0930] "Analysis" is the process in which the server classifies the input data it receives and prepares it for the AI ​​to generate a report.

[0931] A "report" is an AI-generated document containing details of an emergency that is sent to a user's device for final review and sharing.

[0932] The "checkbox format" is a graphical user interface format that allows a user to select specific conditions or items.

[0933] An "AI model" is an artificial intelligence that uses technologies such as machine learning and deep learning to automatically generate reports based on input data.

[0934] A "past case database" is a system that stores data on previously recorded emergency situations, which the AI ​​model references when generating its reports.

[0935] An "emergency reporting system" is a system that allows security guards and other security personnel to easily report emergencies and share information with response teams in real time.

[0936] A "prompt" is a guided sentence used by an AI model to generate a detailed report in natural language.

[0937] This invention is a system that allows security guards and other security personnel to quickly report emergencies and share information with response teams in real time. The system includes a user terminal, a server, an AI model, and a database of past cases.

[0938] First, the user terminal is a device that allows security guards and other security personnel to input information about emergencies. The terminal's graphical user interface (GUI) displays check boxes indicating emergency situations such as "fire," "suspicious person detected," and "intrusion." Security guards select the appropriate check box and enter the necessary information.

[0939] The server then receives and stores the input data sent from the user device in real time. The stored data is protected using a secure communication protocol (SSL / TLS). The received data is first temporarily stored in storage, and then converted into a format that the AI ​​model can analyze.

[0940] The AI ​​model in the server references the received data and a database of past cases, and then uses the generative AI model to automatically generate a report. A detailed natural language report is created using prompts. For example, a prompt such as "A fire has broken out. Please enter the detailed location and time."

[0941] The generated report is then sent back to the user's device from the server, where it is reviewed by security guards and other security personnel. Once reviewed, the report is sent back to the server and shared with the appropriate department in real time, enabling a prompt and accurate response.

[0942] As a concrete example, if a security guard discovers a fire in a certain area, the user selects the "fire" checkbox in the GUI on their device and enters the detailed location and time. This information is sent to the server, and the AI ​​model generates a specific report stating, "A fire broke out in the office area at 2:00 PM." The generated report is sent to the security guard's device, where it is reviewed and shared with the relevant department.

[0943] The components of this system include a server deployed on an AWS EC2 instance, an AI model using Google Cloud AutoML, a database managed by Amazon RDS, and user devices including Android smartphones and Oculus Quest 2 (HMD). These elements work together to create an emergency reporting system that achieves the objective of the invention.

[0944] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0945] Step 1:

[0946] Input from the user terminal

[0947] The user (security guard or security officer) uses the terminal's graphical user interface (GUI) to input information about the emergency. They select specific check boxes (e.g., "fire," "suspicious person detected," "intrusion") and enter additional information such as detailed location and time. The input data is temporarily stored within the terminal.

[0948] Input: Emergency information (checkbox selection, detailed location, time)

[0949] Output: Temporarily saved data

[0950] Specific behavior:

[0951] The security guard selects the "Fire" checkbox and enters "Office Area" and "14:00".

[0952] Step 2:

[0953] Sending input data

[0954] The terminal transmits the temporarily stored input data to the server in real time using a secure communication protocol (SSL / TLS). The transmitted data reaches the server through a secure connection.

[0955] Input:Temporarily saved data

[0956] Output: Data sent to the server

[0957] Specific behavior:

[0958] The terminal sends the input data (selection of "fire", "office area", "14:00") to the server.

[0959] Step 3:

[0960] Receiving and temporarily storing data on the server

[0961] The server receives input data sent from the terminal and temporarily stores it in storage. It checks the format and content of the received data and temporarily stores it in preparation for converting it into the correct format.

[0962] Input: Data sent from the terminal

[0963] Output: Temporarily saved data

[0964] Specific behavior:

[0965] The server receives the data for "fire," "office area," and "14:00" and temporarily stores it.

[0966] Step 4:

[0967] Data analysis and preparation for input to AI models

[0968] The server then analyzes the temporarily stored data and converts it into a format that can be analyzed by the generative AI model. During this analysis process, the data is classified according to the contents of the checkboxes and appropriate keywords and phrases are generated.

[0969] Input:Temporarily saved data

[0970] Output: Parsed data

[0971] Specific behavior:

[0972] The server converts the data "fire", "office area", and "14:00" into "emergency: fire", "location: office area", and "time: 14:00".

[0973] Step 5:

[0974] AI-powered report generation

[0975] The AI ​​model installed on the server generates a detailed natural language report using prompt sentences based on the analyzed data and a database of past cases. The AI ​​model applies a generative AI model.

[0976] Input: Parsed data

[0977] Output: Generated report

[0978] Specific behavior:

[0979] The AI ​​generates a report stating, "A fire broke out in the office area at 14:00."

[0980] Step 6:

[0981] Sending generated reports

[0982] The report generated by the AI ​​is then sent from the server to the user's device, again using a secure communication protocol (SSL / TLS).

[0983] Input: Generated report

[0984] Output: Report sent to user terminal

[0985] Specific behavior:

[0986] A report "A fire broke out in the office area at 14:00" is sent from the server to the terminal.

[0987] Step 7:

[0988] User review of report

[0989] The user (security guard or security officer) checks the report received on the terminal, checks the contents, and makes corrections as necessary.

[0990] Input: Report sent to user terminal

[0991] Output: Confirmed and corrected report

[0992] Specific behavior:

[0993] The security guard will review the report and make corrections if necessary.

[0994] Step 8:

[0995] Share the report

[0996] The confirmed report is sent back to the server and shared in real time with the department in charge, who then promptly analyzes and responds to the problem based on the received report.

[0997] Input: Confirmed and corrected report

[0998] Output: Report shared with corresponding departments

[0999] Specific behavior:

[1000] The confirmed report is sent from the user terminal to the server, and the server shares it with the relevant department.

[1001] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1002] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers, and by combining it with an emotion engine that recognizes the user's emotions, it enables a more humane and appropriate response. By linking the user terminal with a server and emotion engine, the system of this invention automatically generates outage reports based on simple input in the form of checkboxes and an analysis of the user's emotions, making it possible to quickly share these reports with the technical department.

[1003] A natural language description of the program's processing

[1004] 1. Input from the user's device

[1005] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is also collected.

[1006] 2. Sending and saving input data and emotion data

[1007] The information on the checkboxes selected by the user and emotion data are temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol to ensure data safety.

[1008] 3. Data reception and analysis on the server

[1009] The server receives the data sent from the device. The received data is stored in temporary storage. Next, the input data and emotion data are converted into appropriate formats for data analysis. In this analysis process, the checkbox selections are converted into a format that can be input to the AI, and the emotion data is analyzed by the emotion engine.

[1010] 4. AI-based fault report generation

[1011] The AI ​​model on the server generates an outage report based on the analyzed data. The emotion engine's analysis results are used to adjust the tone and content of the report. For example, if the user is in a high stress state, a more detailed message urging a prompt response is generated. An example of a generated message is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1012] 5. Send the generated report to the user's device

[1013] The AI-generated fault report is sent from the server to the user's device, and this transmission also uses a secure communication protocol.

[1014] 6. User confirmation of report

[1015] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[1016] 7. Information sharing with the technical department

[1017] Once the user has confirmed the report, it is sent back to the server and shared with the technical department. The server then notifies the technical department of the report, including the analysis results of the emotion engine. This allows the technical department to analyze and resolve the problem while taking into account priorities based on the user's emotional state.

[1018] Specific examples of operation

[1019] For example, if a user (sales representative) receives information that communication was interrupted in a certain area during a specific time period, they can select checkboxes on their device's GUI, such as "Communication interrupted," "Only in a specific area," or "Specific time period." At this time, the user's emotional state is also recognized. The data is sent to the server in real time, where it is analyzed, and a specific problem report is generated using AI and an emotion engine. The report is then sent to the user's device, where the user confirms it and shares it with the technical department. In this way, users can easily generate accurate and appropriate problem reports based on their emotions, without having to write technical sentences. Furthermore, the technical department can quickly and appropriately resolve the problem by taking the user's emotional state into account.

[1020] As described above, the system of the present invention combines checkbox entry and emotion analysis to realize efficient and accurate fault reporting and information sharing, and aims to improve customer satisfaction with communication faults.

[1021] The processing flow will be explained below.

[1022] Step 1:

[1023] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, they select check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." The device also analyzes the user's facial expressions and tone of voice using an emotion engine, simultaneously acquiring emotional data.

[1024] Step 2:

[1025] The device temporarily stores the information about the checkboxes selected by the user and their emotional data, and transmits it to a server in real time using a secure communication protocol such as HTTPS to prevent data leakage or tampering.

[1026] Step 3:

[1027] The server receives the data sent from the device and stores it in temporary storage, where it is converted into an appropriate format that can be understood by the AI ​​and emotion engine in preparation for the analysis process.

[1028] Step 4:

[1029] As part of the analysis process, the server passes the input data to the AI ​​model and also passes the emotion data to the emotion engine. The AI ​​analyzes the input data and extracts the necessary information, while also referencing past failure data and case databases.

[1030] Step 5:

[1031] The emotion engine analyzes the user's emotional data and determines their emotional state (e.g., stress, anxiety, irritation, etc.). Based on this determination, the server adjusts the tone and content of the AI-generated report.

[1032] Step 6:

[1033] The server's AI model generates an outage report based on the analyzed data and the emotion engine's judgment. For example, it could generate a message like, "Users have reported that communication has been interrupted in a specific area and time period. Users are extremely dissatisfied. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1034] Step 7:

[1035] The server temporarily stores the generated problem report and prepares to send it to the user terminal. At this time, the generated report also includes the emotion analysis results.

[1036] Step 8:

[1037] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[1038] Step 9:

[1039] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[1040] Step 10:

[1041] The user resubmits the confirmed problem report from their device to the server and shares it with the engineering department. The server then notifies the engineering department of the report, including the analysis results of the emotion engine. This allows the engineering department to take action based on the user's emotional state.

[1042] Step 11:

[1043] The technical department will then promptly analyze and resolve the problem based on the received report. The results of the technical department's response will be fed back to the user's device via the server, and the final status will be shared with the user.

[1044] Example 2

[1045] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1046] Conventional communication outage reporting systems require users to manually create detailed reports, which requires a lot of time and effort. Furthermore, the content of the reports is often inconsistent, making it difficult to share information with the technical department. Furthermore, reports are generated in a fixed format without taking into account the user's emotional state, which can make it difficult to take prompt and appropriate action.

[1047] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1048] In this invention, the server includes means for storing input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for analyzing the input data and converting the data into an appropriate format in the server, means for analyzing the user's emotional data using an emotion analysis engine in the server, means for automatically generating a trouble report based on the analyzed data and the emotional data using AI, means for transmitting the generated trouble report to the user terminal, and means for sharing the trouble report confirmed by the user with the technical department. This allows the user to simply provide input data in a simple checkbox format, and accurate and consistent trouble reports generated in real time enable efficient information sharing with the technical department. Furthermore, emotion analysis allows for prompt and appropriate responses to be taken according to the user's emotional state.

[1049] A "user terminal" is a computer device that can be operated by a user and has an interface for inputting information about a communication failure and transmitting it to a server.

[1050] "Input data" refers to information relating to a communication failure that is input by a user via a terminal, and is data that includes selection items such as check boxes.

[1051] "Storage means" refers to a storage device or storage for temporarily or permanently storing input data and analyzed data.

[1052] "Transmission means" refers to a communication protocol or communication device for securely transmitting input data and generated reports to a user terminal or server.

[1053] A "server" is a computer system that analyzes data received from a user terminal and performs the necessary processing.

[1054] "Means of analysis" refers to software or algorithms that convert received input data into an appropriate format and analyze it using an AI model or sentiment analysis engine.

[1055] An "emotion analysis engine" is software or algorithms that analyze a user's emotional data and identify their state.

[1056] "Means for generating" refers to software or algorithms that use AI models to automatically generate incident reports based on analytical data and emotional data.

[1057] A "trouble report" is a document that describes detailed information about a communication failure, and includes analysis results and information that reflects the user's emotional state.

[1058] "Technical department" refers to the organization or team responsible for analyzing and resolving communication problems.

[1059] "Supplementary information" refers to relevant information added to reports generated based on past failure data and case databases.

[1060] The "checkbox format" is a GUI-style input format that allows users to easily enter options.

[1061] The present invention is a system that links a user terminal with a server and an emotion analysis engine to quickly and accurately generate reports about communication failures and share information with the technical department. This system is realized using the following hardware and software.

[1062] 1. Input via user terminal

[1063] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication outage. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is acquired. To acquire this emotional data, the device's sensors and camera are used to analyze facial expressions through facial recognition. This analysis uses common facial recognition algorithms and emotion analysis software.

[1064] 2. Sending input data and emotion data

[1065] The information about the checkboxes selected by the user and the emotion data are temporarily stored on the device. A secure communication protocol (e.g., HTTPS) is used to transmit the data to the server in real time. When the user enters information about the communication failure and presses the send button, the data is sent to the server in encrypted form.

[1066] 3. Data reception and analysis on the server

[1067] The server receives the data sent from the device and stores it in temporary storage. For data analysis, the input data is converted into a format that can be input to the AI, and the emotional data is analyzed by a sentiment analysis engine. This analysis process uses Python libraries (such as Pandas and SciPy). The sentiment analysis engine also uses sentiment analysis APIs from IBM Watson and Microsoft Azure.

[1068] 4. AI-based fault report generation

[1069] The AI ​​model on the server generates an outage report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. Specifically, a machine learning model using TensorFlow and PyTorch is used. If the user is in a high stress state, the report will include more detailed text encouraging a prompt response. An example of the generated text is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1070] 5. Send the generated report to the user's device

[1071] The AI-generated fault report is sent from the server to the user's device via a secure communication protocol (e.g., HTTPS).

[1072] 6. User confirmation of report

[1073] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary. After corrections are completed, the report is sent back to the server.

[1074] 7. Information sharing with the technical department

[1075] Once the user has confirmed and completed the corrections, the report is sent to the server and shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[1076] A specific example of a prompt might be, "We have received information that communication was disrupted in Shibuya Ward between 14:00 and 16:00. The user is feeling very stressed. Based on this information, please generate an outage report that reflects the user's emotional state." When this prompt is input into a generative AI model (e.g., OpenAI's GPT-4), an appropriate outage report is output.

[1077] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1078] Step 1:

[1079] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." In addition, the device's sensors and camera collect the user's emotional data (e.g., facial expression analysis).

[1080] Input: User selected checkbox data and emotion data

[1081] Output: Temporarily saved checkbox data and emotion data

[1082] Step 2:

[1083] The device sends the temporarily stored data to the server in real time using a secure communication protocol (e.g., HTTPS). When the user completes entering the information and presses the send button, the data is sent in encrypted form.

[1084] Input: Temporarily stored checkbox data and emotion data

[1085] Output: Checkbox data and emotion data sent to the server

[1086] Step 3:

[1087] The server receives the data sent from the device and stores it in temporary storage (e.g., a database). Next, for data analysis, the input data (checkbox data) is converted into a format that can be input to AI (e.g., JSON format), and the emotion data is analyzed by an emotion analysis engine.

[1088] Input: Checkbox data and emotion data sent to the server

[1089] Output: Data converted into a format that can be fed into AI and sentiment analysis engines

[1090] Step 4:

[1091] The emotion analysis engine analyzes the received emotion data. Specifically, it analyzes the user's facial expression data to identify their stress level and emotional state. The emotion analysis engine uses common facial recognition algorithms and emotion analysis APIs.

[1092] Input: Emotion data input to the emotion analysis engine

[1093] Output: Emotion analysis results (e.g., stress level, high or low, etc.)

[1094] Step 5:

[1095] The AI ​​model in the server generates an incident report based on the analyzed checkbox data and emotion data, and adjusts the content and tone of the report based on the results of the emotion analysis engine. For example, a machine learning model using TensorFlow or PyTorch is used.

[1096] Input: Parsed checkbox data and sentiment data

[1097] Output: Generated problem report

[1098] Step 6:

[1099] The generated problem report is sent from the server to the user terminal, and this transmission is also performed via a secure communication protocol (e.g., HTTPS).

[1100] Input: Generated problem report

[1101] Output: The problem report sent to the user's terminal

[1102] Step 7:

[1103] The user checks the received failure report on the terminal, checks the contents, corrects the report if necessary, and then sends the corrected report back to the server.

[1104] Input: A trouble report sent to the user's terminal.

[1105] Output: A confirmed and corrected problem report

[1106] Step 8:

[1107] Once the user has confirmed and completed the corrections, the report is sent to the server and ultimately shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[1108] Input: Verified and corrected problem report

[1109] Output: A fault report shared with the technical department

[1110] (Application example 2)

[1111] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1112] Conventional systems have the drawback of being time-consuming to report communication failures or security issues, and are unable to respond to them taking into account the user's emotional state. Furthermore, it is difficult to respond quickly because users may not be able to express the content of their reports accurately, or it may take a long time to report. Therefore, a system that combines emotion analysis is needed to provide appropriate responses that reflect the user's emotions.

[1113] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for the generative AI model to automatically generate a problem report, an emotion analysis engine that analyzes the user's emotional state and includes means for adjusting the tone and content of the generated report, means for transmitting the generated problem report to the user terminal, and means for sharing the problem report confirmed by the user with related departments. This enables quick and appropriate problem reporting and response according to the user's emotions.

[1114] A "user terminal" is a device used by a user to input and manipulate information.

[1115] "Input data" refers to information about a problem that is provided by a user via a user terminal.

[1116] A "storage means" is a mechanism for temporarily or permanently storing input data.

[1117] The "means for transmitting" is a mechanism for transferring input data from the user terminal to the server.

[1118] A "generative AI model" is an artificial intelligence algorithm that automatically generates problem reports based on input data.

[1119] A "problem report" is a document that details a problem generated by a generative AI model.

[1120] An "emotion analysis engine" is a software or hardware mechanism for analyzing a user's emotional state.

[1121] "Means to adjust tone and content" refers to a function that adjusts the wording of problem reports based on the results of the sentiment analysis engine.

[1122] "Sharing means" refers to a mechanism for sharing generated problem reports with related departments.

[1123] "Historical Problem Data" is information about previously recorded problems.

[1124] A "case database" is a database that collects and stores data on past problems and cases.

[1125] A "checkbox format" is a type of GUI that allows users to easily select and enter information.

[1126] "Emotion data" is data relating to the user's emotional state.

[1127] This invention provides an efficient and accurate problem reporting system by linking a user terminal, a server, and an emotion analysis engine. In particular, the use of the emotion analysis engine enables more human-like and appropriate problem reporting and responses according to the user's emotions.

[1128] Hardware and Software

[1129] The hardware and software used in this system will now be described in detail.

[1130] Hardware

[1131] Smartphones: iOS and Android devices

[1132] Smart glasses: Google Glass as an example

[1133] Head-mounted displays: An example is Microsoft HoloLens

[1134] software

[1135] Graphical User Interface (GUI) frameworks: Examples include React Native and Flutter

[1136] Secure communication protocols: e.g. TLS / SSL

[1137] Implementation of AI models: Examples include TensorFlow and PyTorch

[1138] Emotion engine: Examples include Microsoft Azure Emotion API and IBM Watson Tone Analyzer

[1139] Overall system flow

[1140] 1. Input from the user terminal

[1141] Users use the device's graphical user interface (GUI) to report security issues. Specifically, they input information by selecting checkboxes such as "door lock is broken," "surveillance camera footage has been lost," and "this only occurs at certain times of the day." The user's emotional data is also collected via the camera and microphone and analyzed by an emotion analysis engine.

[1142] 2. Sending and saving input data and emotion data

[1143] Data sent from the device is temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol (e.g., TLS / SSL).

[1144] 3. Data reception and analysis on the server

[1145] The server stores the data sent from the device in temporary storage and converts it into an appropriate format for analysis: checkbox selections are converted into a format understandable by the generative AI model, and emotional data is analyzed by the emotion analysis engine.

[1146] 4. AI-powered problem report generation

[1147] The generative AI model on the server generates a problem report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. For example, if the user is in a high stress state, a message urging a quick response is generated.

[1148] 5. Sending the generated report to the user's terminal

[1149] The generated problem report is transmitted to the user terminal, again using a secure communications protocol.

[1150] 6. User review of report

[1151] The user can check the received problem report on the terminal, check the contents, and make corrections if necessary.

[1152] 7. Information sharing with the Security Center

[1153] After verification, the report is sent back to the server and shared with the security center and related departments. The report also includes the results of the emotion analysis engine, allowing the center to analyze and resolve the problem while taking into account the user's emotional state.

[1154] Specific examples of operation

[1155] Consider a case where a facility manager reports a problem where "surveillance camera footage has been lost." From the terminal's GUI, the manager selects "surveillance camera footage has been lost," "specific area only," and "specific time period." At this time, the manager's emotional state is also recognized, and this data is sent to the server. The server analyzes the data and uses a generative AI model to generate a specific problem report, reflecting the emotional tone.

[1156] Examples of generated prompts include:

[1157] A user reported an issue where the surveillance camera footage was cut off. The issue occurred in a specific area and at a specific time of day. This caused great anxiety for the user. The details are as follows: Area - Building B1, Time - 18:00-20:00.

[1158] As described above, the system of the present invention takes into consideration the user's feelings, generates a prompt and appropriate problem report, and shares it with related departments, thereby enabling a quick response and high customer satisfaction.

[1159] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1160] Step 1:

[1161] Question input on user device

[1162] The user inputs information about the problem through the device's graphical user interface (GUI). Specifically, they select options in the form of check boxes, such as "The door lock is broken," "The surveillance camera footage has been lost," or "This only occurs at certain times of the day." At the same time, the user's emotional data is acquired from the device's camera and microphone. The input data and emotional data are temporarily stored on the device.

[1163] Input: Problem selection and user sentiment data

[1164] Output: Temporarily saved input data and emotion data

[1165] Step 2:

[1166] Sending data

[1167] The device sends the saved input data and emotion data to the server using a secure communication protocol (e.g., TLS / SSL). The device notifies the user that the data has been sent.

[1168] Input: Temporarily saved input data and emotion data

[1169] Output: Input data and emotion data sent to the server

[1170] Step 3:

[1171] Receiving and storing data on the server

[1172] The server receives the transmitted data and stores it in temporary storage. The received data is converted into an appropriate format for analysis, and checkbox selections are converted into a format that can be input into a generative AI model. Emotion data is sent to an emotion analysis engine to analyze the emotional state.

[1173] Input: Input data and emotion data sent to the server

[1174] Output: Input data and emotion data converted into a parsable format

[1175] Step 4:

[1176] Generate a problem report

[1177] The server-based generative AI model generates a problem report based on the analyzed input data and emotional data. The tone and content of the report are adjusted using the results of the emotion analysis engine. For example, if the user is in a high-stress state, the report may include a prompt to take action.

[1178] Input: Input data and emotion data converted into a parsable format

[1179] Output: Problem reports with tailored tone and content

[1180] Step 5:

[1181] Submitting and reviewing reports

[1182] The generated problem report is then sent from the server to the terminal using a secure communication protocol, where the user can review the report and make corrections as necessary.

[1183] Input: Coordinated problem report

[1184] Output: Problem report sent to user's terminal, including whether it has been fixed or not

[1185] Step 6:

[1186] Sharing the final report

[1187] After the user has checked and made any necessary corrections, the final report is sent back to the server and shared with the Security Center and relevant departments. The analysis results from the sentiment analysis engine are also shared. Based on this, the Security Center can take prompt and appropriate action.

[1188] Input: Final Problem Report Corrected

[1189] Output: Final problem report shared with security center and relevant departments

[1190] The above are the processing steps of the system program that realizes the application example.

[1191] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1192] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1193] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1194] [Fourth embodiment]

[1195] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1196] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1197] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1198] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1199] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1200] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1201] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1202] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1203] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1204] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1205] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1206] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1207] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1208] This invention is a system that responds quickly and accurately to complaints about communication failures from corporate customers, and it allows sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with the server, the system of this invention automatically generates a failure report from simple input in the form of check boxes, and makes it possible to quickly share the report with the technical department.

[1209] A natural language description of the program's processing

[1210] 1. Input from the user's device

[1211] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, specifically by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods."

[1212] 2. Send and save input data

[1213] The information of the checkboxes selected by the user is temporarily stored on the device and then sent to the server in real time. This transmission uses a secure communication protocol, so the safety of the data is ensured.

[1214] 3. Data reception and analysis on the server

[1215] The server receives the data sent from the device. The received data is first stored in temporary storage and then converted into a format that can be analyzed by AI. In this analysis process, the data is classified by the contents of the checkboxes and appropriate keywords and phrases are generated.

[1216] 4. AI-based fault report generation

[1217] The converted data is provided to an AI model installed on the server. The AI ​​generates an outage report by referencing the received data, past outage data, and a case database. For example, the generated text might read, "According to user reports, communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1218] 5. Send the generated report to the user's device

[1219] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[1220] 6. User confirmation of report

[1221] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[1222] 7. Information sharing with the technical department

[1223] Once the user has confirmed the report, it is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the problem.

[1224] Specific examples of operation

[1225] For example, if a user (sales representative) receives information that communication was disrupted in a certain area during a specific time period, they can select check boxes such as "communication disrupted," "specific area only," or "specific time period" from the GUI on their device. This input data is sent to the server in real time, which analyzes the data and generates a specific failure report using AI. The report is then sent to the user's device, where the user checks the contents and, if there are no problems, shares it with the technical department.

[1226] In this way, the system of the present invention allows users to easily generate accurate trouble reports without having to write technical documents, and enables quick and accurate information sharing with the technical department, thereby aiming to improve customer satisfaction with regard to communication troubles.

[1227] The processing flow will be explained below.

[1228] Step 1:

[1229] The user (sales representative) uses the terminal's graphical user interface (GUI) to input information about the communication failure, for example, by selecting check boxes such as "communication was interrupted," "only in a specific area," and "specific time period."

[1230] Step 2:

[1231] The device temporarily stores the information of the checkboxes selected by the user, and this data is stored in temporary memory or local storage for subsequent processing.

[1232] Step 3:

[1233] The device sends the stored data to the server in real time using secure communication protocols such as HTTPS to prevent data leakage and tampering.

[1234] Step 4:

[1235] The server receives the data sent from the device and stores it in temporary storage.

[1236] Step 5:

[1237] The server parses the incoming data and converts it into a format that the AI ​​can understand. This parsing process involves converting checkbox selections into keywords and phrases.

[1238] Step 6:

[1239] The server provides the analyzed data to the AI ​​model, which then compares the received data with past fault data and a case database.

[1240] Step 7:

[1241] The server's AI generates a problem report based on the provided data, such as "Users have reported that communication has been interrupted in a specific area and time period. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1242] Step 8:

[1243] The server temporarily stores the generated failure report and prepares it for transmission to the user terminal.

[1244] Step 9:

[1245] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[1246] Step 10:

[1247] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[1248] Step 11:

[1249] The user retransmits the confirmed problem report from the terminal to the server and shares it with the technical department, and the server notifies the technical department of the shared report.

[1250] Step 12:

[1251] The technical department will promptly analyze and resolve the problem based on the received report, and will then provide feedback on the results of the response to the user.

[1252] Example 1

[1253] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1254] In conventional communication outage response systems, it is difficult for non-technical sales representatives and customer service representatives to properly convey information about communication outages to the technical department, resulting in delays and mistakes in the response. Furthermore, to reduce the time and effort required to manually create outage reports, automation is required, but creating professional-looking reports is not easy. Therefore, a system is needed that allows users to easily enter information about communication outages, automatically and quickly generate outage reports based on that information, and reliably share them with the technical department.

[1255] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1256] In this invention, the server includes means for saving input data received from a user terminal, means for analyzing the input data received from the user terminal and automatically generating a problem report based on a prompt sentence using a generative AI model, means for sending the generated problem report to the user terminal, and means for comparing past problem data and a case database and adding necessary supplemental information to the generated report. This allows even non-technical personnel to easily input information about communication problems, automatically and quickly create problem reports that meet specifications, and share the information with the technical department.

[1257] A "user terminal" is a device that provides an interface for a user to use, and is used to input information about a communication failure and transmit it to a server.

[1258] The "server" is a central processing system that receives, analyzes, and stores data sent from the user terminal, generates a fault report using a generative AI model, and sends it back to the user terminal.

[1259] "Input data" is information sent from a user terminal to a server, including details about a communication failure.

[1260] A "generative AI model" is an artificial intelligence model that automatically generates a fault report based on data received from the user's device, and constructs the text of the report using prompt sentences.

[1261] A "prompt sentence" is the input sentence that serves as the basis for the generative AI model to generate a fault report, and indicates the details and conditions of the fault.

[1262] A "trouble report" is a document that compiles detailed information about a communication failure, and is created to accurately and promptly communicate the details of the failure to the technical department.

[1263] A "check box" is a GUI element that allows the user to input data in a format that allows the user to select, and indicates options that represent specific conditions of a communication failure.

[1264] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers. In particular, it enables sales representatives and inquiry staff who do not have technical expertise to efficiently share information with the technical department. By linking the user terminal with a server, the system of the present invention automatically generates outage reports from simple checkbox input, enabling them to be quickly shared with the technical department.

[1265] The user terminal provides a graphical user interface (GUI) that allows users to input information about communication disruptions in the form of check boxes. This check box-style input interface is designed to allow users to easily select information visually. For example, the user can select check boxes such as "communication disrupted," "specific areas only," and "specific time periods."

[1266] When a user selects a checkbox, the information is temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol (e.g., HTTPS). The server receives the data sent from the device and first stores it in temporary storage. The stored data is then analyzed using Python scripts or machine learning libraries (e.g., scikit-learn, TensorFlow).

[1267] The analyzed data is provided to a generative AI model installed on the server, which then automatically generates an outage report based on the prompt text. The generative AI model used here is, for example, OpenAI's GPT-3. An example of a prompt text might be, "Users have reported that communication outages are occurring in specific areas and time periods. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1268] The generated problem report is then sent back from the server to the user's device. This communication is also carried out using a secure protocol. The user then checks the problem report received on their device, inspects its contents, and makes any necessary corrections. Finally, the user sends the checked report back to the server and shares it with the technical department. The technical department then uses the received report to quickly analyze and resolve the problem.

[1269] This system allows users to generate simple and accurate fault reports without having to write technical documents, enabling quick and accurate information sharing with the technical department, which will improve customer satisfaction with regard to communication faults.

[1270] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1271] Step 1:

[1272] Input from the user terminal

[1273] The user uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, the user selects check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." This input data is temporarily stored in the device.

[1274] Input: User selected checkbox information

[1275] Output: Input data temporarily saved in the device

[1276] Specific operation: The user selects a checkbox in the GUI and enters conditions such as "communication has been lost," "specific areas only," and "specific time periods."

[1277] Step 2:

[1278] Sending and saving input data

[1279] The information of the checkboxes selected by the user is sent to the server in real time. The data is sent using a secure communication protocol such as HTTPS to ensure data safety. The server then stores the received data in temporary storage.

[1280] Input: Checkbox information sent from the device

[1281] Output: Data stored in temporary storage on the server

[1282] What it does: When a user selects a checkbox, the information is sent in real time to the server using an HTTPS request, and the server stores the data in temporary storage.

[1283] Step 3:

[1284] Data reception and analysis on the server

[1285] The server receives the data sent from the device and then converts it into a format that can be analyzed by AI. This process uses Python scripts and machine learning libraries (e.g., scikit-learn, TensorFlow). The data is classified by the contents of the checkboxes and generates appropriate keywords and phrases.

[1286] Input: Checkbox information stored on the server

[1287] Output: Data converted into a format that can be analyzed by AI

[1288] What it does: The Python script reads the data in JSON format, categorizes it based on the contents of the checkboxes, and extracts the required keywords and phrases.

[1289] Step 4:

[1290] AI-powered fault report generation

[1291] The converted data is provided to a generative AI model installed on the server. The AI ​​generates a fault report based on the prompt text, referring to the received data, past fault data, and a case database. OpenAI's GPT-3 is used as the generative AI model.

[1292] Input: Data converted into a format that can be analyzed by AI

[1293] Output: Generated problem report

[1294] Specific operation: The server sends a prompt to GPT-3 saying, "According to user reports..." and outputs a detailed error report.

[1295] Step 5:

[1296] Sending the generated report to the user's terminal

[1297] The AI-generated fault report is then sent from the server to the user's device, again using a secure communication protocol.

[1298] Input: Generated problem report

[1299] Output: The problem report sent to the user's terminal

[1300] Specific operation: The generated report is sent from the server to the user terminal via a POST request, and the data received on the terminal is displayed in the GUI.

[1301] Step 6:

[1302] User review of report

[1303] The user then checks the received error report on the terminal, checks the contents, and makes any necessary corrections. This checking process is also performed through the GUI of the user terminal.

[1304] Input: A trouble report sent to the user's terminal.

[1305] Output: User-confirmed and corrected fault report

[1306] Specific operation: The user views the generated report on the terminal screen, edits the text if necessary, and clicks the confirmation button.

[1307] Step 7:

[1308] Sharing information with the technical department

[1309] Once the user has confirmed the issue, the report is sent back to the server and shared with the technical department, who then uses the report to quickly analyze and resolve the issue. This information sharing is done using internal communication protocols and custom tools.

[1310] Input: User-confirmed and corrected fault report

[1311] Output: Fault report sent and shared with technical department

[1312] Specific operation: When the user presses the confirmation button, the report is sent to the server and a notification is automatically sent to the technical department's system. The technical department can then view the report on a dedicated dashboard and begin analysis.

[1313] (Application example 1)

[1314] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1315] In current security services, the procedures for security guards and security personnel to report emergencies are extremely complicated, making it difficult to respond in real time. In particular, personnel without technical expertise are required to communicate information accurately and quickly, but the systems to achieve this are not well-developed. As a result, false reports and delayed responses are likely to occur, hindering the rapid resolution of emergencies.

[1316] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1317] In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for analyzing the input data in the server and automatically generating a report using AI, means for transmitting the generated report to the user terminal, means for sharing the report confirmed by the user with the response department, means for enabling security guards and security personnel to use the system as an emergency reporting system, means for providing the terminal with a means for inputting information about the emergency in the form of check boxes, and means for referring to a database of past cases and adding supplemental information when generating a report using AI. This enables security guards and security personnel to quickly and accurately report emergencies and share information with the response team in real time.

[1318] "User terminals" are input / display devices such as smartphones and head-mounted displays used by security guards and other security personnel.

[1319] "Input data" is information about an emergency that is entered by a security guard or security officer using a user terminal. It includes checkbox items such as "fire," "suspicious person found," and "intrusion."

[1320] "Server" means a system that includes data processing equipment and software for receiving and analyzing input data sent from user terminals, and generating and transmitting reports.

[1321] "Analysis" is the process in which the server classifies the input data it receives and prepares it for the AI ​​to generate a report.

[1322] A "report" is an AI-generated document containing details of an emergency that is sent to a user's device for final review and sharing.

[1323] The "checkbox format" is a graphical user interface format that allows a user to select specific conditions or items.

[1324] An "AI model" is an artificial intelligence that uses technologies such as machine learning and deep learning to automatically generate reports based on input data.

[1325] A "past case database" is a system that stores data on previously recorded emergency situations, which the AI ​​model references when generating its reports.

[1326] An "emergency reporting system" is a system that allows security guards and other security personnel to easily report emergencies and share information with response teams in real time.

[1327] A "prompt" is a guided sentence used by an AI model to generate a detailed report in natural language.

[1328] This invention is a system that allows security guards and other security personnel to quickly report emergencies and share information with response teams in real time. The system includes a user terminal, a server, an AI model, and a database of past cases.

[1329] First, the user terminal is a device that allows security guards and other security personnel to input information about emergencies. The terminal's graphical user interface (GUI) displays check boxes indicating emergency situations such as "fire," "suspicious person detected," and "intrusion." Security guards select the appropriate check box and enter the necessary information.

[1330] The server then receives and stores the input data sent from the user device in real time. The stored data is protected using a secure communication protocol (SSL / TLS). The received data is first temporarily stored in storage, and then converted into a format that the AI ​​model can analyze.

[1331] The AI ​​model in the server references the received data and a database of past cases, and then uses the generative AI model to automatically generate a report. A detailed natural language report is created using prompts. For example, a prompt such as "A fire has broken out. Please enter the detailed location and time."

[1332] The generated report is then sent back to the user's device from the server, where it is reviewed by security guards and other security personnel. Once reviewed, the report is sent back to the server and shared with the appropriate department in real time, enabling a prompt and accurate response.

[1333] As a concrete example, if a security guard discovers a fire in a certain area, the user selects the "fire" checkbox in the GUI on their device and enters the detailed location and time. This information is sent to the server, and the AI ​​model generates a specific report stating, "A fire broke out in the office area at 2:00 PM." The generated report is sent to the security guard's device, where it is reviewed and shared with the relevant department.

[1334] The components of this system include a server deployed on an AWS EC2 instance, an AI model using Google Cloud AutoML, a database managed by Amazon RDS, and user devices including Android smartphones and Oculus Quest 2 (HMD). These elements work together to create an emergency reporting system that achieves the objective of the invention.

[1335] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1336] Step 1:

[1337] Input from the user terminal

[1338] The user (security guard or security officer) uses the terminal's graphical user interface (GUI) to input information about the emergency. They select specific check boxes (e.g., "fire," "suspicious person detected," "intrusion") and enter additional information such as detailed location and time. The input data is temporarily stored within the terminal.

[1339] Input: Emergency information (checkbox selection, detailed location, time)

[1340] Output: Temporarily saved data

[1341] Specific behavior:

[1342] The security guard selects the "Fire" checkbox and enters "Office Area" and "14:00".

[1343] Step 2:

[1344] Sending input data

[1345] The terminal transmits the temporarily stored input data to the server in real time using a secure communication protocol (SSL / TLS). The transmitted data reaches the server through a secure connection.

[1346] Input:Temporarily saved data

[1347] Output: Data sent to the server

[1348] Specific behavior:

[1349] The terminal sends the input data (selection of "fire", "office area", "14:00") to the server.

[1350] Step 3:

[1351] Receiving and temporarily storing data on the server

[1352] The server receives input data sent from the terminal and temporarily stores it in storage. It checks the format and content of the received data and temporarily stores it in preparation for converting it into the correct format.

[1353] Input: Data sent from the terminal

[1354] Output: Temporarily saved data

[1355] Specific behavior:

[1356] The server receives the data for "fire," "office area," and "14:00" and temporarily stores it.

[1357] Step 4:

[1358] Data analysis and preparation for input to AI models

[1359] The server then analyzes the temporarily stored data and converts it into a format that can be analyzed by the generative AI model. During this analysis process, the data is classified according to the contents of the checkboxes and appropriate keywords and phrases are generated.

[1360] Input:Temporarily saved data

[1361] Output: Parsed data

[1362] Specific behavior:

[1363] The server converts the data "fire", "office area", and "14:00" into "emergency: fire", "location: office area", and "time: 14:00".

[1364] Step 5:

[1365] AI-powered report generation

[1366] The AI ​​model installed on the server generates a detailed natural language report using prompt sentences based on the analyzed data and a database of past cases. The AI ​​model applies a generative AI model.

[1367] Input: Parsed data

[1368] Output: Generated report

[1369] Specific behavior:

[1370] The AI ​​generates a report stating, "A fire broke out in the office area at 14:00."

[1371] Step 6:

[1372] Sending generated reports

[1373] The report generated by the AI ​​is then sent from the server to the user's device, again using a secure communication protocol (SSL / TLS).

[1374] Input: Generated report

[1375] Output: Report sent to user terminal

[1376] Specific behavior:

[1377] A report "A fire broke out in the office area at 14:00" is sent from the server to the terminal.

[1378] Step 7:

[1379] User review of report

[1380] The user (security guard or security officer) checks the report received on the terminal, checks the contents, and makes corrections as necessary.

[1381] Input: Report sent to user terminal

[1382] Output: Confirmed and corrected report

[1383] Specific behavior:

[1384] The security guard will review the report and make corrections if necessary.

[1385] Step 8:

[1386] Share the report

[1387] The confirmed report is sent back to the server and shared in real time with the department in charge, who then promptly analyzes and responds to the problem based on the received report.

[1388] Input: Confirmed and corrected report

[1389] Output: Report shared with corresponding departments

[1390] Specific behavior:

[1391] The confirmed report is sent from the user terminal to the server, and the server shares it with the relevant department.

[1392] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1393] This invention is a system that responds quickly and accurately to complaints about communication outages from corporate customers, and by combining it with an emotion engine that recognizes the user's emotions, it enables a more humane and appropriate response. By linking the user terminal with a server and emotion engine, the system of this invention automatically generates outage reports based on simple input in the form of checkboxes and an analysis of the user's emotions, making it possible to quickly share these reports with the technical department.

[1394] A natural language description of the program's processing

[1395] 1. Input from the user's device

[1396] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is also collected.

[1397] 2. Sending and saving input data and emotion data

[1398] The information on the checkboxes selected by the user and emotion data are temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol to ensure data safety.

[1399] 3. Data reception and analysis on the server

[1400] The server receives the data sent from the device. The received data is stored in temporary storage. Next, the input data and emotion data are converted into appropriate formats for data analysis. In this analysis process, the checkbox selections are converted into a format that can be input to the AI, and the emotion data is analyzed by the emotion engine.

[1401] 4. AI-based fault report generation

[1402] The AI ​​model on the server generates an outage report based on the analyzed data. The emotion engine's analysis results are used to adjust the tone and content of the report. For example, if the user is in a high stress state, a more detailed message urging a prompt response is generated. An example of a generated message is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1403] 5. Send the generated report to the user's device

[1404] The AI-generated fault report is sent from the server to the user's device, and this transmission also uses a secure communication protocol.

[1405] 6. User confirmation of report

[1406] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary.

[1407] 7. Information sharing with the technical department

[1408] Once the user has confirmed the report, it is sent back to the server and shared with the technical department. The server then notifies the technical department of the report, including the analysis results of the emotion engine. This allows the technical department to analyze and resolve the problem while taking into account priorities based on the user's emotional state.

[1409] Specific examples of operation

[1410] For example, if a user (sales representative) receives information that communication was interrupted in a certain area during a specific time period, they can select checkboxes on their device's GUI, such as "Communication interrupted," "Only in a specific area," or "Specific time period." At this time, the user's emotional state is also recognized. The data is sent to the server in real time, where it is analyzed, and a specific problem report is generated using AI and an emotion engine. The report is then sent to the user's device, where the user confirms it and shares it with the technical department. In this way, users can easily generate accurate and appropriate problem reports based on their emotions, without having to write technical sentences. Furthermore, the technical department can quickly and appropriately resolve the problem by taking the user's emotional state into account.

[1411] As described above, the system of the present invention combines checkbox entry and emotion analysis to realize efficient and accurate fault reporting and information sharing, and aims to improve customer satisfaction with communication faults.

[1412] The processing flow will be explained below.

[1413] Step 1:

[1414] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, they select check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." The device also analyzes the user's facial expressions and tone of voice using an emotion engine, simultaneously acquiring emotional data.

[1415] Step 2:

[1416] The device temporarily stores the information about the checkboxes selected by the user and their emotional data, and transmits it to a server in real time using a secure communication protocol such as HTTPS to prevent data leakage or tampering.

[1417] Step 3:

[1418] The server receives the data sent from the device and stores it in temporary storage, where it is converted into an appropriate format that can be understood by the AI ​​and emotion engine in preparation for the analysis process.

[1419] Step 4:

[1420] As part of the analysis process, the server passes the input data to the AI ​​model and also passes the emotion data to the emotion engine. The AI ​​analyzes the input data and extracts the necessary information, while also referencing past failure data and case databases.

[1421] Step 5:

[1422] The emotion engine analyzes the user's emotional data and determines their emotional state (e.g., stress, anxiety, irritation, etc.). Based on this determination, the server adjusts the tone and content of the AI-generated report.

[1423] Step 6:

[1424] The server's AI model generates an outage report based on the analyzed data and the emotion engine's judgment. For example, it could generate a message like, "Users have reported that communication has been interrupted in a specific area and time period. Users are extremely dissatisfied. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1425] Step 7:

[1426] The server temporarily stores the generated problem report and prepares to send it to the user terminal. At this time, the generated report also includes the emotion analysis results.

[1427] Step 8:

[1428] The server then transmits the generated problem report to the user terminal, again using a secure communication protocol.

[1429] Step 9:

[1430] The user checks the error report received on the terminal, checks whether there are any problems with the content, and makes corrections if necessary.

[1431] Step 10:

[1432] The user resubmits the confirmed problem report from their device to the server and shares it with the engineering department. The server then notifies the engineering department of the report, including the analysis results of the emotion engine. This allows the engineering department to take action based on the user's emotional state.

[1433] Step 11:

[1434] The technical department will then promptly analyze and resolve the problem based on the received report. The results of the technical department's response will be fed back to the user's device via the server, and the final status will be shared with the user.

[1435] Example 2

[1436] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1437] Conventional communication outage reporting systems require users to manually create detailed reports, which requires a lot of time and effort. Furthermore, the content of the reports is often inconsistent, making it difficult to share information with the technical department. Furthermore, reports are generated in a fixed format without taking into account the user's emotional state, which can make it difficult to take prompt and appropriate action.

[1438] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1439] In this invention, the server includes means for storing input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for analyzing the input data and converting the data into an appropriate format in the server, means for analyzing the user's emotional data using an emotion analysis engine in the server, means for automatically generating a trouble report based on the analyzed data and the emotional data using AI, means for transmitting the generated trouble report to the user terminal, and means for sharing the trouble report confirmed by the user with the technical department. This allows the user to simply provide input data in a simple checkbox format, and accurate and consistent trouble reports generated in real time enable efficient information sharing with the technical department. Furthermore, emotion analysis allows for prompt and appropriate responses to be taken according to the user's emotional state.

[1440] A "user terminal" is a computer device that can be operated by a user and has an interface for inputting information about a communication failure and transmitting it to a server.

[1441] "Input data" refers to information relating to a communication failure that is input by a user via a terminal, and is data that includes selection items such as check boxes.

[1442] "Storage means" refers to a storage device or storage for temporarily or permanently storing input data and analyzed data.

[1443] "Transmission means" refers to a communication protocol or communication device for securely transmitting input data and generated reports to a user terminal or server.

[1444] A "server" is a computer system that analyzes data received from a user terminal and performs the necessary processing.

[1445] "Means of analysis" refers to software or algorithms that convert received input data into an appropriate format and analyze it using an AI model or sentiment analysis engine.

[1446] An "emotion analysis engine" is software or algorithms that analyze a user's emotional data and identify their state.

[1447] "Means for generating" refers to software or algorithms that use AI models to automatically generate incident reports based on analytical data and emotional data.

[1448] A "trouble report" is a document that describes detailed information about a communication failure, and includes analysis results and information that reflects the user's emotional state.

[1449] "Technical department" refers to the organization or team responsible for analyzing and resolving communication problems.

[1450] "Supplementary information" refers to relevant information added to reports generated based on past failure data and case databases.

[1451] The "checkbox format" is a GUI-style input format that allows users to easily enter options.

[1452] The present invention is a system that links a user terminal with a server and an emotion analysis engine to quickly and accurately generate reports about communication failures and share information with the technical department. This system is realized using the following hardware and software.

[1453] 1. Input via user terminal

[1454] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication outage. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." At the same time, the user's emotional data is acquired. To acquire this emotional data, the device's sensors and camera are used to analyze facial expressions through facial recognition. This analysis uses common facial recognition algorithms and emotion analysis software.

[1455] 2. Sending input data and emotion data

[1456] The information about the checkboxes selected by the user and the emotion data are temporarily stored on the device. A secure communication protocol (e.g., HTTPS) is used to transmit the data to the server in real time. When the user enters information about the communication failure and presses the send button, the data is sent to the server in encrypted form.

[1457] 3. Data reception and analysis on the server

[1458] The server receives the data sent from the device and stores it in temporary storage. For data analysis, the input data is converted into a format that can be input to the AI, and the emotional data is analyzed by a sentiment analysis engine. This analysis process uses Python libraries (such as Pandas and SciPy). The sentiment analysis engine also uses sentiment analysis APIs from IBM Watson and Microsoft Azure.

[1459] 4. AI-based fault report generation

[1460] The AI ​​model on the server generates an outage report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. Specifically, a machine learning model using TensorFlow and PyTorch is used. If the user is in a high stress state, the report will include more detailed text encouraging a prompt response. An example of the generated text is, "Users are reporting communication outages in specific areas and time periods. Users are very frustrated. Details are as follows: Area - Shibuya Ward, Time - 14:00-16:00."

[1461] 5. Send the generated report to the user's device

[1462] The AI-generated fault report is sent from the server to the user's device via a secure communication protocol (e.g., HTTPS).

[1463] 6. User confirmation of report

[1464] The user checks the error report received on the terminal, checks the contents, and makes corrections as necessary. After corrections are completed, the report is sent back to the server.

[1465] 7. Information sharing with the technical department

[1466] Once the user has confirmed and completed the corrections, the report is sent to the server and shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[1467] A specific example of a prompt might be, "We have received information that communication was disrupted in Shibuya Ward between 14:00 and 16:00. The user is feeling very stressed. Based on this information, please generate an outage report that reflects the user's emotional state." When this prompt is input into a generative AI model (e.g., OpenAI's GPT-4), an appropriate outage report is output.

[1468] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1469] Step 1:

[1470] The user (sales representative) uses the device's graphical user interface (GUI) to input information about the communication failure. Specifically, this is done by selecting check boxes such as "communication was interrupted," "only in specific areas," and "specific time periods." In addition, the device's sensors and camera collect the user's emotional data (e.g., facial expression analysis).

[1471] Input: User selected checkbox data and emotion data

[1472] Output: Temporarily saved checkbox data and emotion data

[1473] Step 2:

[1474] The device sends the temporarily stored data to the server in real time using a secure communication protocol (e.g., HTTPS). When the user completes entering the information and presses the send button, the data is sent in encrypted form.

[1475] Input: Temporarily stored checkbox data and emotion data

[1476] Output: Checkbox data and emotion data sent to the server

[1477] Step 3:

[1478] The server receives the data sent from the device and stores it in temporary storage (e.g., a database). Next, for data analysis, the input data (checkbox data) is converted into a format that can be input to AI (e.g., JSON format), and the emotion data is analyzed by an emotion analysis engine.

[1479] Input: Checkbox data and emotion data sent to the server

[1480] Output: Data converted into a format that can be fed into AI and sentiment analysis engines

[1481] Step 4:

[1482] The emotion analysis engine analyzes the received emotion data. Specifically, it analyzes the user's facial expression data to identify their stress level and emotional state. The emotion analysis engine uses common facial recognition algorithms and emotion analysis APIs.

[1483] Input: Emotion data input to the emotion analysis engine

[1484] Output: Emotion analysis results (e.g., stress level, high or low, etc.)

[1485] Step 5:

[1486] The AI ​​model in the server generates an incident report based on the analyzed checkbox data and emotion data, and adjusts the content and tone of the report based on the results of the emotion analysis engine. For example, a machine learning model using TensorFlow or PyTorch is used.

[1487] Input: Parsed checkbox data and sentiment data

[1488] Output: Generated problem report

[1489] Step 6:

[1490] The generated problem report is sent from the server to the user terminal, and this transmission is also performed via a secure communication protocol (e.g., HTTPS).

[1491] Input: Generated problem report

[1492] Output: The problem report sent to the user's terminal

[1493] Step 7:

[1494] The user checks the received failure report on the terminal, checks the contents, corrects the report if necessary, and then sends the corrected report back to the server.

[1495] Input: A trouble report sent to the user's terminal.

[1496] Output: A confirmed and corrected problem report

[1497] Step 8:

[1498] Once the user has confirmed and completed the corrections, the report is sent to the server and ultimately shared with the technical department. The server then notifies the technical department of the report, including the results of the sentiment analysis engine. The technical department then analyzes and resolves the problem based on this information and prioritizes it.

[1499] Input: Verified and corrected problem report

[1500] Output: A fault report shared with the technical department

[1501] (Application example 2)

[1502] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1503] Conventional systems have the drawback of being time-consuming to report communication failures or security issues, and are unable to respond to them taking into account the user's emotional state. Furthermore, it is difficult to respond quickly because users may not be able to express the content of their reports accurately, or it may take a long time to report. Therefore, a system that combines emotion analysis is needed to provide appropriate responses that reflect the user's emotions.

[1504] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for saving input data received from a user terminal, means for transmitting the input data received from the user terminal to the server, means for the generative AI model to automatically generate a problem report, an emotion analysis engine that analyzes the user's emotional state and includes means for adjusting the tone and content of the generated report, means for transmitting the generated problem report to the user terminal, and means for sharing the problem report confirmed by the user with related departments. This enables quick and appropriate problem reporting and response according to the user's emotions.

[1505] A "user terminal" is a device used by a user to input and manipulate information.

[1506] "Input data" refers to information about a problem that is provided by a user via a user terminal.

[1507] A "storage means" is a mechanism for temporarily or permanently storing input data.

[1508] The "means for transmitting" is a mechanism for transferring input data from the user terminal to the server.

[1509] A "generative AI model" is an artificial intelligence algorithm that automatically generates problem reports based on input data.

[1510] A "problem report" is a document that details a problem generated by a generative AI model.

[1511] An "emotion analysis engine" is a software or hardware mechanism for analyzing a user's emotional state.

[1512] "Means to adjust tone and content" refers to a function that adjusts the wording of problem reports based on the results of the sentiment analysis engine.

[1513] "Sharing means" refers to a mechanism for sharing generated problem reports with related departments.

[1514] "Historical Problem Data" is information about previously recorded problems.

[1515] A "case database" is a database that collects and stores data on past problems and cases.

[1516] A "checkbox format" is a type of GUI that allows users to easily select and enter information.

[1517] "Emotion data" is data relating to the user's emotional state.

[1518] This invention provides an efficient and accurate problem reporting system by linking a user terminal, a server, and an emotion analysis engine. In particular, the use of the emotion analysis engine enables more human-like and appropriate problem reporting and responses according to the user's emotions.

[1519] Hardware and Software

[1520] The hardware and software used in this system will now be described in detail.

[1521] Hardware

[1522] Smartphones: iOS and Android devices

[1523] Smart glasses: Google Glass as an example

[1524] Head-mounted displays: An example is Microsoft HoloLens

[1525] software

[1526] Graphical User Interface (GUI) frameworks: Examples include React Native and Flutter

[1527] Secure communication protocols: e.g. TLS / SSL

[1528] Implementation of AI models: Examples include TensorFlow and PyTorch

[1529] Emotion engine: Examples include Microsoft Azure Emotion API and IBM Watson Tone Analyzer

[1530] Overall system flow

[1531] 1. Input from the user terminal

[1532] Users use the device's graphical user interface (GUI) to report security issues. Specifically, they input information by selecting checkboxes such as "door lock is broken," "surveillance camera footage has been lost," and "this only occurs at certain times of the day." The user's emotional data is also collected via the camera and microphone and analyzed by an emotion analysis engine.

[1533] 2. Sending and saving input data and emotion data

[1534] Data sent from the device is temporarily stored on the device and then transmitted to the server in real time using a secure communication protocol (e.g., TLS / SSL).

[1535] 3. Data reception and analysis on the server

[1536] The server stores the data sent from the device in temporary storage and converts it into an appropriate format for analysis: checkbox selections are converted into a format understandable by the generative AI model, and emotional data is analyzed by the emotion analysis engine.

[1537] 4. AI-powered problem report generation

[1538] The generative AI model in the server generates a problem report based on the analyzed data. The tone and content of the report are adjusted using the results of the sentiment analysis engine. For example, if the user is in a high stress state, a message urging a quick response is generated.

[1539] 5. Sending the generated report to the user's terminal

[1540] The generated problem report is transmitted to the user terminal, again using a secure communications protocol.

[1541] 6. User review of report

[1542] The user can check the received problem report on the terminal, check the contents, and make corrections if necessary.

[1543] 7. Information sharing with the Security Center

[1544] After verification, the report is sent back to the server and shared with the security center and related departments. The report also includes the results of the emotion analysis engine, allowing the center to analyze and resolve the problem while taking into account the user's emotional state.

[1545] Specific examples of operation

[1546] Consider a case where a facility manager reports a problem where "surveillance camera footage has been lost." From the terminal's GUI, the manager selects "surveillance camera footage has been lost," "specific area only," and "specific time period." At this time, the manager's emotional state is also recognized, and this data is sent to the server. The server analyzes the data and uses a generative AI model to generate a specific problem report, reflecting the emotional tone.

[1547] Examples of generated prompts include:

[1548] A user reported an issue where the surveillance camera footage was cut off. The issue occurred in a specific area and at a specific time of day. This caused great anxiety for the user. The details are as follows: Area - Building B1, Time - 18:00-20:00.

[1549] As described above, the system of the present invention takes into consideration the user's feelings, generates a prompt and appropriate problem report, and shares it with related departments, thereby enabling a quick response and high customer satisfaction.

[1550] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1551] Step 1:

[1552] Question input on user device

[1553] The user inputs information about the problem through the device's graphical user interface (GUI). Specifically, they select options in the form of check boxes, such as "The door lock is broken," "The surveillance camera footage has been lost," or "This only occurs at certain times of the day." At the same time, the user's emotional data is acquired from the device's camera and microphone. The input data and emotional data are temporarily stored on the device.

[1554] Input: Problem selection and user sentiment data

[1555] Output: Temporarily saved input data and emotion data

[1556] Step 2:

[1557] Sending data

[1558] The device sends the saved input data and emotion data to the server using a secure communication protocol (e.g., TLS / SSL). The device notifies the user that the data has been sent.

[1559] Input: Temporarily saved input data and emotion data

[1560] Output: Input data and emotion data sent to the server

[1561] Step 3:

[1562] Receiving and storing data on the server

[1563] The server receives the transmitted data and stores it in temporary storage. The received data is converted into an appropriate format for analysis, and checkbox selections are converted into a format that can be input into a generative AI model. Emotion data is sent to an emotion analysis engine to analyze the emotional state.

[1564] Input: Input data and emotion data sent to the server

[1565] Output: Input data and emotion data converted into a parsable format

[1566] Step 4:

[1567] Generate a problem report

[1568] The server-based generative AI model generates a problem report based on the analyzed input data and emotional data. The tone and content of the report are adjusted using the results of the emotion analysis engine. For example, if the user is in a high-stress state, the report may include a prompt to take action.

[1569] Input: Input data and emotion data converted into a parsable format

[1570] Output: Problem reports with tailored tone and content

[1571] Step 5:

[1572] Submitting and reviewing reports

[1573] The generated problem report is then sent from the server to the terminal using a secure communication protocol, where the user can review the report and make corrections as necessary.

[1574] Input: Coordinated problem report

[1575] Output: Problem report sent to user's terminal, including whether it has been fixed or not

[1576] Step 6:

[1577] Sharing the final report

[1578] After the user has checked and made any necessary corrections, the final report is sent back to the server and shared with the Security Center and relevant departments. The analysis results from the sentiment analysis engine are also shared. Based on this, the Security Center can take prompt and appropriate action.

[1579] Input: Final Problem Report Corrected

[1580] Output: Final problem report shared with security center and relevant departments

[1581] The above are the processing steps of the system program that realizes the application example.

[1582] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1583] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1584] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1585] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1586] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1587] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1588] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1589] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1590] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1591] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1592] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1593] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1594] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1595] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1596] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1597] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1598] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1599] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1600] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1601] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1602] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1603] The following is further disclosed regarding the above embodiment.

[1604] (Claim 1)

[1605] means for storing input data received from a user terminal;

[1606] means for transmitting input data received from a user terminal to a server;

[1607] A means for analyzing input data on the server and automatically generating a fault report using AI;

[1608] means for transmitting the generated failure report to a user terminal;

[1609] A system that includes a means for sharing user-confirmed problem reports with the technical department.

[1610] (Claim 2)

[1611] 2. The system according to claim 1, further comprising means for checking past failure data and a case database and adding necessary supplementary information to the generated report.

[1612] (Claim 3)

[1613] 2. The system according to claim 1, further comprising means for receiving input data in the form of check boxes at the user terminal.

[1614] "Example 1"

[1615] (Claim 1)

[1616] means for storing input data received from a user terminal;

[1617] means for transmitting input data received from a user terminal to a server;

[1618] A means for analyzing input data at a server and automatically generating a fault report based on a prompt sentence using a generative AI model;

[1619] means for transmitting the generated failure report to a user terminal;

[1620] A system that includes a means for sharing user-confirmed problem reports with the technical department.

[1621] (Claim 2)

[1622] 2. The system according to claim 1, further comprising means for checking past failure data and a case database and adding necessary supplementary information to the generated report.

[1623] (Claim 3)

[1624] 2. The system according to claim 1, further comprising means for receiving input data in the form of check boxes at the user terminal.

[1625] "Application Example 1"

[1626] (Claim 1)

[1627] means for storing input data received from a user terminal;

[1628] means for transmitting input data received from a user terminal to a server;

[1629] A method for analyzing input data on the server and automatically generating reports using AI;

[1630] means for transmitting the generated report to a user terminal;

[1631] A means for sharing the user-confirmed report with the corresponding department;

[1632] A means to enable it to be used by guards and security personnel as an emergency reporting system;

[1633] means for providing a means for inputting emergency information in the form of check boxes at the terminal;

[1634] A means to add supplementary information by referencing a database of past cases when generating reports using AI, and

[1635] A system including:

[1636] (Claim 2)

[1637] 2. The system according to claim 1, further comprising means for transmitting information in the form of selected checkboxes to a server in real time in the user terminal.

[1638] (Claim 3)

[1639] 10. The system of claim 1, further comprising means for the AI ​​model to utilize prompt sentences to generate detailed natural language text in generating the report.

[1640] "Example 2: Combining Emotion Engines"

[1641] (Claim 1)

[1642] means for storing input data received from a user terminal;

[1643] means for transmitting input data received from a user terminal to a server;

[1644] a means for parsing the input data at the server and converting the data into an appropriate format;

[1645] A means for analyzing user emotion data using an emotion analysis engine in a server;

[1646] A method for AI to automatically generate a crash report based on analytical data and emotional data, and

[1647] means for transmitting the generated failure report to a user terminal;

[1648] A system that includes a means for sharing user-confirmed problem reports with the technical department.

[1649] (Claim 2)

[1650] 2. The system according to claim 1, further comprising means for checking past failure data and a case database and adding necessary supplementary information to the generated report.

[1651] (Claim 3)

[1652] 2. The system according to claim 1, further comprising means for receiving input data in the form of check boxes at the user terminal.

[1653] "Application example 2 when combining emotion engines"

[1654] (Claim 1)

[1655] means for storing input data received from a user terminal;

[1656] means for transmitting input data received from a user terminal to a server;

[1657] A means for analyzing input data on a server and automatically generating a problem report using a generative AI model;

[1658] a sentiment analysis engine for analyzing the emotional state of the user to adjust the tone and content of the generated report;

[1659] means for transmitting the generated problem report to a user terminal;

[1660] A system that includes a means for sharing user-confirmed problem reports with related departments.

[1661] (Claim 2)

[1662] 2. The system according to claim 1, further comprising means for checking past problem data and case databases and adding necessary supplementary information to the generated report.

[1663] (Claim 3)

[1664] 2. The system according to claim 1, further comprising means for receiving input data in the form of check boxes at the user terminal.

[1665] (Claim 4)

[1666] 10. The system of claim 1, further comprising means for generating a report including the emotion data analyzed by the emotion analysis engine. [Explanation of symbols]

[1667] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for storing input data received from a user terminal; means for transmitting input data received from a user terminal to a server; A means for analyzing input data on the server and automatically generating a fault report using AI; means for transmitting the generated failure report to a user terminal; A system that includes a means for sharing user-confirmed problem reports with the technical department.

2. 2. The system according to claim 1, further comprising means for checking past failure data and a case database and adding necessary supplementary information to the generated report.

3. 2. The system according to claim 1, further comprising means for receiving input data in the form of check boxes at the user terminal.

Images