system

Abstract

We provide the system. [Solution] Means for retrieving and maintaining relevant rules for internal audits from a data store, A means of analyzing risk factors from meeting transcripts or documents using natural language processing, A means of automatically generating reports based on past audit information, The means of evaluating the contents of the report and providing it to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional internal audit operations require a lot of time and labor, are inefficient due to a large amount of manual processing, and furthermore, the variation in output has been a problem. Therefore, there is a demand for a system that can obtain efficient and uniform audit results.

Means for Solving the Problems

[0005] The present invention provides a system that efficiently acquires and maintains relevant rules in internal audits, analyzes risk factors by natural language processing, and automatically generates a report based on past audit information. This enables rapid evaluation of the analysis results and provides a uniform output to the user.

[0006] "Internal audit" is an evaluation activity that assesses the business processes and financial status within an organization in order to improve risk management and operational efficiency.

[0007] "Relevant regulations" refers to laws, standards, and internal regulations related to audits.

[0008] A "data store" is a digital database that systematically stores information and makes it accessible as needed.

[0009] "Natural language processing" refers to the techniques and methods by which computers understand, analyze, and generate human natural language.

[0010] "Meeting minutes" are documented records of meetings, interviews, etc., and include the content of conversations and decisions made.

[0011] A "risk factor" refers to an event, condition, or situation that could potentially hinder an organization from achieving its goals.

[0012] "Analysis" is the process of interpreting information and data and extracting hidden meanings.

[0013] "Past audit information" refers to information including data, results, and lessons learned related to audits conducted in the past.

[0014] A "report" is an official document that summarizes the results and analysis of an audit, and is used to explain the findings to the subject.

[0015] A "user" is a designated individual or group that receives the system's output and provides feedback or modifications as needed. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, let's explain the terminology used in the following explanation.

[0019] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0020] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system for automating the internal audit process, and its main functions include retrieving relevant rules, analyzing information using natural language processing, automatically generating reports, and incorporating feedback to users.

[0038] This system is implemented through collaboration between servers, terminals, and users. Specifically, the server first uses a data store to retrieve and manage the latest rules related to the target business. This allows for real-time reflection of the audit impact of rule changes.

[0039] Next, interview audio and supporting documents provided by the user are uploaded from the terminal. The server uses natural language processing technology to analyze these materials. For example, interview audio is converted to text, and keyword detection and risk assessment are performed.

[0040] Furthermore, the server references past audit cases and automatically generates reports based on the data obtained. In this process, it uses similar past cases as models and constructs the report according to evaluation criteria. For example, in audits conducted in accordance with financial transaction regulations, the new report is constructed by referencing the format of past financial transaction audits.

[0041] Finally, the generated report is provided to the user. The user can review the report and provide feedback as needed. The feedback is collected by the server, reflected in the report, and updated as necessary. This cycle enables a dynamic audit process that continuously adapts to user audit requests and feedback.

[0042] The implementation of this system will significantly streamline audit operations and enable the delivery of consistent, high-quality audit reports.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The server identifies the tasks covered by the audit project. The server retrieves relevant rules from the data store via queries and updates the information to the latest state.

[0046] Step 2:

[0047] The terminal is operated by the user, and meeting minutes audio files or submitted evidence documents are uploaded to the system. The terminal then sends this data to the server.

[0048] Step 3:

[0049] The server converts uploaded audio files into text using speech recognition technology. Subsequently, natural language processing is used to analyze the text data and identify risk factors.

[0050] Step 4:

[0051] The server retrieves past audit cases from the data store and compares the extracted risk elements against existing standards. It then generates a report template and writes the report content based on the evaluation results.

[0052] Step 5:

[0053] The server generates an audit report and notifies the user. The user can then review the report details via their terminal and make any necessary corrections or provide feedback.

[0054] Step 6:

[0055] Based on user feedback, the server updates the report and provides it to the user again as a final version. Corrections are made based on the feedback, and the audit process is completed.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] In modern business operations, internal audits need to be conducted quickly and accurately, but traditional methods face challenges such as the time and effort required for updating and analyzing information. Furthermore, it is difficult to dynamically update reports by incorporating user feedback.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for retrieving and managing business-related rules from a database, means for analyzing audio or document data using information processing technology, and means for automatically assembling results based on past cases. This automates the internal audit process, enabling efficient and accurate audits and dynamic report updates.

[0061] A "database" is a digital storage system that provides a mechanism for systematically collecting, storing, and efficiently accessing, managing, and updating information.

[0062] "Rules" are guidelines or rules that direct processes and standards related to business operations or procedures.

[0063] "Information processing technology" refers to the technology used to collect, analyze, store, and transmit data, and to utilize it for specific purposes.

[0064] "Audio or document data" refers to digital information, including audio recordings and text, that is subject to analysis.

[0065] "Past examples" refer to specific records and data from related projects and tasks that have been carried out to date.

[0066] "Automatically assembling" means that a system generates certain results or outputs based on pre-set criteria without requiring human intervention.

[0067] A "user" refers to an individual or organization that uses the system's functions to support their work.

[0068] "Feedback" refers to opinions and information provided by users regarding improvements and corrections.

[0069] A "dynamic report" is a report that is continuously modified and updated after it is generated, allowing it to reflect the latest information and feedback.

[0070] This system is designed to streamline internal audit operations and support the creation of accurate reports. The system consists of servers, terminals, and users working together, each playing a different role.

[0071] The server plays a central role in this system, managing business-related rules retrieved from the database and providing the functionality to update them with the latest information. Advanced information processing technologies are used on the server, utilizing natural language processing (NLP) to analyze speech and text data. Specific software examples include open-source natural language processing libraries and cloud-based data management systems.

[0072] The terminal is used when users upload audio interviews and evidence to the server. Before sending audio files to the server, the terminal converts them to the appropriate format. It also performs processes to optimize data size, such as converting audio files to MP3 format. This allows for efficient data transfer to the server.

[0073] Users are responsible for viewing the generated reports, reviewing their contents, and providing feedback. This feedback is aggregated on the server and used to improve reports and processes. This enables a dynamic and flexible audit process.

[0074] As a concrete example, in audits of financial transactions, if new rules are applied, the server can acquire that information in real time and incorporate it into the analysis process. For instance, by inputting a prompt such as "Prepare an audit report based on the new financial rules" into a generating AI model, it becomes possible to automatically create an advanced report that takes the latest rules into account.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The server retrieves the latest business-related rules from the database. The input is the rule information in the database, and the output is a list of retrieved rules. The server periodically queries the database as a scheduled task to update the necessary rules. This process involves retrieving data from the cloud database via a specific API.

[0078] Step 2:

[0079] Users use a terminal to upload audio interviews and evidence to the server. Input consists of audio and document files provided by the user, and output is a formatted file sent to the server. The terminal converts the audio to a standard format (e.g., MP3) and sends the document file to the server. The HTTP protocol is used for file transfer.

[0080] Step 3:

[0081] The server analyzes uploaded audio and document data. Input is a converted audio or document file, and output is the analyzed text data and risk assessment results. The server utilizes natural language processing (NLP) techniques to convert audio data into text, extract keywords, and evaluate risk factors. NLP libraries are used for the analysis.

[0082] Step 4:

[0083] The server automatically generates a report based on the analysis results, referencing past cases. The input consists of analyzed text data and risk assessment results, while the output is the constructed report. The server selects a template based on similar past cases and generates a customized report to suit the current data. The generated report is constructed using a document generation algorithm.

[0084] Step 5:

[0085] Users review the generated reports and provide feedback. The input is the generated report, and the output is the user's feedback information. Users review the report content and fill in their comments and suggestions for revisions using a stored digital feedback form. This feedback information is then sent to the server.

[0086] Step 6:

[0087] The server incorporates the collected feedback into the report and updates it as needed. The input is user feedback, and the output is the revised and updated final version of the report. The server analyzes the feedback, applies improvements to the report, and generates an updated version. The final version of the report is then saved again to cloud storage and shared with the user.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] Traditional internal audit processes faced challenges in quickly responding to rule changes and collecting and evaluating real-time information from the field, which compromised the accuracy and efficiency of audits. Furthermore, delays in incorporating feedback posed a risk of reduced audit quality.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for retrieving and maintaining rules from a data storage device, means for analyzing risk factors from speech or information using natural language processing, means for automatically generating report texts based on past information, and means for transmitting data directly from the field and evaluating risks. This enables rule changes and rapid information gathering and evaluation from the field, and allows for immediate reflection of feedback to improve the quality of audit results.

[0093] "Internal auditing" is an activity that evaluates the efficiency and adherence to regulations of an organization and identifies risks through the evaluation and auditing of operations and processes conducted within the organization.

[0094] "Relevant regulations" refer to the laws, policies, and standards that must be followed in specific business operations or activities, and are necessary to ensure the legality and appropriateness of the audited entity.

[0095] A "data storage device" is a hardware or software system that stores information long-term or short-term and allows it to be retrieved as needed.

[0096] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, and it involves methods for analyzing speech and text to extract meaning.

[0097] A "risk factor" is a risk element that could potentially cause unexpected problems or damages in the course of performing business operations.

[0098] "Past information" refers to data and events that have been previously recorded or reported, and is information that can be used as a reference for future evaluations and decision-making.

[0099] A "report" is a document created based on an audit or investigation, which details the evaluation results and recommendations.

[0100] A "remote communication terminal" is an electronic device that enables communication from geographically distant locations and is a means of sending and receiving data.

[0101] "Means of risk assessment" are methods or tools for quantitatively or qualitatively analyzing potential risk factors and determining their impact and probability.

[0102] To implement this invention, a server, terminals (such as smartphones or other remote communication devices), and users must cooperate to operate the system. The following details how the system functions.

[0103] The server first uses data storage to retrieve rules related to internal audits and keeps them constantly up-to-date. During this process, the server searches a database of rules to extract necessary information. It also utilizes natural language processing (NLP) techniques to analyze interview audio and document data to identify risk factors. Software libraries such as TENSORFLOW® and NLTK are used for NLP techniques.

[0104] The terminal collects audio and documents from the field via smart devices and uploads them to a server. The terminal's software is typically developed using Swift or Kotlin. This allows users to input data instantly in the field.

[0105] The server automatically generates reports based on data obtained by referencing past information. This process utilizes an AI model that has learned from past cases. As a result, evaluation criteria based on similar cases are provided.

[0106] The generated report is quickly provided to the user. The user can review the report and provide feedback as needed. This feedback is collected by the server and reflected in the report.

[0107] A concrete example is a security audit where an auditor uses their smartphone to record suspicious activity via audio and sends it to a server. The server then analyzes the audio, immediately performs a risk assessment, generates a report, and sends it to the administrator.

[0108] An example of a prompt used for a generative AI model is: "Automate the company's security audit process. Analyze interview audio and incident reports in real time, perform risk assessments, and generate reports quickly."

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The terminal uses a smartphone to input voice and document data from the field and send it to the server. During this process, the application on the terminal converts the user's voice input into text and sends image data and other data as needed. The input data is uploaded to the server via the internet.

[0112] Step 2:

[0113] The server retrieves the latest relevant rules from the data storage device and begins analyzing the input data based on this information. The server uses natural language processing (NLP) to extract keywords from the transcribed speech data and identify risk factors. This is done using NLP libraries such as TensorFlow or NLTK. The output of this process becomes information for risk assessment.

[0114] Step 3:

[0115] The server automatically generates reports based on analyzed data, referencing historical information. Analyzed risk information is used as input. An AI model generates documents using appropriate report templates, referencing past audit data. This output is a document formatted as the final report.

[0116] Step 4:

[0117] The server provides the user with the generated report. This report is displayed to the user via a user interface. The user can review the report and enter their evaluation and feedback. This feedback is also sent to the server and incorporated into the report as needed.

[0118] Step 5:

[0119] Users provide feedback, which is then processed by the server. The server analyzes the feedback and makes necessary corrections to the report. Once this cycle is complete, a final report is created, which is used as the basis for improvement measures and feedback for the next audit.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention provides an automated evaluation system for internal audit processes that takes into account the emotional state of users. In addition to acquiring relevant rules, analyzing documents, automatically generating reports, and incorporating user feedback, this system utilizes an emotion engine to enable more precise risk assessment and process optimization.

[0122] This system is implemented through the cooperation of a server, terminals, and an emotion engine. First, the server retrieves relevant rules from the data store and updates them in real time, thereby responding immediately to changes in the audit environment.

[0123] Next, the process begins when the user uploads interview audio and submitted evidence documents to the system via their device. The server analyzes the provided materials using natural language processing technology and extracts risk factors. As part of this analysis, an emotion engine evaluates the user's statements and determines their emotional state. For example, if the interviewee is experiencing high levels of stress or anxiety, the system can identify the contributing factors.

[0124] The server integrates emotional data analyzed by the emotion engine into risk assessments and automatically generates reports while comparing them with past audit information. Because the assessment includes emotional state-based evaluations, it enables risk assessments from a more multifaceted perspective than before. For example, it may recommend additional audit actions for departments where unstable emotional states are frequently observed.

[0125] Finally, the generated report is provided to the user, and feedback is requested. User feedback is analyzed by an emotion engine and incorporated into the report to further improve its accuracy. This two-way feedback mechanism enables audit work to be conducted in a more efficient and user-friendly manner.

[0126] The following describes the processing flow.

[0127] Step 1:

[0128] The server identifies the business processes subject to internal audit and retrieves relevant rules from the data store. The server maintains up-to-date rules and updates them as needed.

[0129] Step 2:

[0130] The user uploads interview audio files and submitted evidence documents to the system using a terminal. The terminal then sends these files to the server.

[0131] Step 3:

[0132] The server converts uploaded audio files into text using speech recognition technology. Natural language processing is used to analyze the text data, and an emotion engine identifies the user's emotional state from their statements, along with risk factors.

[0133] Step 4:

[0134] The server analyzes the emotional state identified by the emotion engine and integrates it as part of the risk assessment. It combines historical audit information with emotion data to generate a report template.

[0135] Step 5:

[0136] The server automatically generates an audit report and notifies the user, providing the report's contents to their terminal. The user reviews the report and provides feedback to the server.

[0137] Step 6:

[0138] The server analyzes user feedback using an emotion engine and updates the report as needed. Changes in emotions based on the feedback are also evaluated, and the final version is provided to the user again.

[0139] (Example 2)

[0140] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0141] Traditional internal audit systems had a problem with incomplete risk assessment and identification of risk factors because risk assessments were conducted without considering the emotional state of users. Furthermore, user feedback was not adequately reflected in reports, resulting in a failure to optimize processes.

[0142] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0143] In this invention, the server includes means for acquiring and maintaining relevant rules from an information storage device, means for analyzing risk factors from voice data or documents using natural language processing, and means for identifying emotional states using an emotion evaluation function in the analysis. This enables multifaceted risk assessment that takes user emotions into account and improves the accuracy of reports based on feedback.

[0144] "Relevant regulations" refer to standards and guidelines used as reference when conducting internal audits, and include information on audit requirements and procedures.

[0145] An "information storage device" is an electronic device used to temporarily or long-term store data, and includes databases and storage servers.

[0146] "Natural language processing" is a technology that enables computers to understand and analyze human language, and it has the ability to analyze the meaning of audio and text data.

[0147] "Risk factors" refer to problems and challenges that a company or organization may face, and are important points to check during an audit.

[0148] "Emotional assessment functionality" refers to technologies used to automatically analyze and identify a user's emotional state, including technologies that infer emotions from voice tone and text.

[0149] A "report" refers to a document that summarizes the audit results and includes risk assessments and recommendations.

[0150] "Feedback" refers to the opinions and comments that users provide to reports and systems, and the information used to improve the system and enhance the accuracy of evaluations.

[0151] This invention describes embodiments for carrying it out. This system is an automated evaluation system that takes into account the emotional state of users in internal audits. The following are specific embodiments thereof.

[0152] The server first retrieves relevant rules from the information storage device and updates them in real time. This process uses a hosted database system to ensure that the rule data is accurate and up-to-date.

[0153] Next, the user uploads interview audio and submitted documents to the system via a terminal. The terminal can be a network-connected computer or tablet device. Users can easily upload data through a dedicated interface.

[0154] The server receives the uploaded data and analyzes it using natural language processing techniques. This process utilizes the Python programming language and libraries (e.g., NLTK and spaCy). During the analysis, the server uses sentiment assessment capabilities to identify emotional states from the user's utterances and voice data. This allows for the identification of emotions such as stress and anxiety.

[0155] Next, the server integrates sentiment data into the risk assessment and automatically generates a report using a generative AI model, comparing it with past audit information. This report provides a multifaceted evaluation of risk factors that are important to the user.

[0156] Finally, the generated report is provided to the user, and feedback information is received from the user. This feedback is then analyzed again using the sentiment rating function to help improve the accuracy of the report.

[0157] An example of a prompt message might be, "Use the AI ​​model to conduct a risk assessment based on the emotional state of a specific department. Propose specific emotional indicators and the audit actions based on them." This system makes the internal audit process more intuitive and effective for users.

[0158] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0159] Step 1:

[0160] The server retrieves relevant rules from the information storage device. It receives the latest rule data from the database as input and stores it in internal memory. The output is the latest rule dataset. This dataset is used as the basis for the audit process.

[0161] Step 2:

[0162] Users upload interview audio and supporting documents via their terminal. The input here is the file data selected by the user, and the output is the audio and document data stored in the server's storage area. Specifically, the user operates the terminal's interface, selects the file, and presses the upload button.

[0163] Step 3:

[0164] The server receives the uploaded data and performs analysis using natural language processing (NLP) techniques. The input consists of audio and text data uploaded by the user. The output is a list of identified risk factors. Specifically, it utilizes an NLP library to extract important keywords and risk topics from the text.

[0165] Step 4:

[0166] The server uses sentiment assessment functionality to identify the user's emotional state from their utterances. Input is information contained in audio and document data, and output is an indicator of emotional state (e.g., stress level, anxiety level). This process utilizes speech analysis equipment to perform specific actions such as determining the user's voice tone and context.

[0167] Step 5:

[0168] The server integrates sentiment data into risk assessment and automatically generates reports. As input, risk factors and sentiment indicators are integrated, and this data is analyzed by a generating AI model. The output is a detailed assessment report. Specifically, it executes an algorithm that evaluates risk from multiple perspectives, comparing it with past audit reports.

[0169] Step 6:

[0170] The user receives the generated report and provides feedback to the system. The input here consists of the user's opinions and comments based on the report. The system then analyzes this again, and an improved report template is created as output. This includes the specific actions of the user entering feedback and pressing the submit button.

[0171] (Application Example 2)

[0172] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0173] In the internal audit process, risk assessment using traditional methods was often not sufficiently multifaceted, and the emotional state of users was frequently not reflected in the risk assessment. As a result, traditional systems suffered from insufficient audit accuracy and efficiency, and furthermore, there were challenges in real-time situational awareness and appropriate audit actions.

[0174] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0175] In this invention, the server includes means for retrieving and maintaining relevant rules in internal audits from an information database, means for analyzing risk factors from recorded audio or informational materials using natural language processing, and means for performing sentiment analysis and integrating the user's emotional state into the risk assessment. This enables a multifaceted and real-time risk assessment that takes into account the user's emotional state during the audit process.

[0176] An "information database" is a collection of data that stores rules and information related to internal audits, and that can be retrieved and updated as needed.

[0177] "Natural language processing" is a technology that analyzes the language that humans use on a daily basis as data and extracts information from it.

[0178] "Recorded audio or informational materials" refers to audio recordings used in the audit process, as well as documents and digital data that serve as various sources of information.

[0179] "Risk factors" are elements that may affect operations or processes, and they indicate potential dangers or problems for the subject of the audit.

[0180] "Emotional analysis" is the process of identifying a user's emotional state and extracting that information by analyzing their voice or text.

[0181] "Real-time" refers to the immediacy of obtaining output or results the moment information processing or analysis is performed.

[0182] "Means of recommending audit actions" refers to the system's function of suggesting appropriate next steps or actions based on the results of sentiment analysis and risk assessment.

[0183] The system based on this invention automates the internal audit process and integrates sentiment analysis to achieve more sophisticated risk assessment and process optimization. This system is built primarily by leveraging feedback from servers, terminals, and users.

[0184] The server's primary role is to retrieve relevant rules from the information database and maintain them. The information database stores all the rules and related information necessary for internal audits and is used in real time during the daily audit process.

[0185] The device provides an interface for users to input voice data and upload documents. The uploaded data is analyzed using natural language processing technology to extract risk factors. Specifically, natural language processing engines such as Google® Cloud Natural Language API are used to convert voice data into text, and potential risks are detected from that text.

[0186] The server's sentiment analysis engine analyzes the user's emotional state and integrates this into the risk assessment. Sentiment analysis libraries such as IBM Watson® Tone Analyzer are used for this analysis. Based on these results, the system proposes real-time audit actions and provides situation-appropriate recommendations to each stakeholder.

[0187] As a concrete example, imagine an auditor using smart glasses during an audit to capture audio data in real time, and sentiment analysis is performed on the spot. If this analysis detects that the stress level during the conversation is higher than normal, further investigation is recommended.

[0188] An example of a prompt to a generative AI model is, "How can I analyze the emotional state of participants during an audit interview and perform a real-time risk assessment?" Using this prompt, the generative AI can provide guidance on risk assessment tailored to specific situations.

[0189] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0190] Step 1:

[0191] The server retrieves relevant rules from the information database and converts them into a format that users can upload. The input is the information database, and the output is the retrieved relevant rules. The retrieved rules are then formatted as data for use in subsequent analysis processes.

[0192] Step 2:

[0193] The terminal receives audio or document data from the user and uploads it to the system. The input is the audio or document provided by the user, and the output is the uploaded data. The terminal receives the data via the user interface and transfers it to the server.

[0194] Step 3:

[0195] The server converts uploaded audio data into text using a natural language processing engine (Google Cloud Natural Language API). The input is audio data, and the output is converted text data. This automated process ensures high accuracy in transcribing audio content into text.

[0196] Step 4:

[0197] The server performs a process of analyzing text data and extracting risk factors. The input is the transformed text data, and the output is the extracted risk factors. The analysis is performed automatically based on previously acquired rules.

[0198] Step 5:

[0199] The server uses an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state and integrates this into the risk assessment. The input is text data, and the output is the result of the emotional state assessment. The emotional state is used as data to perform adjustments based on the assessment of risk factors.

[0200] Step 6:

[0201] The server proposes audit actions in real time based on emotional states and risk factors. The inputs are emotional states and risk factors, and the output is the proposed audit action. The proposed action is provided to the user immediately.

[0202] Step 7:

[0203] Users review the audit report and proposed actions provided by the server and provide feedback. The input is the proposed audit actions and report, and the output is the user's feedback. This feedback is returned to the system and used for further improvement.

[0204] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0205] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0206] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0207] [Second Embodiment]

[0208] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0209] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0210] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0211] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0212] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0213] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0214] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0215] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0216] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0217] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0218] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0219] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0220] This invention is a system for automating the internal audit process, and its main functions include retrieving relevant rules, analyzing information using natural language processing, automatically generating reports, and incorporating feedback to users.

[0221] This system is implemented through collaboration between servers, terminals, and users. Specifically, the server first uses a data store to retrieve and manage the latest rules related to the target business. This allows for real-time reflection of the audit impact of rule changes.

[0222] Next, interview audio and supporting documents provided by the user are uploaded from the terminal. The server uses natural language processing technology to analyze these materials. For example, interview audio is converted to text, and keyword detection and risk assessment are performed.

[0223] Furthermore, the server references past audit cases and automatically generates reports based on the data obtained. In this process, it uses similar past cases as models and constructs the report according to evaluation criteria. For example, in audits conducted in accordance with financial transaction regulations, the new report is constructed by referencing the format of past financial transaction audits.

[0224] Finally, the generated report is provided to the user. The user can review the report and provide feedback as needed. The feedback is collected by the server, reflected in the report, and updated as necessary. This cycle enables a dynamic audit process that continuously adapts to user audit requests and feedback.

[0225] The implementation of this system will significantly streamline audit operations and enable the delivery of consistent, high-quality audit reports.

[0226] The following describes the processing flow.

[0227] Step 1:

[0228] The server identifies the tasks covered by the audit project. The server retrieves relevant rules from the data store via queries and updates the information to the latest state.

[0229] Step 2:

[0230] The terminal is operated by the user, and meeting minutes audio files or submitted evidence documents are uploaded to the system. The terminal then sends this data to the server.

[0231] Step 3:

[0232] The server converts uploaded audio files into text using speech recognition technology. Subsequently, natural language processing is used to analyze the text data and identify risk factors.

[0233] Step 4:

[0234] The server retrieves past audit cases from the data store and compares the extracted risk elements against existing standards. It then generates a report template and writes the report content based on the evaluation results.

[0235] Step 5:

[0236] The server generates an audit report and notifies the user. The user can then review the report details via their terminal and make any necessary corrections or provide feedback.

[0237] Step 6:

[0238] Based on user feedback, the server updates the report and provides it to the user again as a final version. Corrections are made based on the feedback, and the audit process is completed.

[0239] (Example 1)

[0240] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0241] In modern business operations, internal audits need to be conducted quickly and accurately, but traditional methods face challenges such as the time and effort required for updating and analyzing information. Furthermore, it is difficult to dynamically update reports by incorporating user feedback.

[0242] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0243] In this invention, the server includes means for retrieving and managing business-related rules from a database, means for analyzing audio or document data using information processing technology, and means for automatically assembling results based on past cases. This automates the internal audit process, enabling efficient and accurate audits and dynamic report updates.

[0244] A "database" is a digital storage system that provides a mechanism for systematically collecting, storing, and efficiently accessing, managing, and updating information.

[0245] "Rules" are guidelines or rules that direct processes and standards related to business operations or procedures.

[0246] "Information processing technology" refers to the technology used to collect, analyze, store, and transmit data, and to utilize it for specific purposes.

[0247] "Audio or document data" refers to digital information, including audio recordings and text, that is subject to analysis.

[0248] "Past examples" refer to specific records and data from related projects and tasks that have been carried out to date.

[0249] "Automatically assembling" means that a system generates certain results or outputs based on pre-set criteria without requiring human intervention.

[0250] A "user" refers to an individual or organization that uses the system's functions to support their work.

[0251] "Feedback" refers to opinions and information provided by users regarding improvements and corrections.

[0252] A "dynamic report" is a report that is continuously modified and updated after it is generated, allowing it to reflect the latest information and feedback.

[0253] This system is designed to streamline internal audit operations and support the creation of accurate reports. The system consists of servers, terminals, and users working together, each playing a different role.

[0254] The server plays a central role in this system, managing business-related rules retrieved from the database and providing the functionality to update them with the latest information. Advanced information processing technologies are used on the server, utilizing natural language processing (NLP) to analyze speech and text data. Specific software examples include open-source natural language processing libraries and cloud-based data management systems.

[0255] The terminal is used when users upload audio interviews and evidence to the server. Before sending audio files to the server, the terminal converts them to the appropriate format. It also performs processes to optimize data size, such as converting audio files to MP3 format. This allows for efficient data transfer to the server.

[0256] Users are responsible for viewing the generated reports, reviewing their contents, and providing feedback. This feedback is aggregated on the server and used to improve reports and processes. This enables a dynamic and flexible audit process.

[0257] As a concrete example, in audits of financial transactions, if new rules are applied, the server can acquire that information in real time and incorporate it into the analysis process. For instance, by inputting a prompt such as "Prepare an audit report based on the new financial rules" into a generating AI model, it becomes possible to automatically create an advanced report that takes the latest rules into account.

[0258] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0259] Step 1:

[0260] The server retrieves the latest business-related rules from the database. The input is the rule information in the database, and the output is a list of retrieved rules. The server periodically queries the database as a scheduled task to update the necessary rules. This process involves retrieving data from the cloud database via a specific API.

[0261] Step 2:

[0262] Users use a terminal to upload audio interviews and evidence to the server. Input consists of audio and document files provided by the user, and output is a formatted file sent to the server. The terminal converts the audio to a standard format (e.g., MP3) and sends the document file to the server. The HTTP protocol is used for file transfer.

[0263] Step 3:

[0264] The server analyzes uploaded audio and document data. Input is a converted audio or document file, and output is the analyzed text data and risk assessment results. The server utilizes natural language processing (NLP) techniques to convert audio data into text, extract keywords, and evaluate risk factors. NLP libraries are used for the analysis.

[0265] Step 4:

[0266] The server automatically generates a report based on the analysis results, referencing past cases. The input consists of analyzed text data and risk assessment results, while the output is the constructed report. The server selects a template based on similar past cases and generates a customized report to suit the current data. The generated report is constructed using a document generation algorithm.

[0267] Step 5:

[0268] Users review the generated reports and provide feedback. The input is the generated report, and the output is the user's feedback information. Users review the report content and fill in their comments and suggestions for revisions using a stored digital feedback form. This feedback information is then sent to the server.

[0269] Step 6:

[0270] The server incorporates the collected feedback into the report and updates it as needed. The input is user feedback, and the output is the revised and updated final version of the report. The server analyzes the feedback, applies improvements to the report, and generates an updated version. The final version of the report is then saved again to cloud storage and shared with the user.

[0271] (Application Example 1)

[0272] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0273] Traditional internal audit processes faced challenges in quickly responding to rule changes and collecting and evaluating real-time information from the field, which compromised the accuracy and efficiency of audits. Furthermore, delays in incorporating feedback posed a risk of reduced audit quality.

[0274] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0275] In this invention, the server includes means for retrieving and maintaining rules from a data storage device, means for analyzing risk factors from speech or information using natural language processing, means for automatically generating report texts based on past information, and means for transmitting data directly from the field and evaluating risks. This enables rule changes and rapid information gathering and evaluation from the field, and allows for immediate reflection of feedback to improve the quality of audit results.

[0276] "Internal auditing" is an activity that evaluates the efficiency and adherence to regulations of an organization and identifies risks through the evaluation and auditing of operations and processes conducted within the organization.

[0277] "Relevant regulations" refer to the laws, policies, and standards that must be followed in specific business operations or activities, and are necessary to ensure the legality and appropriateness of the audited entity.

[0278] A "data storage device" is a hardware or software system that stores information long-term or short-term and allows it to be retrieved as needed.

[0279] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, and it involves methods for analyzing speech and text to extract meaning.

[0280] A "risk factor" is a risk element that could potentially cause unexpected problems or damages in the course of performing business operations.

[0281] "Past information" refers to data and events that have been previously recorded or reported, and is information that can be used as a reference for future evaluations and decision-making.

[0282] A "report" is a document created based on an audit or investigation, which details the evaluation results and recommendations.

[0283] A "remote communication terminal" is an electronic device that enables communication from geographically distant locations and is a means of sending and receiving data.

[0284] The "means for evaluating risks" is a method or tool for quantitatively or qualitatively analyzing potential risk factors and judging their impacts and probabilities.

[0285] To implement this invention, a server, a terminal (a remote communication terminal such as a smartphone), and a user cooperate to operate the system. The following details how the system functions.

[0286] First, the server uses a data storage device to obtain rules related to internal audits and always maintains them in the latest state. At this time, the server searches a database related to the rules and extracts the necessary information. In addition, it utilizes natural language processing technology to analyze interview voices and document data to identify risk factors. Software libraries such as TensorFlow and NLTK are used for the NLP technology employed here.

[0287] The terminal collects voices and documents from the site through a smart device and uploads them to the server. The software of the terminal is generally developed in Swift or Kotlin. This enables the user to input data immediately at the site.

[0288] The server automatically generates a report based on the data obtained by referring to past information. An AI model that has learned past cases is utilized in this process. This provides evaluation criteria based on similar cases.

[0289] The generated report is provided to the user side promptly. The user can check the report and provide feedback if necessary. This feedback is collected by the server and reflected in the report.

[0290] As a specific example, in a security audit, an auditor records suspicious activities with a smartphone and transmits them to the server. The server analyzes the voice, immediately conducts a risk assessment, generates a report, and transmits it to the administrator.

[0291] An example of a prompt used for a generative AI model is: "Automate the company's security audit process. Analyze interview audio and incident reports in real time, perform risk assessments, and generate reports quickly."

[0292] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0293] Step 1:

[0294] The terminal uses a smartphone to input voice and document data from the field and send it to the server. During this process, the application on the terminal converts the user's voice input into text and sends image data and other data as needed. The input data is uploaded to the server via the internet.

[0295] Step 2:

[0296] The server retrieves the latest relevant rules from the data storage device and begins analyzing the input data based on this information. The server uses natural language processing (NLP) to extract keywords from the transcribed speech data and identify risk factors. This is done using NLP libraries such as TensorFlow or NLTK. The output of this process becomes information for risk assessment.

[0297] Step 3:

[0298] The server automatically generates reports based on analyzed data, referencing historical information. Analyzed risk information is used as input. An AI model generates documents using appropriate report templates, referencing past audit data. This output is a document formatted as the final report.

[0299] Step 4:

[0300] The server provides the user with the generated report. This report is displayed to the user via a user interface. The user can review the report and enter their evaluation and feedback. This feedback is also sent to the server and incorporated into the report as needed.

[0301] Step 5:

[0302] Users provide feedback, which is then processed by the server. The server analyzes the feedback and makes necessary corrections to the report. Once this cycle is complete, a final report is created, which is used as the basis for improvement measures and feedback for the next audit.

[0303] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0304] This invention provides an automated evaluation system for internal audit processes that takes into account the emotional state of users. In addition to acquiring relevant rules, analyzing documents, automatically generating reports, and incorporating user feedback, this system utilizes an emotion engine to enable more precise risk assessment and process optimization.

[0305] This system is implemented through the cooperation of a server, terminals, and an emotion engine. First, the server retrieves relevant rules from the data store and updates them in real time, thereby responding immediately to changes in the audit environment.

[0306] Next, the process is initiated when the user uploads interview audio and submitted evidentiary materials to the system via the terminal. The server analyzes the materials provided by natural language processing technology and extracts risk factors. As part of this analysis, the emotion engine evaluates the content of the user's speech and discriminates the emotional state. For example, when the interviewee's stress and anxiety are increasing, the factors can be identified.

[0307] The server integrates the emotion data analyzed by the emotion engine into the risk assessment and automatically generates a report while comparing it with past audit information. Since the evaluation based on the emotional state is included, a more multi-faceted risk assessment from a conventional perspective becomes possible. As a specific example, additional audit actions may be recommended for departments where unstable emotional states are frequently observed.

[0308] Finally, the generated report is provided to the user and feedback is requested. The user's feedback is analyzed by the emotion engine and reflected to further improve the accuracy of the report content. Such a two-way feedback mechanism enables the audit work to be carried out in a more efficient and user-friendly manner.

[0309] The following describes the process flow.

[0310] Step 1:

[0311] The server discriminates the target business of the internal audit and obtains relevant rules from the data store. The server maintains the latest information of the rules and updates them as necessary.

[0312] Step 2:

[0313] The user uploads the interview audio file and the submitted evidentiary materials to the system using the terminal. The terminal sends these files to the server.

[0314] Step 3:

[0315] The server converts uploaded audio files into text using speech recognition technology. Natural language processing is used to analyze the text data, and an emotion engine identifies the user's emotional state from their statements, along with risk factors.

[0316] Step 4:

[0317] The server analyzes the emotional state identified by the emotion engine and integrates it as part of the risk assessment. It combines historical audit information with emotion data to generate a report template.

[0318] Step 5:

[0319] The server automatically generates an audit report and notifies the user, providing the report's contents to their terminal. The user reviews the report and provides feedback to the server.

[0320] Step 6:

[0321] The server analyzes user feedback using an emotion engine and updates the report as needed. Changes in emotions based on the feedback are also evaluated, and the final version is provided to the user again.

[0322] (Example 2)

[0323] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0324] Traditional internal audit systems had a problem with incomplete risk assessment and identification of risk factors because risk assessments were conducted without considering the emotional state of users. Furthermore, user feedback was not adequately reflected in reports, resulting in a failure to optimize processes.

[0325] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0326] In this invention, the server includes means for acquiring and maintaining relevant rules from an information storage device, means for analyzing risk factors from voice data or documents using natural language processing, and means for identifying emotional states using an emotion evaluation function in the analysis. This enables multifaceted risk assessment that takes user emotions into account and improves the accuracy of reports based on feedback.

[0327] "Relevant regulations" refer to standards and guidelines used as reference when conducting internal audits, and include information on audit requirements and procedures.

[0328] An "information storage device" is an electronic device used to temporarily or long-term store data, and includes databases and storage servers.

[0329] "Natural language processing" is a technology that enables computers to understand and analyze human language, and it has the ability to analyze the meaning of audio and text data.

[0330] "Risk factors" refer to problems and challenges that a company or organization may face, and are important points to check during an audit.

[0331] "Emotional assessment functionality" refers to technologies used to automatically analyze and identify a user's emotional state, including technologies that infer emotions from voice tone and text.

[0332] A "report" refers to a document that summarizes the audit results and includes risk assessments and recommendations.

[0333] "Feedback" refers to the opinions and comments that users provide to reports and systems, and the information used to improve the system and enhance the accuracy of evaluations.

[0334] This invention describes embodiments for carrying it out. This system is an automated evaluation system that takes into account the emotional state of users in internal audits. The following are specific embodiments thereof.

[0335] The server first retrieves relevant rules from the information storage device and updates them in real time. This process uses a hosted database system to ensure that the rule data is accurate and up-to-date.

[0336] Next, the user uploads interview audio and submitted documents to the system via a terminal. The terminal can be a network-connected computer or tablet device. Users can easily upload data through a dedicated interface.

[0337] The server receives the uploaded data and analyzes it using natural language processing techniques. This process utilizes the Python programming language and libraries (e.g., NLTK and spaCy). During the analysis, the server uses sentiment assessment capabilities to identify emotional states from the user's utterances and voice data. This allows for the identification of emotions such as stress and anxiety.

[0338] Next, the server integrates sentiment data into the risk assessment and automatically generates a report using a generative AI model, comparing it with past audit information. This report provides a multifaceted evaluation of risk factors that are important to the user.

[0339] Finally, the generated report is provided to the user, and feedback information is received from the user. This feedback is then analyzed again using the sentiment rating function to help improve the accuracy of the report.

[0340] An example of a prompt message might be, "Use the AI ​​model to conduct a risk assessment based on the emotional state of a specific department. Propose specific emotional indicators and the audit actions based on them." This system makes the internal audit process more intuitive and effective for users.

[0341] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0342] Step 1:

[0343] The server retrieves relevant rules from the information storage device. It receives the latest rule data from the database as input and stores it in internal memory. The output is the latest rule dataset. This dataset is used as the basis for the audit process.

[0344] Step 2:

[0345] Users upload interview audio and supporting documents via their terminal. The input here is the file data selected by the user, and the output is the audio and document data stored in the server's storage area. Specifically, the user operates the terminal's interface, selects the file, and presses the upload button.

[0346] Step 3:

[0347] The server receives the uploaded data and performs analysis using natural language processing (NLP) techniques. The input consists of audio and text data uploaded by the user. The output is a list of identified risk factors. Specifically, it utilizes an NLP library to extract important keywords and risk topics from the text.

[0348] Step 4:

[0349] The server uses sentiment assessment functionality to identify the user's emotional state from their utterances. Input is information contained in audio and document data, and output is an indicator of emotional state (e.g., stress level, anxiety level). This process utilizes speech analysis equipment to perform specific actions such as determining the user's voice tone and context.

[0350] Step 5:

[0351] The server integrates sentiment data into risk assessment and automatically generates reports. As input, risk factors and sentiment indicators are integrated, and this data is analyzed by a generating AI model. The output is a detailed assessment report. Specifically, it executes an algorithm that evaluates risk from multiple perspectives, comparing it with past audit reports.

[0352] Step 6:

[0353] The user receives the generated report and provides feedback to the system. The input here consists of the user's opinions and comments based on the report. The system then analyzes this again, and an improved report template is created as output. This includes the specific actions of the user entering feedback and pressing the submit button.

[0354] (Application Example 2)

[0355] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0356] In the internal audit process, risk assessment using traditional methods was often not sufficiently multifaceted, and the emotional state of users was frequently not reflected in the risk assessment. As a result, traditional systems suffered from insufficient audit accuracy and efficiency, and furthermore, there were challenges in real-time situational awareness and appropriate audit actions.

[0357] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0358] In this invention, the server includes means for retrieving and maintaining relevant rules in internal audits from an information database, means for analyzing risk factors from recorded audio or informational materials using natural language processing, and means for performing sentiment analysis and integrating the user's emotional state into the risk assessment. This enables a multifaceted and real-time risk assessment that takes into account the user's emotional state during the audit process.

[0359] An "information database" is a collection of data that stores rules and information related to internal audits, and that can be retrieved and updated as needed.

[0360] "Natural language processing" is a technology that analyzes the language that humans use on a daily basis as data and extracts information from it.

[0361] "Recorded audio or informational materials" refers to audio recordings used in the audit process, as well as documents and digital data that serve as various sources of information.

[0362] "Risk factors" are elements that may affect operations or processes, and they indicate potential dangers or problems for the subject of the audit.

[0363] "Emotional analysis" is the process of identifying a user's emotional state and extracting that information by analyzing their voice or text.

[0364] "Real-time" refers to the immediacy of obtaining output or results the moment information processing or analysis is performed.

[0365] "Means of recommending audit actions" refers to the system's function of suggesting appropriate next steps or actions based on the results of sentiment analysis and risk assessment.

[0366] The system based on this invention automates the internal audit process and integrates sentiment analysis to achieve more sophisticated risk assessment and process optimization. This system is built primarily by leveraging feedback from servers, terminals, and users.

[0367] The server's primary role is to retrieve relevant rules from the information database and maintain them. The information database stores all the rules and related information necessary for internal audits and is used in real time during the daily audit process.

[0368] The device provides an interface for users to input voice data and upload documents. The uploaded data is analyzed using natural language processing technology to extract risk factors. Specifically, natural language processing engines such as the Google Cloud Natural Language API are used to convert voice data into text, and potential risks are detected from that text.

[0369] The server's sentiment analysis engine analyzes the user's emotional state and integrates this into the risk assessment. Sentiment analysis libraries such as IBM Watson Tone Analyzer are used for this analysis. Based on these results, the system proposes real-time audit actions and provides situation-appropriate recommendations to each stakeholder.

[0370] As a concrete example, imagine an auditor using smart glasses during an audit to capture audio data in real time, and sentiment analysis is performed on the spot. If this analysis detects that the stress level during the conversation is higher than normal, further investigation is recommended.

[0371] An example of a prompt to a generative AI model is, "How can I analyze the emotional state of participants during an audit interview and perform a real-time risk assessment?" Using this prompt, the generative AI can provide guidance on risk assessment tailored to specific situations.

[0372] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0373] Step 1:

[0374] The server retrieves relevant rules from the information database and converts them into a format that users can upload. The input is the information database, and the output is the retrieved relevant rules. The retrieved rules are then formatted as data for use in subsequent analysis processes.

[0375] Step 2:

[0376] The terminal receives audio or document data from the user and uploads it to the system. The input is the audio or document provided by the user, and the output is the uploaded data. The terminal receives the data via the user interface and transfers it to the server.

[0377] Step 3:

[0378] The server converts uploaded audio data into text using a natural language processing engine (Google Cloud Natural Language API). The input is audio data, and the output is converted text data. This automated process ensures high accuracy in transcribing audio content into text.

[0379] Step 4:

[0380] The server performs a process of analyzing text data and extracting risk factors. The input is the transformed text data, and the output is the extracted risk factors. The analysis is performed automatically based on previously acquired rules.

[0381] Step 5:

[0382] The server uses an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state and integrates this into the risk assessment. The input is text data, and the output is the result of the emotional state assessment. The emotional state is used as data to perform adjustments based on the assessment of risk factors.

[0383] Step 6:

[0384] The server proposes audit actions in real time based on emotional states and risk factors. The inputs are emotional states and risk factors, and the output is the proposed audit action. The proposed action is provided to the user immediately.

[0385] Step 7:

[0386] Users review the audit report and proposed actions provided by the server and provide feedback. The input is the proposed audit actions and report, and the output is the user's feedback. This feedback is returned to the system and used for further improvement.

[0387] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0388] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0389] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0390] [Third Embodiment]

[0391] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0392] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0393] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0394] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0395] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0396] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0397] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0398] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0399] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0400] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0401] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0402] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0403] This invention is a system for automating the internal audit process, and its main functions include retrieving relevant rules, analyzing information using natural language processing, automatically generating reports, and incorporating feedback to users.

[0404] This system is implemented through collaboration between servers, terminals, and users. Specifically, the server first uses a data store to retrieve and manage the latest rules related to the target business. This allows for real-time reflection of the audit impact of rule changes.

[0405] Next, interview audio and supporting documents provided by the user are uploaded from the terminal. The server uses natural language processing technology to analyze these materials. For example, interview audio is converted to text, and keyword detection and risk assessment are performed.

[0406] Furthermore, the server references past audit cases and automatically generates reports based on the data obtained. In this process, it uses similar past cases as models and constructs the report according to evaluation criteria. For example, in audits conducted in accordance with financial transaction regulations, the new report is constructed by referencing the format of past financial transaction audits.

[0407] Finally, the generated report is provided to the user. The user can review the report and provide feedback as needed. The feedback is collected by the server, reflected in the report, and updated as necessary. This cycle enables a dynamic audit process that continuously adapts to user audit requests and feedback.

[0408] The implementation of this system will significantly streamline audit operations and enable the delivery of consistent, high-quality audit reports.

[0409] The following describes the processing flow.

[0410] Step 1:

[0411] The server identifies the tasks covered by the audit project. The server retrieves relevant rules from the data store via queries and updates the information to the latest state.

[0412] Step 2:

[0413] The terminal is operated by the user, and meeting minutes audio files or submitted evidence documents are uploaded to the system. The terminal then sends this data to the server.

[0414] Step 3:

[0415] The server converts uploaded audio files into text using speech recognition technology. Subsequently, natural language processing is used to analyze the text data and identify risk factors.

[0416] Step 4:

[0417] The server retrieves past audit cases from the data store and compares the extracted risk elements against existing standards. It then generates a report template and writes the report content based on the evaluation results.

[0418] Step 5:

[0419] The server generates an audit report and notifies the user. The user can then review the report details via their terminal and make any necessary corrections or provide feedback.

[0420] Step 6:

[0421] Based on user feedback, the server updates the report and provides it to the user again as a final version. Corrections are made based on the feedback, and the audit process is completed.

[0422] (Example 1)

[0423] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0424] In modern business operations, internal audits need to be conducted quickly and accurately, but traditional methods face challenges such as the time and effort required for updating and analyzing information. Furthermore, it is difficult to dynamically update reports by incorporating user feedback.

[0425] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0426] In this invention, the server includes means for retrieving and managing business-related rules from a database, means for analyzing audio or document data using information processing technology, and means for automatically assembling results based on past cases. This automates the internal audit process, enabling efficient and accurate audits and dynamic report updates.

[0427] A "database" is a digital storage system that provides a mechanism for systematically collecting, storing, and efficiently accessing, managing, and updating information.

[0428] "Rules" are guidelines or rules that direct processes and standards related to business operations or procedures.

[0429] "Information processing technology" refers to the technology used to collect, analyze, store, and transmit data, and to utilize it for specific purposes.

[0430] "Audio or document data" refers to digital information, including audio recordings and text, that is subject to analysis.

[0431] "Past examples" refer to specific records and data from related projects and tasks that have been carried out to date.

[0432] "Automatically assembling" means that a system generates certain results or outputs based on pre-set criteria without requiring human intervention.

[0433] A "user" refers to an individual or organization that uses the system's functions to support their work.

[0434] "Feedback" refers to opinions and information provided by users regarding improvements and corrections.

[0435] A "dynamic report" is a report that is continuously modified and updated after it is generated, allowing it to reflect the latest information and feedback.

[0436] This system is designed to streamline internal audit operations and support the creation of accurate reports. The system consists of servers, terminals, and users working together, each playing a different role.

[0437] The server plays a central role in this system, managing business-related rules retrieved from the database and providing the functionality to update them with the latest information. Advanced information processing technologies are used on the server, utilizing natural language processing (NLP) to analyze speech and text data. Specific software examples include open-source natural language processing libraries and cloud-based data management systems.

[0438] The terminal is used when users upload audio interviews and evidence to the server. Before sending audio files to the server, the terminal converts them to the appropriate format. It also performs processes to optimize data size, such as converting audio files to MP3 format. This allows for efficient data transfer to the server.

[0439] Users are responsible for viewing the generated reports, reviewing their contents, and providing feedback. This feedback is aggregated on the server and used to improve reports and processes. This enables a dynamic and flexible audit process.

[0440] As a concrete example, in audits of financial transactions, if new rules are applied, the server can acquire that information in real time and incorporate it into the analysis process. For instance, by inputting a prompt such as "Prepare an audit report based on the new financial rules" into a generating AI model, it becomes possible to automatically create an advanced report that takes the latest rules into account.

[0441] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0442] Step 1:

[0443] The server retrieves the latest business-related rules from the database. The input is the rule information in the database, and the output is a list of retrieved rules. The server periodically queries the database as a scheduled task to update the necessary rules. This process involves retrieving data from the cloud database via a specific API.

[0444] Step 2:

[0445] Users use a terminal to upload audio interviews and evidence to the server. Input consists of audio and document files provided by the user, and output is a formatted file sent to the server. The terminal converts the audio to a standard format (e.g., MP3) and sends the document file to the server. The HTTP protocol is used for file transfer.

[0446] Step 3:

[0447] The server analyzes uploaded audio and document data. Input is a converted audio or document file, and output is the analyzed text data and risk assessment results. The server utilizes natural language processing (NLP) techniques to convert audio data into text, extract keywords, and evaluate risk factors. NLP libraries are used for the analysis.

[0448] Step 4:

[0449] The server automatically generates a report based on the analysis results, referencing past cases. The input consists of analyzed text data and risk assessment results, while the output is the constructed report. The server selects a template based on similar past cases and generates a customized report to suit the current data. The generated report is constructed using a document generation algorithm.

[0450] Step 5:

[0451] Users review the generated reports and provide feedback. The input is the generated report, and the output is the user's feedback information. Users review the report content and fill in their comments and suggestions for revisions using a stored digital feedback form. This feedback information is then sent to the server.

[0452] Step 6:

[0453] The server incorporates the collected feedback into the report and updates it as needed. The input is user feedback, and the output is the revised and updated final version of the report. The server analyzes the feedback, applies improvements to the report, and generates an updated version. The final version of the report is then saved again to cloud storage and shared with the user.

[0454] (Application Example 1)

[0455] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0456] Traditional internal audit processes faced challenges in quickly responding to rule changes and collecting and evaluating real-time information from the field, which compromised the accuracy and efficiency of audits. Furthermore, delays in incorporating feedback posed a risk of reduced audit quality.

[0457] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0458] In this invention, the server includes means for retrieving and maintaining rules from a data storage device, means for analyzing risk factors from speech or information using natural language processing, means for automatically generating report texts based on past information, and means for transmitting data directly from the field and evaluating risks. This enables rule changes and rapid information gathering and evaluation from the field, and allows for immediate reflection of feedback to improve the quality of audit results.

[0459] "Internal auditing" is an activity that evaluates the efficiency and adherence to regulations of an organization and identifies risks through the evaluation and auditing of operations and processes conducted within the organization.

[0460] "Relevant regulations" refer to the laws, policies, and standards that must be followed in specific business operations or activities, and are necessary to ensure the legality and appropriateness of the audited entity.

[0461] A "data storage device" is a hardware or software system that stores information long-term or short-term and allows it to be retrieved as needed.

[0462] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, and it involves methods for analyzing speech and text to extract meaning.

[0463] A "risk factor" is a risk element that could potentially cause unexpected problems or damages in the course of performing business operations.

[0464] "Past information" refers to data and events that have been previously recorded or reported, and is information that can be used as a reference for future evaluations and decision-making.

[0465] A "report" is a document created based on an audit or investigation, which details the evaluation results and recommendations.

[0466] A "remote communication terminal" is an electronic device that enables communication from geographically distant locations and is a means of sending and receiving data.

[0467] "Means of risk assessment" are methods or tools for quantitatively or qualitatively analyzing potential risk factors and determining their impact and probability.

[0468] To implement this invention, a server, terminals (such as smartphones or other remote communication devices), and users must cooperate to operate the system. The following details how the system functions.

[0469] The server first uses data storage to retrieve rules related to internal audits and keeps them constantly up-to-date. During this process, the server searches a database of rules to extract necessary information. It also utilizes natural language processing (NLP) techniques to analyze interview audio and document data to identify risk factors. Software libraries such as TensorFlow and NLTK are used for NLP techniques in this process.

[0470] The terminal collects audio and documents from the field via smart devices and uploads them to a server. The terminal's software is typically developed using Swift or Kotlin. This allows users to input data instantly in the field.

[0471] The server automatically generates reports based on data obtained by referencing past information. This process utilizes an AI model that has learned from past cases. As a result, evaluation criteria based on similar cases are provided.

[0472] The generated report is quickly provided to the user. The user can review the report and provide feedback as needed. This feedback is collected by the server and reflected in the report.

[0473] A concrete example is a security audit where an auditor uses their smartphone to record suspicious activity via audio and sends it to a server. The server then analyzes the audio, immediately performs a risk assessment, generates a report, and sends it to the administrator.

[0474] An example of a prompt used for a generative AI model is: "Automate the company's security audit process. Analyze interview audio and incident reports in real time, perform risk assessments, and generate reports quickly."

[0475] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0476] Step 1:

[0477] The terminal uses a smartphone to input voice and document data from the field and send it to the server. During this process, the application on the terminal converts the user's voice input into text and sends image data and other data as needed. The input data is uploaded to the server via the internet.

[0478] Step 2:

[0479] The server retrieves the latest relevant rules from the data storage device and begins analyzing the input data based on this information. The server uses natural language processing (NLP) to extract keywords from the transcribed speech data and identify risk factors. This is done using NLP libraries such as TensorFlow or NLTK. The output of this process becomes information for risk assessment.

[0480] Step 3:

[0481] The server automatically generates reports based on analyzed data, referencing historical information. Analyzed risk information is used as input. An AI model generates documents using appropriate report templates, referencing past audit data. This output is a document formatted as the final report.

[0482] Step 4:

[0483] The server provides the user with the generated report. This report is displayed to the user via a user interface. The user can review the report and enter their evaluation and feedback. This feedback is also sent to the server and incorporated into the report as needed.

[0484] Step 5:

[0485] Users provide feedback, which is then processed by the server. The server analyzes the feedback and makes necessary corrections to the report. Once this cycle is complete, a final report is created, which is used as the basis for improvement measures and feedback for the next audit.

[0486] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0487] This invention provides an automated evaluation system for internal audit processes that takes into account the emotional state of users. In addition to acquiring relevant rules, analyzing documents, automatically generating reports, and incorporating user feedback, this system utilizes an emotion engine to enable more precise risk assessment and process optimization.

[0488] This system is implemented through the cooperation of a server, terminals, and an emotion engine. First, the server retrieves relevant rules from the data store and updates them in real time, thereby responding immediately to changes in the audit environment.

[0489] Next, the process begins when the user uploads interview audio and submitted evidence documents to the system via their device. The server analyzes the provided materials using natural language processing technology and extracts risk factors. As part of this analysis, an emotion engine evaluates the user's statements and determines their emotional state. For example, if the interviewee is experiencing high levels of stress or anxiety, the system can identify the contributing factors.

[0490] The server integrates emotional data analyzed by the emotion engine into risk assessments and automatically generates reports while comparing them with past audit information. Because the assessment includes emotional state-based evaluations, it enables risk assessments from a more multifaceted perspective than before. For example, it may recommend additional audit actions for departments where unstable emotional states are frequently observed.

[0491] Finally, the generated report is provided to the user, and feedback is requested. User feedback is analyzed by an emotion engine and incorporated into the report to further improve its accuracy. This two-way feedback mechanism enables audit work to be conducted in a more efficient and user-friendly manner.

[0492] The following describes the processing flow.

[0493] Step 1:

[0494] The server identifies the business processes subject to internal audit and retrieves relevant rules from the data store. The server maintains up-to-date rules and updates them as needed.

[0495] Step 2:

[0496] The user uploads interview audio files and submitted evidence documents to the system using a terminal. The terminal then sends these files to the server.

[0497] Step 3:

[0498] The server converts uploaded audio files into text using speech recognition technology. Natural language processing is used to analyze the text data, and an emotion engine identifies the user's emotional state from their statements, along with risk factors.

[0499] Step 4:

[0500] The server analyzes the emotional state identified by the emotion engine and integrates it as part of the risk assessment. It combines historical audit information with emotion data to generate a report template.

[0501] Step 5:

[0502] The server automatically generates an audit report and notifies the user, providing the report's contents to their terminal. The user reviews the report and provides feedback to the server.

[0503] Step 6:

[0504] The server analyzes user feedback using an emotion engine and updates the report as needed. Changes in emotions based on the feedback are also evaluated, and the final version is provided to the user again.

[0505] (Example 2)

[0506] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0507] Traditional internal audit systems had a problem with incomplete risk assessment and identification of risk factors because risk assessments were conducted without considering the emotional state of users. Furthermore, user feedback was not adequately reflected in reports, resulting in a failure to optimize processes.

[0508] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0509] In this invention, the server includes means for acquiring and maintaining relevant rules from an information storage device, means for analyzing risk factors from voice data or documents using natural language processing, and means for identifying emotional states using an emotion evaluation function in the analysis. This enables multifaceted risk assessment that takes user emotions into account and improves the accuracy of reports based on feedback.

[0510] "Relevant regulations" refer to standards and guidelines used as reference when conducting internal audits, and include information on audit requirements and procedures.

[0511] An "information storage device" is an electronic device used to temporarily or long-term store data, and includes databases and storage servers.

[0512] "Natural language processing" is a technology that enables computers to understand and analyze human language, and it has the ability to analyze the meaning of audio and text data.

[0513] "Risk factors" refer to problems and challenges that a company or organization may face, and are important points to check during an audit.

[0514] "Emotional assessment functionality" refers to technologies used to automatically analyze and identify a user's emotional state, including technologies that infer emotions from voice tone and text.

[0515] A "report" refers to a document that summarizes the audit results and includes risk assessments and recommendations.

[0516] "Feedback" refers to the opinions and comments that users provide to reports and systems, and the information used to improve the system and enhance the accuracy of evaluations.

[0517] This invention describes embodiments for carrying it out. This system is an automated evaluation system that takes into account the emotional state of users in internal audits. The following are specific embodiments thereof.

[0518] The server first retrieves relevant rules from the information storage device and updates them in real time. This process uses a hosted database system to ensure that the rule data is accurate and up-to-date.

[0519] Next, the user uploads interview audio and submitted documents to the system via a terminal. The terminal can be a network-connected computer or tablet device. Users can easily upload data through a dedicated interface.

[0520] The server receives the uploaded data and analyzes it using natural language processing techniques. This process utilizes the Python programming language and libraries (e.g., NLTK and spaCy). During the analysis, the server uses sentiment assessment capabilities to identify emotional states from the user's utterances and voice data. This allows for the identification of emotions such as stress and anxiety.

[0521] Next, the server integrates sentiment data into the risk assessment and automatically generates a report using a generative AI model, comparing it with past audit information. This report provides a multifaceted evaluation of risk factors that are important to the user.

[0522] Finally, the generated report is provided to the user, and feedback information is received from the user. This feedback is then analyzed again using the sentiment rating function to help improve the accuracy of the report.

[0523] An example of a prompt message might be, "Use the AI ​​model to conduct a risk assessment based on the emotional state of a specific department. Propose specific emotional indicators and the audit actions based on them." This system makes the internal audit process more intuitive and effective for users.

[0524] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0525] Step 1:

[0526] The server retrieves relevant rules from the information storage device. It receives the latest rule data from the database as input and stores it in internal memory. The output is the latest rule dataset. This dataset is used as the basis for the audit process.

[0527] Step 2:

[0528] Users upload interview audio and supporting documents via their terminal. The input here is the file data selected by the user, and the output is the audio and document data stored in the server's storage area. Specifically, the user operates the terminal's interface, selects the file, and presses the upload button.

[0529] Step 3:

[0530] The server receives the uploaded data and performs analysis using natural language processing (NLP) techniques. The input consists of audio and text data uploaded by the user. The output is a list of identified risk factors. Specifically, it utilizes an NLP library to extract important keywords and risk topics from the text.

[0531] Step 4:

[0532] The server uses sentiment assessment functionality to identify the user's emotional state from their utterances. Input is information contained in audio and document data, and output is an indicator of emotional state (e.g., stress level, anxiety level). This process utilizes speech analysis equipment to perform specific actions such as determining the user's voice tone and context.

[0533] Step 5:

[0534] The server integrates sentiment data into risk assessment and automatically generates reports. As input, risk factors and sentiment indicators are integrated, and this data is analyzed by a generating AI model. The output is a detailed assessment report. Specifically, it executes an algorithm that evaluates risk from multiple perspectives, comparing it with past audit reports.

[0535] Step 6:

[0536] The user receives the generated report and provides feedback to the system. The input here consists of the user's opinions and comments based on the report. The system then analyzes this again, and an improved report template is created as output. This includes the specific actions of the user entering feedback and pressing the submit button.

[0537] (Application Example 2)

[0538] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0539] In the internal audit process, risk assessment using traditional methods was often not sufficiently multifaceted, and the emotional state of users was frequently not reflected in the risk assessment. As a result, traditional systems suffered from insufficient audit accuracy and efficiency, and furthermore, there were challenges in real-time situational awareness and appropriate audit actions.

[0540] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0541] In this invention, the server includes means for retrieving and maintaining relevant rules in internal audits from an information database, means for analyzing risk factors from recorded audio or informational materials using natural language processing, and means for performing sentiment analysis and integrating the user's emotional state into the risk assessment. This enables a multifaceted and real-time risk assessment that takes into account the user's emotional state during the audit process.

[0542] An "information database" is a collection of data that stores rules and information related to internal audits, and that can be retrieved and updated as needed.

[0543] "Natural language processing" is a technology that analyzes the language that humans use on a daily basis as data and extracts information from it.

[0544] "Recorded audio or informational materials" refers to audio recordings used in the audit process, as well as documents and digital data that serve as various sources of information.

[0545] "Risk factors" are elements that may affect operations or processes, and they indicate potential dangers or problems for the subject of the audit.

[0546] "Emotional analysis" is the process of identifying a user's emotional state and extracting that information by analyzing their voice or text.

[0547] "Real-time" refers to the immediacy of obtaining output or results the moment information processing or analysis is performed.

[0548] "Means of recommending audit actions" refers to the system's function of suggesting appropriate next steps or actions based on the results of sentiment analysis and risk assessment.

[0549] The system based on this invention automates the internal audit process and integrates sentiment analysis to achieve more sophisticated risk assessment and process optimization. This system is built primarily by leveraging feedback from servers, terminals, and users.

[0550] The server's primary role is to retrieve relevant rules from the information database and maintain them. The information database stores all the rules and related information necessary for internal audits and is used in real time during the daily audit process.

[0551] The device provides an interface for users to input voice data and upload documents. The uploaded data is analyzed using natural language processing technology to extract risk factors. Specifically, natural language processing engines such as the Google Cloud Natural Language API are used to convert voice data into text, and potential risks are detected from that text.

[0552] The server's sentiment analysis engine analyzes the user's emotional state and integrates this into the risk assessment. Sentiment analysis libraries such as IBM Watson Tone Analyzer are used for this analysis. Based on these results, the system proposes real-time audit actions and provides situation-appropriate recommendations to each stakeholder.

[0553] As a concrete example, imagine an auditor using smart glasses during an audit to capture audio data in real time, and sentiment analysis is performed on the spot. If this analysis detects that the stress level during the conversation is higher than normal, further investigation is recommended.

[0554] An example of a prompt to a generative AI model is, "How can I analyze the emotional state of participants during an audit interview and perform a real-time risk assessment?" Using this prompt, the generative AI can provide guidance on risk assessment tailored to specific situations.

[0555] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0556] Step 1:

[0557] The server retrieves relevant rules from the information database and converts them into a format that users can upload. The input is the information database, and the output is the retrieved relevant rules. The retrieved rules are then formatted as data for use in subsequent analysis processes.

[0558] Step 2:

[0559] The terminal receives audio or document data from the user and uploads it to the system. The input is the audio or document provided by the user, and the output is the uploaded data. The terminal receives the data via the user interface and transfers it to the server.

[0560] Step 3:

[0561] The server converts uploaded audio data into text using a natural language processing engine (Google Cloud Natural Language API). The input is audio data, and the output is converted text data. This automated process ensures high accuracy in transcribing audio content into text.

[0562] Step 4:

[0563] The server performs a process of analyzing text data and extracting risk factors. The input is the transformed text data, and the output is the extracted risk factors. The analysis is performed automatically based on previously acquired rules.

[0564] Step 5:

[0565] The server uses an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state and integrates this into the risk assessment. The input is text data, and the output is the result of the emotional state assessment. The emotional state is used as data to perform adjustments based on the assessment of risk factors.

[0566] Step 6:

[0567] The server proposes audit actions in real time based on emotional states and risk factors. The inputs are emotional states and risk factors, and the output is the proposed audit action. The proposed action is provided to the user immediately.

[0568] Step 7:

[0569] Users review the audit report and proposed actions provided by the server and provide feedback. The input is the proposed audit actions and report, and the output is the user's feedback. This feedback is returned to the system and used for further improvement.

[0570] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0571] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0572] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0573] [Fourth Embodiment]

[0574] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0575] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0576] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0577] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0578] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0579] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0580] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0581] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0582] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0583] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0584] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0585] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0586] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0587] This invention is a system for automating the internal audit process, and its main functions include retrieving relevant rules, analyzing information using natural language processing, automatically generating reports, and incorporating feedback to users.

[0588] This system is implemented through collaboration between servers, terminals, and users. Specifically, the server first uses a data store to retrieve and manage the latest rules related to the target business. This allows for real-time reflection of the audit impact of rule changes.

[0589] Next, interview audio and supporting documents provided by the user are uploaded from the terminal. The server uses natural language processing technology to analyze these materials. For example, interview audio is converted to text, and keyword detection and risk assessment are performed.

[0590] Furthermore, the server references past audit cases and automatically generates reports based on the data obtained. In this process, it uses similar past cases as models and constructs the report according to evaluation criteria. For example, in audits conducted in accordance with financial transaction regulations, the new report is constructed by referencing the format of past financial transaction audits.

[0591] Finally, the generated report is provided to the user. The user can review the report and provide feedback as needed. The feedback is collected by the server, reflected in the report, and updated as necessary. This cycle enables a dynamic audit process that continuously adapts to user audit requests and feedback.

[0592] The implementation of this system will significantly streamline audit operations and enable the delivery of consistent, high-quality audit reports.

[0593] The following describes the processing flow.

[0594] Step 1:

[0595] The server identifies the tasks covered by the audit project. The server retrieves relevant rules from the data store via queries and updates the information to the latest state.

[0596] Step 2:

[0597] The terminal is operated by the user, and meeting minutes audio files or submitted evidence documents are uploaded to the system. The terminal then sends this data to the server.

[0598] Step 3:

[0599] The server converts uploaded audio files into text using speech recognition technology. Subsequently, natural language processing is used to analyze the text data and identify risk factors.

[0600] Step 4:

[0601] The server retrieves past audit cases from the data store and compares the extracted risk elements against existing standards. It then generates a report template and writes the report content based on the evaluation results.

[0602] Step 5:

[0603] The server generates an audit report and notifies the user. The user can then review the report details via their terminal and make any necessary corrections or provide feedback.

[0604] Step 6:

[0605] Based on user feedback, the server updates the report and provides it to the user again as a final version. Corrections are made based on the feedback, and the audit process is completed.

[0606] (Example 1)

[0607] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0608] In modern business operations, internal audits need to be conducted quickly and accurately, but traditional methods face challenges such as the time and effort required for updating and analyzing information. Furthermore, it is difficult to dynamically update reports by incorporating user feedback.

[0609] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0610] In this invention, the server includes means for retrieving and managing business-related rules from a database, means for analyzing audio or document data using information processing technology, and means for automatically assembling results based on past cases. This automates the internal audit process, enabling efficient and accurate audits and dynamic report updates.

[0611] A "database" is a digital storage system that provides a mechanism for systematically collecting, storing, and efficiently accessing, managing, and updating information.

[0612] "Rules" are guidelines or rules that direct processes and standards related to business operations or procedures.

[0613] "Information processing technology" refers to the technology used to collect, analyze, store, and transmit data, and to utilize it for specific purposes.

[0614] "Audio or document data" refers to digital information, including audio recordings and text, that is subject to analysis.

[0615] "Past examples" refer to specific records and data from related projects and tasks that have been carried out to date.

[0616] "Automatically assembling" means that a system generates certain results or outputs based on pre-set criteria without requiring human intervention.

[0617] A "user" refers to an individual or organization that uses the system's functions to support their work.

[0618] "Feedback" refers to opinions and information provided by users regarding improvements and corrections.

[0619] A "dynamic report" is a report that is continuously modified and updated after it is generated, allowing it to reflect the latest information and feedback.

[0620] This system is designed to streamline internal audit operations and support the creation of accurate reports. The system consists of servers, terminals, and users working together, each playing a different role.

[0621] The server plays a central role in this system, managing business-related rules retrieved from the database and providing the functionality to update them with the latest information. Advanced information processing technologies are used on the server, utilizing natural language processing (NLP) to analyze speech and text data. Specific software examples include open-source natural language processing libraries and cloud-based data management systems.

[0622] The terminal is used when users upload audio interviews and evidence to the server. Before sending audio files to the server, the terminal converts them to the appropriate format. It also performs processes to optimize data size, such as converting audio files to MP3 format. This allows for efficient data transfer to the server.

[0623] Users are responsible for viewing the generated reports, reviewing their contents, and providing feedback. This feedback is aggregated on the server and used to improve reports and processes. This enables a dynamic and flexible audit process.

[0624] As a concrete example, in audits of financial transactions, if new rules are applied, the server can acquire that information in real time and incorporate it into the analysis process. For instance, by inputting a prompt such as "Prepare an audit report based on the new financial rules" into a generating AI model, it becomes possible to automatically create an advanced report that takes the latest rules into account.

[0625] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0626] Step 1:

[0627] The server retrieves the latest business-related rules from the database. The input is the rule information in the database, and the output is a list of retrieved rules. The server periodically queries the database as a scheduled task to update the necessary rules. This process involves retrieving data from the cloud database via a specific API.

[0628] Step 2:

[0629] Users use a terminal to upload audio interviews and evidence to the server. Input consists of audio and document files provided by the user, and output is a formatted file sent to the server. The terminal converts the audio to a standard format (e.g., MP3) and sends the document file to the server. The HTTP protocol is used for file transfer.

[0630] Step 3:

[0631] The server analyzes uploaded audio and document data. Input is a converted audio or document file, and output is the analyzed text data and risk assessment results. The server utilizes natural language processing (NLP) techniques to convert audio data into text, extract keywords, and evaluate risk factors. NLP libraries are used for the analysis.

[0632] Step 4:

[0633] The server automatically generates a report based on the analysis results, referencing past cases. The input consists of analyzed text data and risk assessment results, while the output is the constructed report. The server selects a template based on similar past cases and generates a customized report to suit the current data. The generated report is constructed using a document generation algorithm.

[0634] Step 5:

[0635] Users review the generated reports and provide feedback. The input is the generated report, and the output is the user's feedback information. Users review the report content and fill in their comments and suggestions for revisions using a stored digital feedback form. This feedback information is then sent to the server.

[0636] Step 6:

[0637] The server incorporates the collected feedback into the report and updates it as needed. The input is user feedback, and the output is the revised and updated final version of the report. The server analyzes the feedback, applies improvements to the report, and generates an updated version. The final version of the report is then saved again to cloud storage and shared with the user.

[0638] (Application Example 1)

[0639] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0640] Traditional internal audit processes faced challenges in quickly responding to rule changes and collecting and evaluating real-time information from the field, which compromised the accuracy and efficiency of audits. Furthermore, delays in incorporating feedback posed a risk of reduced audit quality.

[0641] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0642] In this invention, the server includes means for retrieving and maintaining rules from a data storage device, means for analyzing risk factors from speech or information using natural language processing, means for automatically generating report texts based on past information, and means for transmitting data directly from the field and evaluating risks. This enables rule changes and rapid information gathering and evaluation from the field, and allows for immediate reflection of feedback to improve the quality of audit results.

[0643] "Internal auditing" is an activity that evaluates the efficiency and adherence to regulations of an organization and identifies risks through the evaluation and auditing of operations and processes conducted within the organization.

[0644] "Relevant regulations" refer to the laws, policies, and standards that must be followed in specific business operations or activities, and are necessary to ensure the legality and appropriateness of the audited entity.

[0645] A "data storage device" is a hardware or software system that stores information long-term or short-term and allows it to be retrieved as needed.

[0646] "Natural language processing" is a technology that enables computers to understand, interpret, and generate human language, and it involves methods for analyzing speech and text to extract meaning.

[0647] A "risk factor" is a risk element that could potentially cause unexpected problems or damages in the course of performing business operations.

[0648] "Past information" refers to data and events that have been previously recorded or reported, and is information that can be used as a reference for future evaluations and decision-making.

[0649] A "report" is a document created based on an audit or investigation, which details the evaluation results and recommendations.

[0650] A "remote communication terminal" is an electronic device that enables communication from geographically distant locations and is a means of sending and receiving data.

[0651] "Means of risk assessment" are methods or tools for quantitatively or qualitatively analyzing potential risk factors and determining their impact and probability.

[0652] To implement this invention, a server, terminals (such as smartphones or other remote communication devices), and users must cooperate to operate the system. The following details how the system functions.

[0653] The server first uses data storage to retrieve rules related to internal audits and keeps them constantly up-to-date. During this process, the server searches a database of rules to extract necessary information. It also utilizes natural language processing (NLP) techniques to analyze interview audio and document data to identify risk factors. Software libraries such as TensorFlow and NLTK are used for NLP techniques in this process.

[0654] The terminal collects audio and documents from the field via smart devices and uploads them to a server. The terminal's software is typically developed using Swift or Kotlin. This allows users to input data instantly in the field.

[0655] The server automatically generates reports based on data obtained by referencing past information. This process utilizes an AI model that has learned from past cases. As a result, evaluation criteria based on similar cases are provided.

[0656] The generated report is quickly provided to the user. The user can review the report and provide feedback as needed. This feedback is collected by the server and reflected in the report.

[0657] A concrete example is a security audit where an auditor uses their smartphone to record suspicious activity via audio and sends it to a server. The server then analyzes the audio, immediately performs a risk assessment, generates a report, and sends it to the administrator.

[0658] An example of a prompt used for a generative AI model is: "Automate the company's security audit process. Analyze interview audio and incident reports in real time, perform risk assessments, and generate reports quickly."

[0659] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0660] Step 1:

[0661] The terminal uses a smartphone to input voice and document data from the field and send it to the server. During this process, the application on the terminal converts the user's voice input into text and sends image data and other data as needed. The input data is uploaded to the server via the internet.

[0662] Step 2:

[0663] The server retrieves the latest relevant rules from the data storage device and begins analyzing the input data based on this information. The server uses natural language processing (NLP) to extract keywords from the transcribed speech data and identify risk factors. This is done using NLP libraries such as TensorFlow or NLTK. The output of this process becomes information for risk assessment.

[0664] Step 3:

[0665] The server automatically generates reports based on analyzed data, referencing historical information. Analyzed risk information is used as input. An AI model generates documents using appropriate report templates, referencing past audit data. This output is a document formatted as the final report.

[0666] Step 4:

[0667] The server provides the user with the generated report. This report is displayed to the user via a user interface. The user can review the report and enter their evaluation and feedback. This feedback is also sent to the server and incorporated into the report as needed.

[0668] Step 5:

[0669] Users provide feedback, which is then processed by the server. The server analyzes the feedback and makes necessary corrections to the report. Once this cycle is complete, a final report is created, which is used as the basis for improvement measures and feedback for the next audit.

[0670] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0671] This invention provides an automated evaluation system for internal audit processes that takes into account the emotional state of users. In addition to acquiring relevant rules, analyzing documents, automatically generating reports, and incorporating user feedback, this system utilizes an emotion engine to enable more precise risk assessment and process optimization.

[0672] This system is implemented through the cooperation of a server, terminals, and an emotion engine. First, the server retrieves relevant rules from the data store and updates them in real time, thereby responding immediately to changes in the audit environment.

[0673] Next, the process begins when the user uploads interview audio and submitted evidence documents to the system via their device. The server analyzes the provided materials using natural language processing technology and extracts risk factors. As part of this analysis, an emotion engine evaluates the user's statements and determines their emotional state. For example, if the interviewee is experiencing high levels of stress or anxiety, the system can identify the contributing factors.

[0674] The server integrates emotional data analyzed by the emotion engine into risk assessments and automatically generates reports while comparing them with past audit information. Because the assessment includes emotional state-based evaluations, it enables risk assessments from a more multifaceted perspective than before. For example, it may recommend additional audit actions for departments where unstable emotional states are frequently observed.

[0675] Finally, the generated report is provided to the user, and feedback is requested. User feedback is analyzed by an emotion engine and incorporated into the report to further improve its accuracy. This two-way feedback mechanism enables audit work to be conducted in a more efficient and user-friendly manner.

[0676] The following describes the processing flow.

[0677] Step 1:

[0678] The server identifies the business processes subject to internal audit and retrieves relevant rules from the data store. The server maintains up-to-date rules and updates them as needed.

[0679] Step 2:

[0680] The user uploads interview audio files and submitted evidence documents to the system using a terminal. The terminal then sends these files to the server.

[0681] Step 3:

[0682] The server converts uploaded audio files into text using speech recognition technology. Natural language processing is used to analyze the text data, and an emotion engine identifies the user's emotional state from their statements, along with risk factors.

[0683] Step 4:

[0684] The server analyzes the emotional state identified by the emotion engine and integrates it as part of the risk assessment. It combines historical audit information with emotion data to generate a report template.

[0685] Step 5:

[0686] The server automatically generates an audit report and notifies the user, providing the report's contents to their terminal. The user reviews the report and provides feedback to the server.

[0687] Step 6:

[0688] The server analyzes user feedback using an emotion engine and updates the report as needed. Changes in emotions based on the feedback are also evaluated, and the final version is provided to the user again.

[0689] (Example 2)

[0690] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0691] Traditional internal audit systems had a problem with incomplete risk assessment and identification of risk factors because risk assessments were conducted without considering the emotional state of users. Furthermore, user feedback was not adequately reflected in reports, resulting in a failure to optimize processes.

[0692] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0693] In this invention, the server includes means for acquiring and maintaining relevant rules from an information storage device, means for analyzing risk factors from voice data or documents using natural language processing, and means for identifying emotional states using an emotion evaluation function in the analysis. This enables multifaceted risk assessment that takes user emotions into account and improves the accuracy of reports based on feedback.

[0694] "Relevant regulations" refer to standards and guidelines used as reference when conducting internal audits, and include information on audit requirements and procedures.

[0695] An "information storage device" is an electronic device used to temporarily or long-term store data, and includes databases and storage servers.

[0696] "Natural language processing" is a technology that enables computers to understand and analyze human language, and it has the ability to analyze the meaning of audio and text data.

[0697] "Risk factors" refer to problems and challenges that a company or organization may face, and are important points to check during an audit.

[0698] "Emotional assessment functionality" refers to technologies used to automatically analyze and identify a user's emotional state, including technologies that infer emotions from voice tone and text.

[0699] A "report" refers to a document that summarizes the audit results and includes risk assessments and recommendations.

[0700] "Feedback" refers to the opinions and comments that users provide to reports and systems, and the information used to improve the system and enhance the accuracy of evaluations.

[0701] This invention describes embodiments for carrying it out. This system is an automated evaluation system that takes into account the emotional state of users in internal audits. The following are specific embodiments thereof.

[0702] The server first retrieves relevant rules from the information storage device and updates them in real time. This process uses a hosted database system to ensure that the rule data is accurate and up-to-date.

[0703] Next, the user uploads interview audio and submitted documents to the system via a terminal. The terminal can be a network-connected computer or tablet device. Users can easily upload data through a dedicated interface.

[0704] The server receives the uploaded data and analyzes it using natural language processing techniques. This process utilizes the Python programming language and libraries (e.g., NLTK and spaCy). During the analysis, the server uses sentiment assessment capabilities to identify emotional states from the user's utterances and voice data. This allows for the identification of emotions such as stress and anxiety.

[0705] Next, the server integrates sentiment data into the risk assessment and automatically generates a report using a generative AI model, comparing it with past audit information. This report provides a multifaceted evaluation of risk factors that are important to the user.

[0706] Finally, the generated report is provided to the user, and feedback information is received from the user. This feedback is then analyzed again using the sentiment rating function to help improve the accuracy of the report.

[0707] An example of a prompt message might be, "Use the AI ​​model to conduct a risk assessment based on the emotional state of a specific department. Propose specific emotional indicators and the audit actions based on them." This system makes the internal audit process more intuitive and effective for users.

[0708] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0709] Step 1:

[0710] The server retrieves relevant rules from the information storage device. It receives the latest rule data from the database as input and stores it in internal memory. The output is the latest rule dataset. This dataset is used as the basis for the audit process.

[0711] Step 2:

[0712] Users upload interview audio and supporting documents via their terminal. The input here is the file data selected by the user, and the output is the audio and document data stored in the server's storage area. Specifically, the user operates the terminal's interface, selects the file, and presses the upload button.

[0713] Step 3:

[0714] The server receives the uploaded data and performs analysis using natural language processing (NLP) techniques. The input consists of audio and text data uploaded by the user. The output is a list of identified risk factors. Specifically, it utilizes an NLP library to extract important keywords and risk topics from the text.

[0715] Step 4:

[0716] The server uses sentiment assessment functionality to identify the user's emotional state from their utterances. Input is information contained in audio and document data, and output is an indicator of emotional state (e.g., stress level, anxiety level). This process utilizes speech analysis equipment to perform specific actions such as determining the user's voice tone and context.

[0717] Step 5:

[0718] The server integrates sentiment data into risk assessment and automatically generates reports. As input, risk factors and sentiment indicators are integrated, and this data is analyzed by a generating AI model. The output is a detailed assessment report. Specifically, it executes an algorithm that evaluates risk from multiple perspectives, comparing it with past audit reports.

[0719] Step 6:

[0720] The user receives the generated report and provides feedback to the system. The input here consists of the user's opinions and comments based on the report. The system then analyzes this again, and an improved report template is created as output. This includes the specific actions of the user entering feedback and pressing the submit button.

[0721] (Application Example 2)

[0722] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0723] In the internal audit process, risk assessment using traditional methods was often not sufficiently multifaceted, and the emotional state of users was frequently not reflected in the risk assessment. As a result, traditional systems suffered from insufficient audit accuracy and efficiency, and furthermore, there were challenges in real-time situational awareness and appropriate audit actions.

[0724] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0725] In this invention, the server includes means for retrieving and maintaining relevant rules in internal audits from an information database, means for analyzing risk factors from recorded audio or informational materials using natural language processing, and means for performing sentiment analysis and integrating the user's emotional state into the risk assessment. This enables a multifaceted and real-time risk assessment that takes into account the user's emotional state during the audit process.

[0726] An "information database" is a collection of data that stores rules and information related to internal audits, and that can be retrieved and updated as needed.

[0727] "Natural language processing" is a technology that analyzes the language that humans use on a daily basis as data and extracts information from it.

[0728] "Recorded audio or informational materials" refers to audio recordings used in the audit process, as well as documents and digital data that serve as various sources of information.

[0729] "Risk factors" are elements that may affect operations or processes, and they indicate potential dangers or problems for the subject of the audit.

[0730] "Emotional analysis" is the process of identifying a user's emotional state and extracting that information by analyzing their voice or text.

[0731] "Real-time" refers to the immediacy of obtaining output or results the moment information processing or analysis is performed.

[0732] "Means of recommending audit actions" refers to the system's function of suggesting appropriate next steps or actions based on the results of sentiment analysis and risk assessment.

[0733] The system based on this invention automates the internal audit process and integrates sentiment analysis to achieve more sophisticated risk assessment and process optimization. This system is built primarily by leveraging feedback from servers, terminals, and users.

[0734] The server's primary role is to retrieve relevant rules from the information database and maintain them. The information database stores all the rules and related information necessary for internal audits and is used in real time during the daily audit process.

[0735] The device provides an interface for users to input voice data and upload documents. The uploaded data is analyzed using natural language processing technology to extract risk factors. Specifically, natural language processing engines such as the Google Cloud Natural Language API are used to convert voice data into text, and potential risks are detected from that text.

[0736] The server's sentiment analysis engine analyzes the user's emotional state and integrates this into the risk assessment. Sentiment analysis libraries such as IBM Watson Tone Analyzer are used for this analysis. Based on these results, the system proposes real-time audit actions and provides situation-appropriate recommendations to each stakeholder.

[0737] As a concrete example, imagine an auditor using smart glasses during an audit to capture audio data in real time, and sentiment analysis is performed on the spot. If this analysis detects that the stress level during the conversation is higher than normal, further investigation is recommended.

[0738] An example of a prompt to a generative AI model is, "How can I analyze the emotional state of participants during an audit interview and perform a real-time risk assessment?" Using this prompt, the generative AI can provide guidance on risk assessment tailored to specific situations.

[0739] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0740] Step 1:

[0741] The server retrieves relevant rules from the information database and converts them into a format that users can upload. The input is the information database, and the output is the retrieved relevant rules. The retrieved rules are then formatted as data for use in subsequent analysis processes.

[0742] Step 2:

[0743] The terminal receives audio or document data from the user and uploads it to the system. The input is the audio or document provided by the user, and the output is the uploaded data. The terminal receives the data via the user interface and transfers it to the server.

[0744] Step 3:

[0745] The server converts uploaded audio data into text using a natural language processing engine (Google Cloud Natural Language API). The input is audio data, and the output is converted text data. This automated process ensures high accuracy in transcribing audio content into text.

[0746] Step 4:

[0747] The server performs a process of analyzing text data and extracting risk factors. The input is the transformed text data, and the output is the extracted risk factors. The analysis is performed automatically based on previously acquired rules.

[0748] Step 5:

[0749] The server uses an emotion analysis engine (IBM Watson Tone Analyzer) to analyze the user's emotional state and integrates this into the risk assessment. The input is text data, and the output is the result of the emotional state assessment. The emotional state is used as data to perform adjustments based on the assessment of risk factors.

[0750] Step 6:

[0751] The server proposes audit actions in real time based on emotional states and risk factors. The inputs are emotional states and risk factors, and the output is the proposed audit action. The proposed action is provided to the user immediately.

[0752] Step 7:

[0753] Users review the audit report and proposed actions provided by the server and provide feedback. The input is the proposed audit actions and report, and the output is the user's feedback. This feedback is returned to the system and used for further improvement.

[0754] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0755] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0756] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0757] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0758] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0759] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0760] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0761] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0762] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0763] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0764] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0765] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0766] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0767] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0768] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0769] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0770] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0771] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0772] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0773] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0774] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0775] The following is further disclosed regarding the embodiments described above.

[0776] (Claim 1)

[0777] Means for retrieving and maintaining relevant rules for internal audits from a data store,

[0778] A means of analyzing risk factors from meeting transcripts or documents using natural language processing,

[0779] A means of automatically generating reports based on past audit information,

[0780] The means of evaluating the contents of the report and providing it to the user,

[0781] A system that includes this.

[0782] (Claim 2)

[0783] The system according to claim 1, comprising means for scoring and comparing analyzed data with existing standards in order to assess risks in internal audits.

[0784] (Claim 3)

[0785] The system according to claim 1, comprising means for incorporating user feedback into a report and generating a final version thereof.

[0786] "Example 1"

[0787] (Claim 1)

[0788] A means of obtaining and managing business-related rules from a database,

[0789] A means of analyzing audio or document data using information processing technology,

[0790] A means of automatically generating results based on past cases,

[0791] Means for verifying the results and providing them to users,

[0792] A means of collecting information from users and reflecting it in the results,

[0793] A system that includes this.

[0794] (Claim 2)

[0795] The system according to claim 1, comprising means for comparing and evaluating the analyzed information using a standard.

[0796] (Claim 3)

[0797] The system according to claim 1, comprising means for updating content automatically generated using a generative AI model.

[0798] "Application Example 1"

[0799] (Claim 1)

[0800] Means for retrieving and maintaining relevant rules in internal audits from data storage devices,

[0801] A means of analyzing risk factors from interview audio or information using natural language processing,

[0802] A means of automatically generating reports based on past audit information,

[0803] A means of evaluating the content of the report and providing it to the user,

[0804] A means of quickly assessing risks by transmitting data directly from the site using remote communication terminals,

[0805] A system that includes this.

[0806] (Claim 2)

[0807] The system according to claim 1, comprising means for evaluating and comparing analyzed data with existing standards in order to assess risks in internal audits.

[0808] (Claim 3)

[0809] The system according to claim 1, comprising means for incorporating user feedback into a report and generating a final version thereof.

[0810] "Example 2 of combining an emotion engine"

[0811] (Claim 1)

[0812] Means for obtaining and maintaining relevant rules from an information storage device,

[0813] A means of analyzing risk factors from audio data or documents using natural language processing,

[0814] A means of identifying emotional states using an emotion evaluation function in the analysis,

[0815] A means of integrating emotional data and automatically generating reports based on past audit information,

[0816] A means of evaluating the generated report and providing it to the user,

[0817] A system that includes this.

[0818] (Claim 2)

[0819] The system according to claim 1, comprising means for scoring and comparing the analyzed information with existing criteria and evaluating the risk.

[0820] (Claim 3)

[0821] The system according to claim 1, comprising means for incorporating user feedback into a report and generating a final version.

[0822] "Application example 2 when combining with an emotional engine"

[0823] (Claim 1)

[0824] Means for obtaining and maintaining relevant rules for internal audits from an information database,

[0825] A means of analyzing risk factors from recorded audio or informational materials using natural language processing,

[0826] A means of automatically generating reports based on past audit information,

[0827] The means of evaluating the contents of the report and providing it to users,

[0828] A means of performing emotion analysis and integrating the user's emotional state into risk assessment,

[0829] A means of recommending audit actions in real time based on the user's emotional state,

[0830] A system that includes this.

[0831] (Claim 2)

[0832] The system according to claim 1, comprising means for comparing analyzed information with existing standards and numerical evaluations in order to assess risks in internal audits.

[0833] (Claim 3)

[0834] The system according to claim 1, comprising means for incorporating user feedback into a report and generating a final version thereof. [Explanation of symbols]

[0835] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for retrieving and maintaining relevant rules for internal audits from a data store, A means of analyzing risk factors from meeting transcripts or documents using natural language processing, A means of automatically generating reports based on past audit information, The means of evaluating the contents of the report and providing it to the user, A system that includes this.

2. The system according to claim 1, comprising means for scoring and comparing analyzed data with existing standards in order to assess risks in internal audits.

3. The system according to claim 1, comprising means for incorporating user feedback into a report and generating a final version thereof.

Images