system

Abstract

We provide the system. [Solution] The present invention relates to a business support system centered on generative AI, and includes a corporate-specific information management department, a communication analysis department, a business support proposal department, etc. It comprehensively stores and manages corporate business knowledge, business rules, workflows, management indicators, employee skills, project information, past communication history, etc., as long-term memory, and analyzes all digital communications such as emails, chats, and meeting records in real time to detect progress delays, uneven workload distribution, skill deficiencies, risk factors, etc. Furthermore, it is a system that uses generative AI to execute optimal proposals and support in real time and comprehensively for improving business efficiency, quality improvement, risk reduction, skill development, project management, information sharing, and business automation, thereby realizing improved productivity and sustainable growth for the entire organization.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method executed by at least one processor, the method comprising the steps of: receiving a user utterance; adding the user utterance to a prompt including an instruction sentence associated with a description related to a character of the chatbot; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] The problem that this invention aims to solve is that in modern organizations, where the digitalization of operations and communication has progressed, it is difficult to extend DX and AI technologies beyond mere trends or partial automation to concrete and sustainable impacts on the entire organization's business processes, project management, and employee work styles and skill development. In other words, conventional business support systems and project management tools are unable to integrate and grasp vast and complex information, such as company-specific business knowledge and rules, workflows, management indicators, individual employee skills and career goals, and detailed progress of ongoing projects, in a long-term manner, and to provide optimal proposals and support in real time. Furthermore, it is difficult to understand the context of all communications and provide objective and logical advice in a timely manner. As a result, essential organizational advancements such as improved operational efficiency, quality enhancement, risk reduction, employee skill development, and increased project success rates cannot be achieved, and employees end up spending a lot of time on mundane and uncreative tasks, unable to allocate sufficient resources to the creative and strategic work they should be focusing on. Furthermore, conventional systems have had to rely on individual responses and person-dependent operations to address a variety of problems that occur at the operational level, such as uneven workload distribution, increased overtime, stagnant information sharing, project delays due to lack of skills, and overlooked risks, hindering improvements in overall organizational productivity and work-life balance. This invention aims to fundamentally solve these problems by utilizing generative AI to combine company-specific knowledge, advanced expertise, and contextual understanding through long-term memory. By acting as virtual colleagues and a virtual project management department, it comprehensively grasps all communication and business information within the organization and provides real-time, comprehensive, and optimal business support and proposals, thereby improving organizational productivity, reducing risks, enhancing employee skills, and improving work-life balance. [Means for solving the problem]

[0005] The means for solving the problems according to the present invention is to provide a system that operates using generative AI, which comprises a corporate-specific information management department, a communication analysis department, and a business support proposal department. The corporate-specific information management department integrates and manages corporate-specific information such as internal business knowledge, business rules, workflows, management indicators, KPIs, issues, ongoing initiatives, project information, individual employee skill sets and career goals, and past communication history as long-term memory, and also has the function of retaining general industry knowledge and specialized knowledge. The communication analysis department constantly participates in all digital communications such as email, chat, and meeting minutes, and uses natural language processing by generative AI to analyze the context of communication content and address issues such as progress delays, frequent manual work, stagnation of information sharing, uneven workload, increased overtime, lack of skills, and risk factors. Equipped with real-time detection capabilities, the Business Support Proposal Department, based on information obtained from the aforementioned Corporate-Specific Information Management Department and Communication Analysis Department, uses generative AI to propose specific improvement measures for streamlining and improving the quality of business processes and workflows, determine the necessity of meetings and schedule them, support the automation of applications and procedures, propose task redistribution and work adjustments, propose learning opportunities and training for skill development and career building for each employee, automatically generate project plans and documents, provide budget management and cost reduction proposals, monitor task progress and notify of delays, and conduct risk assessments and propose countermeasures. By executing these tasks in real time and comprehensively in accordance with the organization's actual situation and context, it is a means to improve the overall operational efficiency of the organization, reduce risks, improve employee skills, increase project success rates, and improve work-life balance. [Brief explanation of the drawing]

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0007] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0008] First, let's explain the terminology used in the following explanation.

[0009] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0010] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0011] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0012] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be A alone, B alone, or a combination of A and B. In addition, in the present specification, when three or more matters are expressed in a connected manner using "and / or", the same concept as that for "A and / or B" applies.

[0014] [First Embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] As shown in FIG. 1, the data processing system 10 includes a data processing apparatus 12 and a smart device 14. An example of the data processing apparatus 12 includes a server.

[0017] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of the "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network) and the like.

[0018] The smart device 14 includes a computer 36, a reception apparatus 38, an output apparatus 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception apparatus 38, output apparatus 40, camera 42, and communication I / F 44 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user inputs. The touch panel 38A receives a user input by contact of an indicator (for example, a pen, a finger, or the like) by detecting the contact of the indicator. The microphone 38B receives a user input by voice by detecting the user's voice. A control unit 46A in a processor 46 transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, a specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A, a speaker 40B, and the like, and presents data to the user 20 by outputting the data in a representation form perceptible by the user 20 (for example, voice and / or text). The display 40A displays visible information such as text and images in accordance with an instruction from the processor 46. The speaker 40B outputs voice in accordance with an instruction from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 manage the exchange of various types of information between the processor 46 and the processor 28 via the network 54.

[0022] Figure 2 illustrates an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0027] (Example 1)

[0028] The flow of a specific process in Example 1 will now be described. Each part of the system described below is implemented by a data processing device 12 and a smart device 14. The data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0029] (Embodiments for carrying out the present invention)

[0030] An embodiment for carrying out the present invention is an advanced business support system centered on a generative AI and realized through cooperation between a server and a terminal. Each component will be described in more specific and detailed terms below.

[0031] First, the Corporate Information Management Department is implemented on a server, centrally storing and managing all information assets within the organization. For example, it stores vast amounts of information as structured and unstructured data, such as the quotation creation procedures of the sales department, the expense reimbursement flow of the accounting department, the source code management rules of the development department, the inspection standards of the quality control department, the ordering process of the purchasing department, the recruitment, evaluation, and transfer history of the human resources department, the contract templates and legal compliance checklists of the legal department, the minutes of regular meetings of each department, the progress reports of each project, the qualification acquisition history and goal setting sheets of each employee, past trouble cases and their response history, internal FAQs, various manuals, internal regulations, industry standards and legal revision information, and external benchmark data. This information is utilized as a knowledge base that goes beyond a simple database through functions such as knowledge graphing, semantic tagging, time-series management, importance scoring, and relationship network construction using generative AI. For example, if an employee is assigned to a new project, the system can automatically extract information such as the employee's past project participation history, challenges and success factors encountered in similar projects, relevant operational manuals and FAQs, necessary internal procedures and application form templates, and risk analysis results from similar past cases, and present this information to relevant parties in a timely manner.

[0032] The communication analysis unit distributes its functions across both the server and the terminal. On the terminal side, it collects data in real time from all communication channels, including email, chat, web conferencing, groupware, social networking services, task management tools, calendars, and file sharing services, and sends it to the server. For example, it uses the chat tool's API to retrieve all messages, automatically uploads meeting system recordings and minutes, and periodically synchronizes the sending and receiving history from the mail server. On the server side, a generative AI-based natural language processing engine analyzes this communication data sequentially. For example, it extracts keywords and phrases from meeting minutes and chat exchanges such as "tight deadline," "additional work has arisen," "person in charge is unwell," "approval is delayed," "information sharing is insufficient," "a lot of manual work," "overtime is increasing," "lack of skills," and "risk is rising," and performs multifaceted contextual analysis, including linking the speaker's position and department, past speaking trends, sentiment analysis, conversation flow, and related business events. For example, if multiple instances of statements like "design is behind schedule," "test environment is not ready," or "additional requirements have come in from the customer" are detected in a project chat, the AI ​​will comprehensively analyze these and detect delays, resource shortages, and requirements change risks early on. It can also automatically transcribe meeting audio data into text, extract important decisions, unresolved issues, and action items, and notify relevant parties.

[0033] The business support proposal department is primarily implemented on the server, but a user interface is also provided on the terminal side. The generative AI on the server comprehensively analyzes information obtained from the corporate-specific information management department and the communication analysis department, and generates optimal business support and proposals in real time. For example, if it detects a task that involves manually entering large amounts of data on a regular basis, the AI ​​automatically generates prototypes of RPA or macros and presents them to the person in charge. For example, if the accounting department is manually entering monthly invoice data into the system, the AI ​​learns past input patterns and data formats, and automatically generates RPA scenarios and Excel macros and proposes them to the person in charge. If information sharing is stalled, it proposes the best way to use chat tools and project management tools, and presents specific setting examples, operational rules, templates, automatic creation of notification rules, and workflow automation proposals. For example, if the sales department is behind in sharing deal information, the AI ​​proposes custom views for deal management tools, automatic notification settings, and automatic generation of weekly reports.

[0034] If the AI ​​detects that a project is behind schedule, it automatically checks the schedules of all stakeholders and proposes and sets up an emergency meeting at the optimal time. For example, if the design phase of a development project is delayed, the AI ​​automatically matches the schedules of the designers, project managers, and related departments, extracts the earliest time slot when everyone can attend, automatically sets up a meeting, and automatically prepares the necessary agenda and materials. When launching a new project, it automatically extracts relevant departments and members, adjusts the schedule for the kick-off meeting, automatically generates necessary materials, creates a draft project plan, automatically generates a risk list, and automatically generates a role assignment chart.

[0035] The system monitors each employee's workload and overtime in real time, and if tasks are concentrated on a particular employee, it proposes task redistribution considering the availability and skill sets of other employees. For example, if a bug-fixing task is concentrated on a specific engineer in the development department, the AI ​​will refer to the schedules, past performance, skill maps, and current task status of other engineers to propose an optimized task allocation. If an employee's overtime is increasing, the system will alert their supervisor about the need for work adjustments and automatically generate specific adjustment proposals, work reduction proposals, outsourcing proposals, and task automation proposals. For example, if a particular employee's overtime is increasing in the sales department, the AI ​​will analyze that employee's work content and propose automation of routine tasks, task delegation to other members, and revisions to business processes.

[0036] The system automatically matches employees' skill sets with the skills required for ongoing or planned projects. If a skill deficiency is detected, it automatically searches for and suggests relevant in-house training or external online courses. For example, if an employee assigned to an AI project has limited experience in machine learning, the AI ​​will present information on appropriate e-learning courses, in-house study groups, external seminars, reference books, and certification programs, and automatically adjust the training schedule. It also creates long-term skill acquisition plans aligned with employees' career goals, regularly follows up on their progress, and, if goals are not met, suggests additional learning opportunities, mentor recommendations, or on-the-job training.

[0037] In project management, the system analyzes the success factors and failures of similar past projects during the planning phase and automatically generates project plan templates. For example, if a construction project has frequently experienced delays in the past, the system automatically incorporates the analysis of causes and countermeasures into the plan, and automatically generates risk lists, milestones, Gantt charts, budget allocation proposals, role assignment charts, and progress management sheets. During the execution phase, it aggregates expenditure data in real time, immediately issues alerts if there are signs of budget overruns, and proposes cost reduction and alternative solutions. For example, if outsourcing costs are likely to exceed the budget, the AI ​​suggests quotes from other outsourcing partners, past performance data, proposals for utilizing internal resources, proposals for shortening the process, and proposals for changing specifications. In task progress management, it constantly monitors the planned completion date and actual progress of each task, automatically notifies the person in charge and the manager if delays occur, and also analyzes the causes of delays, proposes recovery plans, suggests the allocation of additional resources, and proposes schedule readjustments. In risk management, it analyzes the project content and external environment to list potential risk factors, evaluates their impact and probability of occurrence, and proposes specific countermeasures for high-priority risks. For example, if the risk of supply chain delays increases, the AI ​​will suggest alternative suppliers, measures to shorten delivery times, inventory adjustments, and proposed revisions to contract terms.

[0038] The user interface is implemented on the terminal side, allowing employees and administrators to receive AI-generated suggestions, alerts, and automatically generated documents in real time via a web browser or dedicated application. For example, the dashboard allows for centralized viewing of project progress, risk lists, workload distribution, skill maps, AI suggestion history, action items, unresolved task lists, learning progress, meeting minutes summaries, and past response history. Inquiries to the AI ​​and input of additional information can also be made from the terminal at any time. For example, the AI ​​will respond immediately to natural language instructions such as "Summarize this week's project progress," "Analyze A's workload," "Provide a risk list for project B," and "Suggest suitable training for C."

[0039] The use of generative AI is central to this invention, and generative AI plays an indispensable role in a wide range of situations, such as analyzing communication in natural language, summarizing and extracting historical data, automatically generating optimization proposals for business processes, automatically creating documents and plans, simulating risk scenarios, analyzing skill gaps for each employee, automatically formulating learning plans, summarizing meeting minutes and extracting action items, handling inquiries via chatbots, automatically generating project progress reports, automatically visualizing business flows, automatically analyzing KPIs, detecting anomalies, issuing alerts, automatically generating FAQs, automatically organizing internal knowledge, automatically updating business manuals, creating draft revisions for internal regulations, automatically linking with external information, automatically responding to legal changes, automatically summarizing customer interaction history, automatically analyzing survey results, and automatically generating reports from employee satisfaction surveys.

[0040] For example, it can provide advanced business support such as automatically transcribing meeting audio data into text, extracting important decisions and unresolved issues, and notifying relevant parties; cross-analyzing progress data from multiple projects and automatically suggesting optimal resource allocation; matching employee skill maps with project requirements and automatically formulating optimal staffing and training plans; automatically summarizing daily and weekly work reports and generating digest reports for management; automatically updating internal FAQs and knowledge bases; and automating inquiry handling with chatbots.

[0041] Thus, the embodiments of the present invention combine the core functions of the corporate-specific information management unit, communication analysis unit, and business support proposal unit on the server with data collection and user interface functions on the terminal side, and utilize generative AI as the foundation for all analysis, proposal, and automatic generation processes, thereby comprehensively and sustainably achieving improved operational efficiency across the entire organization, risk reduction, employee skill development, increased project success rates, improved work-life balance, enhanced management decision-making, optimized knowledge management, and accelerated innovation creation.

[0042] (System Configuration)

[0043] The system according to this embodiment comprises a Corporate-Specific Information Management Department, a Communication Analysis Department, a Business Support Proposal Department, a User Interface Department, a Data Integration Department, a Security Management Department, and a Log Management Department. The Corporate-Specific Information Management Department has the function of centrally accumulating and managing all information assets within the organization. For example, it stores a vast amount of information, both structured and unstructured, as a database, such as the quotation creation procedures of the sales department, the expense settlement flow of the accounting department, the source code management rules of the development department, the inspection standards of the quality control department, the ordering process of the purchasing department, the recruitment, evaluation, and transfer history of the human resources department, the contract templates and legal compliance checklists of the legal department, the minutes of regular meetings of each department, the progress reports of each project, the qualification acquisition history and goal setting sheets for each employee, past trouble cases and their response history, internal FAQs, various manuals, internal regulations, industry standards and legal revision information, and external benchmark data. It then performs knowledge graphing, semantic tagging, time-series management, importance scoring, and relationship network construction using generating AI. For example, by providing the AI ​​with prompts such as "Extract the success factors of similar past projects needed for a new project," "Summarize the sales department's operational manual and list the key points to note," "Create a draft revision of internal regulations in response to legal changes," or "Compile employee A's past evaluation history and skill map," it is possible to extract and summarize knowledge and automatically organize related information.

[0044] The Communication Analysis Department has the capability to collect data in real time from all communication channels, including email, chat, web conferencing, groupware, social networking services, task management tools, calendars, and file sharing services, and to analyze it sequentially using a natural language processing engine powered by generation AI. For example, it can extract keywords and phrases from meeting minutes and chat exchanges such as "tight deadline," "additional work has arisen," "person in charge is unwell," "approval is delayed," "insufficient information sharing," "a lot of manual work," "increased overtime," "lack of skills," and "increasing risk," and perform multifaceted contextual analysis, linking them to the speaker's position and department, past speaking trends, sentiment analysis, conversation flow, and related business events. For example, by providing the generation AI with prompts such as "extract unresolved issues from this week's meeting minutes and list them by person in charge," "detect signs of progress delays from chat history and generate alerts," "summarize frequently occurring issues and risks in conversations," and "extract signs of resource shortages from Project X communication," it visualizes the overall picture of communication and potential issues.

[0045] The Business Support Proposal Department has the function of comprehensively analyzing information obtained from the Corporate-Specific Information Management Department and the Communication Analysis Department to generate optimal business support and proposals in real time. For example, if it detects a task that involves regularly manually entering large amounts of data, it automatically generates RPA and macro prototypes and presents them to the person in charge. If information sharing is stalled, it proposes the best way to use chat tools and project management tools, and presents specific setting examples, operational rules, templates, automatic creation of notification rules, and workflow automation proposals. If it detects that a project is behind schedule, it automatically checks the schedules of all stakeholders and proposes and sets up an emergency meeting at the optimal date and time. When a new project is launched, it automatically extracts relevant departments and members, and also handles scheduling kick-off meetings, automatically generates necessary documents, creates draft project plans, automatically generates risk lists, and automatically generates role assignment tables. For example, by providing the AI ​​with prompts such as "Create an RPA scenario to automate routine tasks in the accounting department," "Propose a chatbot operation plan to optimize the sharing of deal information in the sales department," "Analyze the causes of delays in the design process and present a recovery plan," or "Automatically generate materials for the kick-off meeting of a new project," the AI ​​can immediately present concrete business improvement and automation proposals.

[0046] The user interface is implemented on the terminal side, allowing employees and administrators to receive AI-generated suggestions, alerts, and automatically generated documents in real time via a web browser or dedicated application. For example, project progress, risk lists, workload distribution, skill maps, AI suggestion history, action items, unresolved task lists, learning progress, meeting minutes summaries, and past response history can all be viewed in one place on the dashboard. Inquiries to the AI ​​and input of additional information can also be made from the terminal side at any time. For example, the AI ​​will respond immediately to natural language instructions such as "Summarize this week's project progress," "Analyze A's workload," "Provide a list of risks for project B," and "Suggest suitable training for C."

[0047] The Data Integration Department has the function of automatically acquiring and integrating necessary data by linking with various internal and external systems and services. For example, it acquires expense data from the accounting system, task progress information from project management tools, employee skill information and evaluation history from the HR system, and benchmark information and legal revision information from external industry databases. This makes it possible for the entire system to always perform analysis and make recommendations based on the latest and most accurate information. For example, by providing prompts to the generating AI such as "Acquire this month's expense data from the accounting system and analyze the risk of budget overruns," "Recommend the optimal project members based on the evaluation history from the HR system," or "Acquire the latest legal revision information from the external database and create a draft revision of the internal regulations," data integration and analysis are automated.

[0048] The Security Management Department has the function of ensuring the information security of the entire system, and handles access rights management, data encryption, unauthorized access detection, login history monitoring, implementation of information leakage prevention measures, and personal information protection measures. For example, it sets detailed access rights to confidential information for each department and job title, constantly monitors access logs, and immediately issues alerts if there is suspicious access. For example, it optimizes security operations by providing prompts to a generating AI such as "Analyze access history to confidential data and detect suspicious operations," "Audit the handling of personal information and extract risks of legal violations," "Automatically generate proposed revisions to access rights," and "Propose operational rules to reduce the risk of information leakage."

[0049] The log management department centrally records and manages the operation history and event logs of the entire system, and uses them for troubleshooting, audit response, and optimization of system operations. For example, it records detailed operation history for each user, execution history of AI-generated automatic suggestions and actions, data linkage history with external systems, and security event occurrence history, and makes them searchable and analyzeable as needed. For example, by giving prompts to the generating AI such as "Extract AI suggestion history for the past month and compare the suggestions that were actually adopted with those that were not," "Recreate the operation history at the time of a system failure in chronological order," "Compile a list of access logs for all users for audit response," and "Analyze the occurrence of data linkage errors with external systems," the utilization of log data is enhanced.

[0050] In this manner, the system according to this embodiment operates with each part working in coordination, providing the generating AI with diverse and sophisticated prompts such as "extract risk factors for new projects from past data," "summarize action items from meeting minutes and notify the person in charge," "analyze skill gaps for each employee and automatically generate an optimal training plan," "detect bottlenecks in business processes and propose improvement plans," and "obtain the latest industry trends from an external database and create a report for management." This enables the system to comprehensively and sustainably achieve overall organizational efficiency improvements, risk reduction, skill development, increased project success rates, optimized knowledge management, enhanced security, and optimized operations.

[0051] (Implementation Steps)

[0052] Step 1: Collection and storage of company-specific information First, the system collects all information assets within the organization and stores them in the company-specific information management department. Specifically, it stores a vast amount of information, both structured and unstructured, as a database, including the sales department's quotation creation procedures, the accounting department's expense reimbursement flow, the development department's source code management rules, the quality control department's inspection standards, the purchasing department's ordering process, the human resources department's recruitment, evaluation, and transfer history, the legal department's contract templates and legal compliance checklists, minutes of regular meetings from each department, progress reports for each project, employee qualification acquisition history and goal setting sheets, past trouble cases and their response history, internal FAQs, various manuals, internal regulations, industry standards and legal revision information, and external benchmark data. This forms the organization's knowledge base.

[0053] Step 2: Real-time collection of communication data The system collects data from all communication channels, including email, chat, web conferencing, groupware, social networking services, task management tools, calendars, and file sharing services, in real time on the terminal side and sends it to the server. For example, it retrieves all messages using the API of a chat tool, automatically uploads recordings and minutes of meeting systems, and periodically synchronizes sent and received history from the mail server.

[0054] Step 3: Communication analysis using generative AI The server-side generating AI analyzes collected communication data using a natural language processing engine, performing multifaceted contextual analysis including the flow of conversation, the roles of speakers, sentiment analysis, past speaking trends, and linking to related business events. For example, it extracts keywords and phrases from meeting minutes and chat exchanges such as "tight deadline," "additional work has arisen," "person in charge is unwell," "approval is delayed," "insufficient information sharing," "a lot of manual work," "overtime is increasing," "lack of skills," and "risk is rising." Examples of prompts to be fed into the generating AI include "Extract unresolved issues from this week's meeting minutes and list them by person in charge," "Detect signs of progress delays from chat history and generate alerts," "Summarize frequently occurring issues and risks in conversations," and "Extract signs of resource shortages from Project X communication."

[0055] Step 4: Integrated Analysis of Corporate Knowledge and Communication The generating AI comprehensively analyzes knowledge accumulated in the company's proprietary information management department and real-time data obtained from the communication analysis department. For example, it refers to issues and success factors from past projects, operational manuals, FAQs, relevant internal regulations, and external benchmark data, and compares them with the status of currently ongoing projects and operations. This makes it possible to identify issues and areas for improvement that are relevant to the organization's current situation. Examples of prompts that can be fed into the generating AI include: "Extract the success factors of similar past projects needed for the new project," "Summarize the sales department's operational manual and list the key points to note," "Create a draft revision of internal regulations in response to legal changes," and "Summarize employee A's past evaluation history and skill map."

[0056] Step 5: Automatic generation of business support and improvement suggestions The Business Support Proposal Department uses a generation AI to automatically generate optimal business support and improvement proposals in real time based on integrated analysis results. For example, if it detects a task that involves regularly manually entering large amounts of data, it automatically generates RPA or macro prototypes and presents them to the person in charge. If information sharing is stalled, it proposes the best way to use chat tools and project management tools, and presents specific setting examples, operational rules, templates, automatic creation of notification rules, and workflow automation proposals. Examples of prompts that can be fed into the generation AI include: "Create an RPA scenario to automate routine tasks in the accounting department," "Propose a chatbot operation plan to optimize the sharing of case information in the sales department," "Analyze the causes of delays in the design process and present a recovery plan," and "Automatically generate materials for the kickoff meeting of a new project."

[0057] Step 6: Project Management, Risk Analysis, and Progress Monitoring The generating AI analyzes the success factors and failures of similar past projects during the planning phase of project management and automatically generates project plan templates. During the execution phase, it aggregates expenditure data in real time, immediately issues alerts if there are signs of budget overruns, and proposes cost reduction and alternative solutions. In task progress management, it constantly monitors the planned completion date and actual progress of each task, automatically notifies the person in charge and the manager if delays occur, and also analyzes the cause of the delay, proposes recovery plans, suggests the allocation of additional resources, and proposes schedule readjustments. In risk management, it analyzes the project content and external environment to list potential risk factors, evaluates the impact and probability of occurrence, and proposes specific countermeasures for high-priority risks. Examples of prompts to be fed into the generating AI include: "Summarize the progress of Project X and list delayed tasks and their causes," "Analyze this month's expenditure data and extract budget overrun risks," "Automatically generate a project plan template," and "Create a risk list and propose countermeasures in order of priority."

[0058] Step 7: Suggestions, alerts, and dashboard displays for the user. The user interface is implemented on the terminal side, allowing employees and administrators to receive AI-generated suggestions, alerts, and automatically generated documents in real time via a web browser or dedicated application. For example, project progress, risk lists, workload distribution, skill maps, AI suggestion history, action items, unresolved task lists, learning progress, meeting minutes summaries, and past response history can all be viewed in one place on the dashboard. Inquiries to the AI ​​and input of additional information can also be made from the terminal side at any time. For example, the AI ​​will respond immediately to natural language instructions such as "Summarize this week's project progress," "Analyze A's workload," "Provide a list of risks for project B," and "Suggest suitable training for C."

[0059] Step 8: System Operation, Security, and Log Management The Security Management Department ensures the information security of the entire system, handling access rights management, data encryption, unauthorized access detection, login history monitoring, implementation of information leakage prevention measures, and personal information protection. The Log Management Department centrally records and manages the operation history and event logs of the entire system, and uses them for troubleshooting, audit response, and optimization of system operations. Examples of prompts to be fed into the generating AI include: "Analyze access history to confidential data and detect suspicious operations," "Audit the handling of personal information and extract risks of legal violations," "Extract AI proposal history for the past month and compare proposals that were actually adopted with those that were not," and "Recreate the operation history in chronological order when a system failure occurred."

[0060] (Specific use cases)

[0061] For example, when a new product development project is being promoted in the design department of a manufacturing company, the project leader activates the system and inputs the project objectives, participating members, planned period, required technical elements, budget, and information on similar past projects. The Corporate-Specific Information Management Department automatically extracts progress reports, design procedures, quality control standards, bills of materials, external benchmark data, relevant legal and regulatory information, internal design know-how collections, past trouble cases, each member's skill set and qualifications, and past evaluation history of past new product development projects, and presents them to the project leader. The Communication Analysis Department collects all chats, emails, meeting minutes, comments from task management tools, and update history of file sharing services in real time from the start of the project, and uses natural language processing by generating AI to sequentially detect signs such as progress delays, design changes, quality problems, parts procurement risks, information sharing stagnation, increased workload on personnel, and skill deficiencies. The Business Support Proposal Department analyzes this information comprehensively and, for example, if manual work is frequent in the design process, proposes automated generation using RPA or macros; if quality problems are frequent, proposes improvement measures referencing past quality control standards and external benchmarks; if parts procurement risks increase, automatically generates alternative parts candidates and supplier lists; and if progress delays occur, automatically adjusts the schedules of stakeholders to set up emergency meetings and automatically generates necessary agendas and materials. Furthermore, it matches each member's skill set with project requirements, and if a skill gap is identified, automatically recommends relevant in-house training, external online courses, and OJT plans, and adjusts the training schedule. In project management, it analyzes the success factors and failure cases of similar past projects, automatically generates project plans, risk lists, Gantt charts, role assignment charts, and progress management sheets; during the execution phase, it aggregates expenditure data in real time, immediately issues alerts if there is a risk of budget overrun, and proposes cost reduction and process shortening plans. Task progress management constantly monitors the planned completion date and actual progress of each task. If delays occur, the system automatically notifies the person in charge and the manager, and also analyzes the cause of the delay, proposes recovery plans, suggests allocating additional resources, and proposes schedule adjustments.In risk management, project details and the external environment are analyzed to list potential risk factors, their impact and probability of occurrence are evaluated, and specific countermeasures are proposed for high-priority risks. In the user interface section, project leaders and members can centrally view progress, risk lists, workload distribution, skill maps, AI suggestion history, action items, list of unaddressed tasks, learning progress, meeting minutes summaries, and past response history on a dashboard, and inquiries to the AI ​​and input of additional information can be made at any time using natural language.

[0062] Specific examples of prompts to be fed into the generating AI include: "Extract the main delay factors and countermeasures from the past five progress reports of new product development projects and reflect them in the current project plan," "Summarize the design department's work manual and list the points that require particular attention in this project," "Extract unresolved issues and action items for each person in charge from meeting minutes and chat history and display them on the dashboard," "Automatically generate a list of alternative parts candidates and suppliers in case the parts procurement risk increases," "Match each member's skill set with project requirements and propose the optimal training plan if there is a skills gap," "Analyze this month's expenditure data, extract the risk of budget overruns, and propose cost reduction proposals," "Summarize the progress of Project X and list the delayed tasks and their causes," and "Create a risk list and propose countermeasures in order of priority."

[0063] Through these use cases, the system of the present invention can comprehensively and sustainably realize business support, project management, risk analysis, skill development, information sharing, and automation proposals using generated AI, thereby contributing to improved productivity, risk reduction, quality improvement, employee skill development, work-life balance optimization, enhanced management decision-making, optimized knowledge management, and accelerated innovation creation across the entire organization.

[0064] (Application Example 1) The flow of a specific process in Application Example 1 will be explained below. Each part of the system described below is implemented by a data processing device 12 and a smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] (Embodiments for carrying out the present invention)

[0066] An embodiment of the present invention involves introducing a virtual AI colleague system centered on generative AI in a robot operation site within a factory. This system provides an extremely specific and comprehensive configuration that centrally and autonomously supports robot operation monitoring, anomaly detection, failure prediction, maintenance planning, quality control, on-site worker support, knowledge sharing, education and training, safety management, and on-site improvement suggestions.

[0067] First, the robot operation information management unit aggregates a variety of IoT data acquired in real time from various robots in the factory, such as operating status (operating, stopped, error occurring), temperature, vibration, power consumption, cycle time, work content, operation history, alarm history, error codes, parts replacement history, lubricant usage, sensor values, robot arm movement patterns, load on each axis, and communication status, onto a server. For example, it comprehensively manages detailed operation data that differs depending on the type and application of the robot, such as arc current values ​​and welding quality judgment data for welding robots, travel distance and battery level for transport robots, pick-and-place counts and gripper wear degree for assembly robots, and coating amount and nozzle clogging detection data for painting robots. Furthermore, it records detailed maintenance history, such as "Robot A's speed reducer replaced on May 10, 2024," "Robot B's sensor cleaned on April 15, 2024," and "Robot C's control board replaced on March 20, 2024," including the work date, work content, replaced parts, worker, work time, post-work confirmation results, and recurrence prevention measures. Work instructions include digitized procedures and work flows actually used on-site, such as "Setup changeover procedure for robot D," "Periodic inspection procedure for robot E," "Anomaly response flow for robot F," and "Production item changeover procedure for robot G," with version control implemented. Quality control records include chronologically accumulated quality data for each process and robot, such as "Dimensional measurement results of parts produced by robot H," "Defect rate in the inspection process of robot I," "Visual inspection images of robot J," and "Traceability data for robot K." Equipment manuals include centralized management of manufacturer-provided documents and on-site-specific supplementary materials, such as "Operation manual for robot L," "Maintenance manual for robot M," "Safety precautions for robot N," and "Troubleshooting manual for robot O." Past incidents include detailed records of events such as "Details of an arm stoppage incident that occurred with robot P," "A case of line stoppage due to a sensor malfunction of robot Q," "A malfunction due to a control software defect in robot R," and "A gripper detachment incident of robot S." These records include the date and time of occurrence, circumstances, cause, countermeasures, measures to prevent recurrence, details of emergency measures taken on-site, related photos and videos, and comments from those involved.The system manages detailed skill maps and training histories for each individual worker, including, for example, "Worker A's robot operation qualification acquisition date," "Worker B's OJT attendance history," "Worker C's e-learning attendance status," "Worker D's evaluation sheet," and "Worker E's on-site improvement proposal history." It also accumulates information on on-site troubles and improvement cases, such as "on-site improvement proposals," "near-miss reports," "on-site improvement activity records," "examples of unique on-site innovations," and "points raised during on-site inspections," all within the knowledge base.

[0068] The Communication Analysis Department collects all kinds of communication data in real time, including daily reports, chats, trouble reports, meeting minutes, handover notes, on-site bulletin board posts, feedback on work instructions, voice memos from on-site inspections, speech recognition data of verbal instructions on-site, question-and-answer history between workers, and on-site survey results. Using natural language processing with generative AI, it comprehensively analyzes the raw voices and challenges of the field, operational errors, unclear procedures, workload, training needs, atmosphere and emotions on-site, discrepancies in information transmission among workers, trends in on-site improvement suggestions, changes in safety awareness on-site, signs of worker stress, and the transmission status of instructions from on-site leaders. For example, we identify on-site issues and signs such as "Robot A's operation is unstable," "Robot B's gripper frequently stops," "Many operational errors occur during the night shift," "The procedure for periodic inspections is unclear," "New workers make mistakes with the procedure," "Insufficient information sharing on-site," "Workload is unevenly distributed among certain workers," "Training is not keeping up," "Instructions from the site leader are not being communicated to the entire site," "There are differences in procedure interpretation among workers," and "Safety awareness on-site is declining," thereby gaining a multifaceted understanding of the situation on-site.

[0069] Based on information obtained from the Robot Operation Information Management Department and the Communication Analysis Department, the Business Support Proposal Department utilizes generative AI to automatically generate and present to the site, in real time, specific countermeasures, work instructions, training plans, improvement measures, troubleshooting procedures, optimization proposals, quality improvement measures, safety measures, and site improvement proposals for robot operation monitoring, anomaly detection, failure prediction, maintenance planning, quality control, work procedure optimization, training support for on-site workers, knowledge sharing, overall productivity improvement, safety management, and on-site improvement proposals. For example, by analyzing robot operation data, if the temperature or vibration is higher than normal, it automatically generates specific countermeasures such as "Robot A's motor temperature is trending upward. We recommend checking the cooling fan or adjusting the operating interval." When an error occurs, it refers to past trouble cases and manuals and immediately presents troubleshooting procedures to the site, such as "Robot B's error code E102 has occurred 5 times in the past. Last time it was resolved by cleaning the sensor. We recommend the same procedure this time." If the cycle time exceeds the standard value, the AI ​​will suggest improvements such as, "Robot C's cycle time is 20% longer than the standard. We propose reviewing the work procedure and optimizing parameters. Refer to past optimization examples." For example, if the arc current value of a welding robot shows an abnormal value, the AI ​​will refer to similar past cases and suggest electrode replacement or resetting the welding conditions. If the battery level of a transport robot is decreasing, the AI ​​will automatically generate the optimal charging timing and battery replacement plan. If the number of pick-and-place operations of an assembly robot exceeds the standard value, the AI ​​will estimate the degree of gripper wear and suggest the timing of preventive maintenance. If nozzle clogging occurs frequently in a painting robot, the AI ​​will suggest reviewing the nozzle cleaning procedure and coating conditions.

[0070] In maintenance planning, the generative AI creates maintenance work instructions based on maintenance history and operational data, such as "Robot D requires scheduled maintenance this weekend. Automatically generate a list of necessary parts and work procedures, and notify the person in charge." For example, based on the robot's operating time and past failure trends, the AI ​​automatically calculates the optimal timing for preventive maintenance and notifies the site of the necessary parts, tools, and work procedures. In quality control, for robots with high defect rates, the AI ​​proposes quality improvement measures such as "Robot E's defect rate exceeds 5%. Refer to past examples of countermeasures taken when defects occurred and propose adding inspection processes or replacing parts." For example, by comparing inspection records and production history, if defects are concentrated in a particular process or part, the AI ​​automatically proposes a cause analysis and improvement measures (e.g., changing parts manufacturers, adding inspection processes, reviewing work procedures).

[0071] In supporting on-site workers, if "operational errors" or "uncertainty about procedures" are detected from communication logs, the system provides training support such as "automatically suggesting relevant operation manuals, video materials, and e-learning courses to workers who frequently make operational errors, and creating an OJT plan as needed." For example, if a new worker repeatedly makes mistakes in a specific process, the AI ​​will refer to the worker's past training history and skill map, automatically create an optimal training plan (e.g., on-site OJT, e-learning, practical training, paired work with an experienced worker), and propose it to the site manager. Furthermore, new troubles and improvement examples that occur on-site are automatically registered in the knowledge base and shared with other workers, thereby raising the overall knowledge level of the workplace. For example, the AI ​​will automatically summarize efficient setup change methods devised by a worker or emergency response methods when troubles occur, register them in the knowledge base, and notify all workers.

[0072] Furthermore, to improve productivity on site, the AI ​​analyzes operational data from multiple robots across systems and automatically proposes optimal resource allocation solutions and bottleneck processes. For example, it compares the operational status of multiple lines and automatically generates proposals to site managers, such as reallocating processes of underutilized robots to other robots, optimizing setup timing, and reviewing night shift staffing. In addition, the AI ​​contributes to on-site safety management. For instance, it analyzes past near-miss reports and accident history to alert the site of potential dangers and the need for safety training. For example, the AI ​​analyzes past incidents such as robot arm pinching accidents, transport robot collision accidents, and painting robot organic solvent leak accidents, and immediately issues warnings to the site if similar risks increase.

[0073] Furthermore, regarding on-site improvement proposals, the AI ​​automatically summarizes and evaluates improvement proposals and on-site improvement activity records submitted by on-site workers, simulates their feasibility and effectiveness, and recommends high-priority proposals to the on-site manager. For example, the AI ​​automatically evaluates and proposes a variety of on-site improvement proposals, such as "proposals for improving jigs to shorten setup time," "proposals for automating inspection processes," "proposals for improving work efficiency by reviewing parts supply methods," and "proposals for optimizing worker movement."

[0074] Thus, in this embodiment of the present invention, the robot operation information management unit, the communication analysis unit, and the business support proposal unit work together, with a generative AI at its core, to comprehensively and autonomously support all factory operations, including robot operation monitoring, anomaly detection, failure prediction, maintenance planning, quality control, work procedure optimization, on-site worker support, knowledge sharing, education and training, safety management, and on-site improvement proposals. This enables overall productivity improvement, quality stabilization, downtime reduction, worker skill development, organizational sharing of know-how, safety improvement, and advanced on-site improvement activities.

[0075] (System Configuration)

[0076] The system according to this embodiment comprises a robot operation information management unit, a communication analysis unit, a business support proposal unit, a knowledge base management unit, an education and training support unit, a safety management unit, a user interface unit, a data linkage unit, and a log management unit. The robot operation information management unit aggregates various IoT data acquired in real time from various robots in the factory, such as operating status, temperature, vibration, power consumption, cycle time, work content, operation history, alarm history, error codes, parts replacement history, lubrication oil usage, sensor values, robot arm movement patterns, load on each axis, and communication status, into a server. It also comprehensively manages detailed operation data that differs depending on the type and application of the robot, such as arc current values ​​and welding quality judgment data for welding robots, travel distance and battery level for transport robots, pick-and-place counts and gripper wear degree for assembly robots, and coating amount and nozzle clogging detection data for painting robots. Furthermore, it records detailed past maintenance history, including work date, work content, replaced parts, worker, work time, post-work confirmation results, and recurrence prevention measures, and sets up each robot as a work instruction sheet. Work procedures, periodic inspection procedures, abnormality response flows, and production item changeover procedures are digitized, quality control records such as part dimension measurement results, defect rates in the inspection process, visual inspection images, and traceability data are accumulated chronologically, equipment manuals such as operation manuals, maintenance manuals, safety precautions, and troubleshooting manuals are centrally managed, past trouble cases are recorded in detail including the date and time of occurrence, circumstances, cause, countermeasures, recurrence prevention measures, emergency measures, related photos and videos, and comments from those involved, on-site workers' skills and training history are managed in detail for each individual, including qualification acquisition dates, OJT attendance history, e-learning attendance status, evaluation sheets, and on-site improvement proposal history, and on-site trouble and improvement cases that occurred on-site are all accumulated as a knowledge base, including on-site improvement proposals, near-miss reports, on-site improvement activity records, unique improvement examples, and points raised during on-site inspections.

[0077] The Communication Analysis Department collects all communication data in real time, including daily reports, chats, trouble reports, meeting minutes, handover notes, on-site bulletin board posts, feedback on work instructions, voice memos from on-site inspections, speech recognition data of verbal instructions, question and answer history, and survey results. Using natural language processing powered by AI, it analyzes the raw voices and challenges of the field, operational errors, unclear procedures, workload, training needs, the atmosphere and emotions of the workplace, discrepancies in information transmission among workers, trends in on-site improvement suggestions, changes in safety awareness, and signs of worker stress. In addition, the system analyzes the communication status of instructions from the site leader from multiple perspectives to identify on-site issues and signs, such as "unstable operation of robot A," "frequent stopping of robot B's gripper," "many operational errors during night shifts," "difficulty understanding the procedures for periodic inspections," "new workers making procedural errors," "insufficient information sharing on-site," "workload being unevenly distributed among certain workers," "training not keeping pace," "instructions from the site leader not being communicated to the entire site," "differences in procedure interpretation among workers," and "decreased safety awareness on-site," thereby gaining a comprehensive understanding of the on-site situation.

[0078] Based on information obtained from the Robot Operation Information Management Department and the Communication Analysis Department, the Business Support Proposal Department utilizes AI to automatically generate, in real time, comprehensive solutions for robot operation monitoring, anomaly detection, failure prediction, maintenance planning, quality control, work procedure optimization, training support for on-site workers, knowledge sharing, overall productivity improvement, safety management, and on-site improvement proposals. These solutions include specific countermeasures, work instructions, training plans, improvement measures, troubleshooting procedures, optimization proposals, quality improvement measures, safety measures, and on-site improvement proposals. These solutions are then presented to the site. For example, it analyzes robot operation data and automatically generates specific countermeasures such as "Robot A's motor temperature is on the rise. We recommend checking the cooling fan or adjusting the operating interval" if the temperature or vibration is higher than normal. When an error occurs, it refers to past trouble cases and manuals and immediately presents troubleshooting procedures to the site, such as "Robot B's error code E102 has occurred 5 times in the past. Last time it was resolved by cleaning the sensor. We recommend the same procedure this time as well." If the cycle time exceeds the standard value, it presents improvement suggestions such as "Robot C's cycle time is 20% longer than the standard. We suggest reviewing the work procedure and optimizing the parameters. Refer to past optimization cases." If the arc current value of a welding robot shows an abnormal value, it suggests electrode replacement or resetting the welding conditions. If the battery level of a transport robot is on the decline, it automatically generates the optimal charging timing and battery replacement plan. If the number of pick-and-place operations of an assembly robot exceeds the standard value, it estimates the degree of gripper wear and suggests the timing of preventive maintenance. If nozzle clogging occurs frequently in a painting robot, it suggests reviewing the nozzle cleaning procedure and application conditions.

[0079] The Knowledge Base Management Department automatically summarizes, classifies, and tags on-site troubles and improvement cases, on-site improvement proposals, near-miss reports, on-site improvement activity records, unique innovation examples, points raised during on-site inspections, on-site emergency response methods, on-site success stories, on-site failure stories, and on-site best practices, and stores them as a knowledge base so that on-site workers and managers can easily search and refer to them. For example, by providing prompts to the generating AI such as "Summarize robot trouble cases from the past year and classify them by cause," "List implemented and unimplemented proposals from on-site improvement proposals," "Extract examples from near-miss reports that should be used for safety training," and "Reflect unique innovation examples from on-site in work procedure manuals," the system enables the automatic organization and utilization of knowledge.

[0080] The Education and Training Support Department uses AI to automatically generate optimal training plans, on-the-job training (OJT) plans, e-learning courses, practical training programs, video materials, manuals, and evaluation tests based on the skills, training history, operational errors, unclear procedures, and training needs of field workers, and proposes them to field workers and managers. For example, by providing the AI ​​with prompts such as "Automatically suggest relevant operation manuals, video materials, and e-learning courses for workers who frequently make operational errors," "Create an OJT plan for new workers," "Automatically generate a practical training program that corresponds to new work procedures on site," and "Create a skill map for field workers and analyze skill gaps," the department aims to optimize education and training.

[0081] The Safety Management Department uses AI to alert the worksite about signs of hazardous work and the need for safety training, based on past near-miss reports, accident history, on-site safety training history, on-site work environment data, stress indicators among on-site workers, and the occurrence of hazardous work at the worksite. It also automatically generates safety countermeasures, safety training programs, feedback on issues found during on-site inspections, and proposed revisions to the safety manual. For example, by providing the AI ​​with prompts such as "Issue an immediate warning to the worksite if a similar risk increases based on past accident history," "Extract examples from near-miss reports that should be used for safety training," "Analyze stress indicators among on-site workers and propose mental health measures," and "Analyze the occurrence of hazardous work at the worksite and automatically generate safety countermeasures," the department can achieve advanced safety management.

[0082] The user interface allows field workers and managers to receive AI-generated suggestions and alerts, automatically generated documents, knowledge bases, training plans, and safety management information in real time via a web browser or dedicated application. The dashboard allows for centralized viewing of robot operating status, anomaly alerts, maintenance plans, quality control status, worker skill maps, training history, safety management status, and field improvement suggestion history. Inquiries to the AI ​​and input of additional information can also be made at any time using natural language.

[0083] The data integration unit has the function of automatically acquiring and integrating necessary data by linking with various systems and services inside and outside the factory. For example, it acquires production plans and actual data from the production management system, inspection data from the quality management system, maintenance history from the maintenance management system, worker skill information and training history from the human resources system, and benchmark information and legal revision information from external industry databases. For example, by providing prompts to the generating AI such as "Acquire this month's production performance data from the production management system and analyze the robot utilization rate," "Extract trends in quality defects based on inspection data from the quality management system," "Automatically generate a preventive maintenance plan based on the maintenance history from the maintenance management system," and "Acquire the latest legal revision information from an external database and create a draft revision of the safety manual," data integration and analysis are automated.

[0084] The log management department centrally records and manages the operation history and event logs of the entire system, utilizing them for troubleshooting, audit response, and system operation optimization. For example, it meticulously records operation history for each user, execution history of AI-generated automatic suggestions and actions, data linkage history with external systems, and security event occurrence history, making it searchable and analyzeable as needed. For instance, by providing prompts to the generating AI such as "Extract AI suggestion history for the past month and compare adopted and unadopted suggestions," "Recreate the operation history at the time of a system failure in chronological order," "Compile a list of access logs for all users for audit response," and "Analyze the occurrence of data linkage errors with external systems," the utilization of log data is enhanced.

[0085] As described above, the system according to this embodiment works in cooperation with each other, and provides the generating AI with a variety of sophisticated prompts, such as "Analyze the operation data and maintenance history of robot A for the past month and propose countermeasures if there are signs of failure," "Summarize past occurrences and recovery procedures for robot B's error code E102 and explain them clearly to the on-site personnel," "Analyze the cause of robot C's cycle time exceeding the standard value and propose an optimization plan," "Automatically generate a periodic maintenance work instruction sheet and a list of necessary parts for robot D," "Propose quality improvement measures for robot E's increasing defect rate," "Create an optimal training plan for workers who frequently make operational errors," "Summarize this week's on-site trouble cases and register them in the knowledge base," "Issue an immediate warning to the site if a similar risk increases based on past accident history," and "Extract high-priority proposals from on-site improvement proposals and recommend them to the on-site manager." This enables the system to comprehensively and sustainably improve productivity, stabilize quality, reduce downtime, enhance worker skills, share know-how within the organization, improve safety, and advance on-site improvement activities in the factory.

[0086] (Implementation Steps)

[0087] Step 1: Collection and integration of robot operation data From various robots within the factory, diverse IoT data such as operating status, temperature, vibration, power consumption, cycle time, work content, operation history, alarm history, error codes, parts replacement history, lubrication usage, sensor values, robot arm movement patterns, load on each axis, and communication status are collected in real time on a server. Furthermore, detailed operational data that differs depending on the type and application of the robot is also managed in an integrated manner, including arc current values ​​and welding quality judgment data for welding robots, travel distance and battery level for transport robots, pick-and-place counts and gripper wear degree for assembly robots, and coating amount and nozzle clogging detection data for painting robots.

[0088] Step 2: Collection of field communication data All communication data, including daily reports, chats, trouble reports, meeting minutes, handover notes, on-site bulletin board posts, feedback on work instructions, voice memos from on-site inspections, speech recognition data of verbal instructions, question and answer history, and survey results, is collected in real time and stored on the server.

[0089] Step 3: On-site situation analysis using generative AI The generating AI comprehensively analyzes robot operation information and communication data to extract from multiple perspectives a wealth of information, including firsthand accounts and challenges from the field, operational errors, unclear procedures, workload, training needs, the atmosphere and emotions on site, discrepancies in information transmission among workers, trends in on-site improvement suggestions, changes in safety awareness, signs of worker stress, and the transmission status of instructions from on-site leaders. Specific examples of prompts to be fed into the generating AI include: "Extract operational errors and unclear procedures from on-site chats and daily reports over the past week and classify them by cause," "Summarize trends in on-site improvement suggestions from on-site bulletin board posts," "Extract on-site leader instructions and transmission status from voice memos taken during on-site inspections," and "Analyze signs of stress among on-site workers from survey results."

[0090] Step 4: Automatic generation of anomaly detection, fault prediction, and maintenance plans using AI. The generating AI analyzes abnormal values ​​such as temperature and vibration, cycle time exceeding standards, frequent error codes, and the progress of parts wear based on robot operation data and maintenance history to detect anomalies and predict failures, and automatically generates work instructions, necessary parts lists, and work procedures for periodic and preventive maintenance as needed. Specific examples of prompts to be fed into the generating AI include: "Analyze the operation data and maintenance history of robot A for the past month, and propose countermeasures if there are signs of failure," "Summarize past occurrences and recovery procedures for error code E102 of robot B, and explain them clearly to the on-site personnel," "Analyze the cause of robot C's cycle time exceeding the standard value and propose an optimization plan," and "Automatically generate a periodic maintenance work instruction and necessary parts list for robot D."

[0091] Step 5: Quality control and improvement suggestions using generated AI The generating AI analyzes quality control records, inspection data, defect rates, visual inspection images, and traceability data. If the defect rate is high, it automatically generates a cause analysis and quality improvement measures, which it then proposes to the factory floor. Specific examples of prompts that can be fed into the generating AI include: "Propose quality improvement measures to address the increased defect rate of robot E," "Compare inspection data with production history and analyze the cause of the defect," "Extract the characteristics of defective products from visual inspection images and propose improvements to the inspection process," and "Based on traceability data, propose measures to prevent the recurrence of quality problems."

[0092] Step 6: Automatic generation of field worker support and training plans using AI. The generating AI automatically generates optimal training plans, on-the-job training (OJT) plans, e-learning courses, practical training programs, video materials, manuals, and evaluation tests based on the skills and training history of field workers, operational errors, unclear procedures, and training needs, and proposes them to field workers and managers. Specific examples of prompts that can be fed into the generating AI include: "Create an optimal training plan for workers who frequently make operational errors," "Automatically generate an OJT plan for new workers," "Create a practical training program that corresponds to new work procedures on-site," and "Create a worker skill map and analyze skill gaps."

[0093] Step 7: Promoting knowledge sharing and on-site improvement proposals through generational AI. The generating AI automatically summarizes, classifies, and tags on-site troubles and improvement cases, on-site improvement proposals, near-miss reports, on-site improvement activity records, unique innovation examples, points raised during on-site inspections, on-site emergency response methods, on-site success stories, on-site failure stories, and on-site best practices, storing them as a knowledge base so that on-site workers and managers can easily search and refer to them. Furthermore, it automatically evaluates and prioritizes on-site improvement proposals and simulates their feasibility. Specific examples of prompts to be fed into the generating AI include: "Summarize robot trouble cases from the past year and classify them by cause," "List implemented and unimplemented proposals from on-site improvement proposals," "Extract examples from near-miss reports that should be used for safety training," "Reflect unique innovation examples from the field in work procedures," and "Extract high-priority proposals from on-site improvement proposals and recommend them to the site manager."

[0094] Step 8: Safety management and risk alerts generated by AI The generating AI uses past near-miss reports and accident history, on-site safety training history, on-site work environment data, signs of stress among on-site workers, and the occurrence of hazardous work at the site to alert the site of signs of hazardous work and the need for safety training. It also automatically generates safety countermeasures, safety training programs, feedback on points raised during site inspections, and proposed revisions to the safety manual. Specific examples of prompts that can be fed into the generating AI include: "Issue an immediate warning to the site if a similar risk increases based on past accident history," "Extract examples from near-miss reports that should be used for safety training," "Analyze signs of stress among on-site workers and propose mental health measures," and "Analyze the occurrence of hazardous work at the site and automatically generate safety countermeasures."

[0095] Step 9: Providing information and feedback through the user interface The user interface allows field workers and managers to receive AI-generated suggestions and alerts, automatically generated documents, knowledge bases, training plans, and safety management information in real time via a web browser or dedicated application. They can also centrally view robot operating status, anomaly alerts, maintenance plans, quality control status, worker skill maps, training history, safety management status, and field improvement suggestion history on a dashboard, and can make inquiries to the AI ​​and input additional information at any time using natural language.

[0096] (Specific use cases)

[0097] For example, when applying the system of the present invention to an assembly line in an automobile parts factory where many articulated robots and transport robots are in operation, IoT data such as operating status, temperature, vibration, power consumption, cycle time, error code, parts replacement history, lubrication usage, gripper wear degree, transport distance, and battery level are collected in real time from each robot. In addition to this data, the robot operation information management unit integrates and manages past maintenance history, work instructions, inspection records, visual inspection images, traceability data, on-site worker qualifications and training history, on-site improvement proposals, and near-miss reports as long-term memory. The communication analysis unit collects daily reports, chats, trouble reports, meeting minutes, handover memos, on-site bulletin board posts, voice memos, etc. entered by workers in real time, and uses natural language processing by generating AI to extract on-site issues such as "Robot A's operation is unstable," "Many operation errors occur during the night shift," "The procedure for periodic inspection is difficult to understand," "New workers made mistakes in the procedure," "Insufficient information sharing on-site," and "Workload is skewed towards specific workers." The Business Support Proposal Department, for example, if the temperature or vibration of robot B exceeds the standard value, will use its AI to automatically generate countermeasures such as "Robot B's motor temperature is on the rise. We recommend checking the cooling fan or adjusting the operating interval," and present them to the site. If error codes occur frequently, the AI ​​will automatically generate troubleshooting procedures in response to a prompt such as "Summarize past occurrences and recovery procedures for robot C's error code E102, and explain them clearly to the site personnel." If the cycle time exceeds the standard value, the AI ​​will present improvement suggestions such as "Robot D's cycle time is 20% longer than the standard. We propose reviewing the work procedure and optimizing parameters. Refer to past optimization examples." In the maintenance plan, the AI ​​will create optimal maintenance work instructions based on maintenance history and operational data in response to a prompt such as "Automatically generate a periodic maintenance work instruction sheet and a list of necessary parts for robot E." In quality control, prompts such as "Propose quality improvement measures for the increased defect rate of robot F" and "Compare inspection data with production history and analyze the cause of the defect" automatically generate cause analysis and improvement measures.In field worker support, prompts such as "Create an optimal training plan for workers who frequently make operational errors" and "Automatically generate an OJT plan for new workers" allow the generating AI to automatically create training plans based on the workers' training history and skill maps. In knowledge sharing, prompts such as "Summarize robot trouble cases from the past year and classify them by cause" and "List implemented and unimplemented proposals from field improvement proposals" allow the generating AI to automatically organize the knowledge base and share it with the entire site. In safety management, prompts such as "Issue an immediate warning to the site if a similar risk increases based on past accident history" and "Extract examples from near-miss reports that should be used for safety education" allow the generating AI to automatically generate risk alerts and safety education proposals. Thus, the system of the present invention enables the generation AI to centrally and autonomously support robot operation monitoring, anomaly detection, failure prediction, maintenance planning, quality control, work procedure optimization, on-site worker support, knowledge sharing, education and training, safety management, and on-site improvement suggestions, thereby achieving overall productivity improvement, quality stabilization, downtime reduction, worker skill development, organizational sharing of know-how, improved safety, and advanced on-site improvement activities.

[0098] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0099] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0100] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0101] For example, the data collection unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the smart device 14 and processes it with the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using generation AI. For example, the serving unit is implemented by the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the devices and control units is not limited to the examples described above and can be modified in various ways.

[0102] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0103] [Second Embodiment]

[0104] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0105] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0106] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0107] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and communication interface 44 are also connected to the bus 52.

[0108] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0109] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0110] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0111] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0112] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0113] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0114] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the identification processing unit 290 using these models.

[0115] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is realized by the data processing device 12 and the smart glasses 214. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0116] (Example 1)

[0117] Since the process flow is the same as in Example 1 described in the First Embodiment above, the explanation will be omitted.

[0118] (Application Example 1) Since the process flow is the same as in Application Example 1 described in the first embodiment above, the explanation will be omitted.

[0119] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0120] Furthermore, the processing by the data processing system 10 described above is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0121] For example, the data collection unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the smart glasses 214 and processes it using the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using generation AI. For example, the serving unit is implemented by the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0122] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0123] [Third Embodiment]

[0124] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0125] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0126] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0127] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, display 343, and communication interface 44 are also connected to the bus 52.

[0128] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0129] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0130] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0131] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0132] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0133] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0134] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0135] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the headset terminal 314. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0136] (Example 1) Since the process flow is the same as in Example 1 described in the First Embodiment above, the explanation will be omitted.

[0137] (Application Example 1) Since the process flow is the same as in Application Example 1 described in the first embodiment above, the explanation will be omitted.

[0138] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0139] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0140] For example, the data collection unit is implemented by the control unit 46A of the headset terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the headset terminal 314 and processes it using the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using generation AI. For example, the serving unit is implemented by the speaker 240 and display 343 of the headset terminal 314 or the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0141] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0142] [Fourth Embodiment]

[0143] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0144] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0145] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0146] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, controlled object 443, and communication interface 44 are also connected to the bus 52.

[0147] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0148] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0149] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0150] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0151] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0152] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0153] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0154] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0155] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the robot 414. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0156] (Example 1) Since the process flow is the same as in Example 1 described in the First Embodiment above, the explanation will be omitted.

[0157] (Application Example 1) Since the process flow is the same as in Application Example 1 described in the first embodiment above, the explanation will be omitted.

[0158] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0159] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or external devices, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or external devices.

[0160] For example, the data collection unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the robot 414 and processes it using the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using a generation AI. For example, the serving unit is implemented by the speaker 240 and controlled object 443 of the robot 414 and the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0161] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0162] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0163] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0164] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0165] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0166] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0167] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0168] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0169] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0170] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0171] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0172] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0173] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0174] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0175] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0176] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0177] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0178] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0179] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0180] Furthermore, the following additional information is disclosed regarding the above explanation.

[0181] (Claim 1)

[0182] This system operates using generative AI in a factory robot operation site and comprises a robot operation information management unit, a communication analysis unit, and a business support proposal unit. The robot operation information management unit comprehensively stores and manages information such as the operating status, temperature, vibration, power consumption, cycle time, work content, operation history, maintenance history, work instructions, quality control records, equipment manuals, and past trouble cases of each robot as long-term memory. It also has the function of managing the skills and training history of on-site workers, as well as troubles and improvement cases that occurred on-site. The communication analysis unit analyzes communication data such as daily reports, chats, trouble reports, and meeting minutes entered by on-site workers. The system has the function of understanding the situation on site by collecting data in real time and using natural language processing by generative AI to analyze from multiple angles the raw voices, challenges, operational errors, unclear procedures, workload, and training needs of the field. The Business Support Proposal Department, based on the information obtained from the Robot Operation Information Management Department and the Communication Analysis Department, uses generative AI to automatically generate in real time and comprehensively specific countermeasures, work instructions, training plans, improvement measures, troubleshooting procedures, optimization proposals, and quality improvement measures for robot operation monitoring, anomaly detection, failure prediction, maintenance planning, quality control, work procedure optimization, training support for field workers, knowledge sharing, and overall productivity improvement of the site, and presents them to the field. (Claim 2)

[0183] The system according to claim 1, wherein the business support proposal unit analyzes robot operation data and, when it detects abnormal values ​​such as temperature or vibration or an exceedance of the cycle time standard, automatically generates proposals for cooling measures, operation interval adjustments, and optimization of work procedures and parameters using a generation AI, immediately presents specific improvement measures and work instructions to the on-site personnel, and further, when an error occurs, automatically generates the optimal troubleshooting procedure using a generation AI by referring to past trouble cases and manuals, and notifies the on-site personnel. (Claim 3)

[0184] The system according to claim 1, wherein the business support proposal unit automatically generates work instructions and necessary parts lists for periodic maintenance and preventive maintenance using generative AI based on maintenance history and operational data, analyzes quality control records and automatically proposes cause analysis and quality improvement measures if the defect rate is high, detects operational errors or training needs from the communication logs of on-site workers and automatically generates optimal training plans, manuals, training materials, and OJT plans, and further automatically registers new troubles and improvement cases that occur on-site in a knowledge base and shares them with other workers. [Explanation of symbols]

[0185] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. In an organization where business operations and communication have been digitized, this system operates using generative AI and comprises a Corporate-Specific Information Management Department, a Communication Analysis Department, and a Business Support Proposal Department. The Corporate-Specific Information Management Department integrates and manages corporate-specific information such as internal business knowledge, business rules, workflows, management indicators, KPIs, issues, ongoing initiatives, project information, individual employee skill sets and career goals, and past communication history as long-term memory, and also has the function of maintaining general industry knowledge and specialized knowledge. The Communication Analysis Department constantly participates in all digital communications such as email, chat, and meeting minutes, and uses generative AI-based natural language processing to analyze the context of communication content, addressing progress delays, excessive manual work, and information sharing. The system has the ability to detect issues such as stagnation, imbalances in workload, increased overtime, lack of skills, and risk factors in real time. Based on information obtained from the aforementioned Corporate-Specific Information Management Department and Communication Analysis Department, the Business Support Proposal Department uses generative AI to propose specific improvement measures for streamlining and improving the quality of business processes and workflows, determine the necessity of meetings and schedule them, support the automation of applications and procedures, propose task redistribution and work adjustments, propose learning opportunities and training for skill development and career advancement for each employee, automatically generate project plans and documents, provide budget management and cost reduction proposals, monitor task progress and notify of delays, and conduct risk assessments and propose countermeasures. These functions are performed in real time and comprehensively in accordance with the organization's actual situation and context.

2. The system according to claim 1, wherein the business support proposal unit monitors each employee's workload and overtime hours in real time, proposes task redistribution considering the availability and skill sets of other employees if tasks are concentrated on a particular employee, alerts supervisors to the need for work adjustments if it detects an employee whose overtime hours are increasing, automatically matches each employee's skill set with the skills required for ongoing or planned projects, automatically searches for relevant in-house training or external online courses and proposes participation if a skill deficiency is found, and in addition creates a long-term skill acquisition plan in line with the employee's career goals and regularly follows up on its progress.

3. The system according to claim 1, wherein the business support proposal unit, in project management, analyzes the success factors and failure cases of similar past projects during the planning stage to automatically generate a plan template; in the execution stage, aggregates expenditure data in real time and immediately issues an alert if there are signs of budget overruns, and presents cost reduction proposals and alternatives; in task progress management, constantly monitors the scheduled completion date and actual progress of each task and automatically notifies the person in charge and the manager if delays occur; and in risk management, analyzes the project content and external environment to list potential risk factors, evaluates the impact and probability of occurrence, and proposes specific countermeasures for high-priority risks.

Images