system

Abstract

Provide a system. A method for receiving drawing data from a user; A means for analyzing the received drawing data and extracting information on the area to be cleaned; means for automatically calculating a cleaning route based on the extracted information; means for transmitting the calculated cleaning route to the cleaning robot; A means for the cleaning robot to perform cleaning work according to a route; means for notifying the user of the progress of the cleaning operation; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Efficient cleaning work is an important issue in modern buildings and large facilities. However, manual cleaning and manual operation of cleaning robots are time-consuming and labor-intensive, and often require specialized knowledge. Therefore, there is a need for a system that can be easily operated by cleaning staff and field workers and that can clean efficiently. In addition, it is necessary to know the progress of cleaning work in real time, which is desirable for reducing costs for managers and improving work efficiency. Against this background, there is a need for a simple and efficient cleaning route generation and automated cleaning system. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: a system including a means for receiving drawing data from a user, a means for analyzing the received drawing data to extract information about the area to be cleaned, a means for automatically calculating a cleaning route based on the extracted information, a means for transmitting the calculated cleaning route to a cleaning robot, a means for the cleaning robot to perform cleaning work according to the route, and a means for notifying the user of the progress of the cleaning work. The system also includes a means for applying different parsing logic depending on the format of the drawing data, and a means for taking into account the layout of rooms, the location of hallways, and the presence or absence of obstacles when calculating the optimal cleaning route. This system enables cleaning staff and field workers to efficiently set cleaning routes and perform cleaning work without requiring specialized knowledge, thereby reducing costs for managers and improving work efficiency.

[0006] "Drawing Data" means a digital file format (e.g., PDF, CAD, JPG) that contains information showing the layout of an area within a building or facility.

[0007] "User" refers to the person who operates the system to set and manage cleaning routes.

[0008] The "receiving means" is a function that allows the system to take in drawing data sent from the user.

[0009] The "analysis means" is a function for analyzing the received drawing data and extracting information such as the layout of rooms and the structure of corridors.

[0010] The "means for extracting" is a function for extracting necessary information about the area to be cleaned from the analyzed data.

[0011] The "calculating means" is a function for automatically determining the movement route of the cleaning robot based on the extracted information.

[0012] The "transmitting means" is a function for transferring the calculated cleaning route to the cleaning robot.

[0013] The "means for executing" is a function that enables the cleaning robot to actually perform cleaning along the specified route.

[0014] The "notification means" is a function for notifying the user of the progress and completion information of the cleaning work.

[0015] "Parsing logic" refers to algorithms and techniques for applying different parsing methods depending on the format of the drawing data.

[0016] "Room layout" refers to the location and shape of each room within a building or facility.

[0017] "Corridor location" refers to the location of aisles or corridors within a building or facility.

[0018] "Presence or absence of obstacles" refers to whether there are any objects or structures on the cleaning route that can obstruct the movement of the cleaning robot. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is an automated system for improving the efficiency of cleaning work in buildings and facilities. Specific embodiments of this system are described below.

[0041] System configuration

[0042] Receiving drawing data

[0043] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system. This can be easily done using a dedicated application or a web interface.

[0044] Drawing data analysis

[0045] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0046] Cleaning route calculation

[0047] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0048] Sending cleaning routes to the robot

[0049] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0050] Cleaning by cleaning robot

[0051] The terminal (cleaning robot) automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[0052] Cleaning completion notification

[0053] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0054] Specific examples

[0055] For example, consider a user who wants to clean the ninth floor of an office building. The user uploads a CAD file of the office floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[0056] The cleaning robot automatically cleans the ninth office floor according to the designated route. When cleaning is complete, the server sends a notification to the user, who can then instruct the robot to clean other floors as needed.

[0057] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and allows managers to reduce costs and improve work efficiency.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0061] Step 2:

[0062] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[0063] Step 3:

[0064] The server analyzes the received drawing data, applying parsing logic specific to the specific file format (PDF, CAD, JPG, etc.) to extract detailed layout information, including room layout, corridor location, and the presence or absence of obstacles.

[0065] Step 4:

[0066] The server calculates the optimal cleaning route based on the analyzed layout data, taking into account the layout of rooms, the location of hallways, and the location of obstacles, and sets the shortest route that allows the cleaning robot to move efficiently.

[0067] Step 5:

[0068] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start and end points and movement instructions for each segment.

[0069] Step 6:

[0070] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[0071] Step 7:

[0072] The server receives progress reports from the cleaning robot and uses that data to confirm completion of the cleaning task.

[0073] Step 8:

[0074] The server notifies the user that the cleaning is complete, and the user can then specify what to do next.

[0075] Example 1

[0076] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0077] Traditional cleaning work in buildings and facilities requires manual operation, resulting in problems of inefficiency and inaccuracy. In particular, when cleaning large areas or complex layouts, cleaning planning and execution are often inadequate, consuming a significant amount of time and effort. To solve these problems, a system is needed that automates cleaning work and cleans efficiently and accurately.

[0078] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0079] In this invention, the server includes a means for receiving drawing data from a user, a means for analyzing the received drawing data to extract information on the area to be cleaned, a means for automatically calculating a cleaning route based on the extracted information, and a means for transmitting the information to an automated machine. This allows a user to automatically calculate an optimal cleaning route without specialized knowledge and have the automated machine perform the cleaning work. By notifying the user of the progress of the cleaning work in real time, overall management efficiency can also be improved.

[0080] "Drawing data" is a digital file (PDF, CAD, JPG, etc.) that contains information showing the layout of a building or facility.

[0081] A "user" is a person or organization that operates the system to upload drawing data of the area to be cleaned and manages the execution of the cleaning.

[0082] The "analysis means" is a function that processes the drawing data received by the server and extracts information such as the layout of rooms, the location of corridors, and whether or not there are obstacles.

[0083] A "cleaning route" is a path along which a cleaning robot moves efficiently within an area to be cleaned.

[0084] An "automated machine" is a robot or cleaning device that automatically performs cleaning tasks based on a calculated cleaning route.

[0085] The "notification means" is a function that notifies the user of the progress and completion of the cleaning work.

[0086] An "analysis method" is a technique for analyzing data in different ways depending on the format of the drawing data.

[0087] An "obstacle" is an object or situation that impedes the robot's movement within the area to be cleaned.

[0088] The present invention relates to a system for streamlining and automating cleaning work in buildings and facilities. This system involves receiving and analyzing drawing data, calculating optimal cleaning routes, and issuing instructions to automated machines, aiming to improve the efficiency and management of cleaning work.

[0089] System configuration

[0090] Receiving drawing data

[0091] Users upload blueprints of the areas they wish to clean to the system. The blueprints can be in a variety of formats, including PDF, CAD, and JPG, and can be easily uploaded using a dedicated application or web interface. For example, they can use software or web applications such as "CleanMaster."

[0092] Drawing data analysis

[0093] The server analyzes the drawing data received from the user. Different analysis methods are used depending on the data format. For example, PDF Parser is used to analyze PDFs, and CADAnalyzer is used to analyze CAD data. The server uses this software to extract information such as room layout, aisle locations, and the presence or absence of obstacles.

[0094] Cleaning route calculation

[0095] The server calculates the optimal cleaning route based on the analyzed layout data. It takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles, and generates a route that includes specific movement instructions from the start point to the end point of the cleaning. Software called "Route Optimizer" is used to calculate the route.

[0096] Sending cleaning routes to automated machines

[0097] The server sends the calculated cleaning route to the automated machine (cleaning robot). Software called "RobotCommunicator" is used for communication. The information sent includes the location where cleaning should start, the route to follow, and the location where cleaning should end.

[0098] Carrying out cleaning tasks with automated machines

[0099] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with software called "AutoCleanPro" and efficiently performs work by following the specified route.

[0100] Cleaning completion notification

[0101] When the cleaning is complete, the server notifies the user using the "NotificationManager." The user can receive this notification, understand the progress of the cleaning work in real time, and issue instructions for the next work if necessary.

[0102] Specific examples

[0103] For example, if a user wants to clean the 9th floor of an office building, the system will operate as follows:

[0104] 1. A user uses the "CleanMaster" application to upload a CAD file of an office floor to the system.

[0105] 2. The server uses "CADAnalyzer" to analyze this file and extract the room layout and corridor structure.

[0106] 3. The server uses "Route Optimizer" to calculate the optimal cleaning route based on the analyzed information and sends this route information to the cleaning robot.

[0107] 4. The terminal (automatic machine) uses "AutoCleanPro" to automatically clean the 9th office floor according to the specified route.

[0108] 5. When the cleaning is complete, the server uses "NotificationManager" to send a notification to the user's smartphone saying "Cleaning of the 9th floor is complete."

[0109] Prompt Sentence Examples

[0110] "Please provide the specific processing steps for a system that calculates the optimal cleaning route for cleaning the ninth floor of an office building and sends the results to an automated machine. Please provide a detailed explanation, including the names of the specific hardware and software."

[0111] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and allows managers to reduce costs and improve work efficiency.

[0112] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0113] Step 1:

[0114] Users upload blueprint data of the area they wish to clean to the system. The blueprint data can be in a variety of formats, including PDF, CAD, and JPG, and can be uploaded using a dedicated application or web interface. Specifically, users open the "CleanMaster" application, click the blueprint data upload button, and select a file from their local device. The input is the blueprint data file selected by the user, which is sent to the server. The output is the blueprint data received by the server.

[0115] Step 2:

[0116] The server analyzes the drawing data received from the user. First, it identifies the data format and applies the appropriate analysis method. For example, it uses "PDFParser" for PDF and "CADAnalyzer" for CAD. From the analyzed data, it extracts information such as the layout of rooms, the location of aisles, and the presence or absence of obstacles. Specifically, the server reads the drawing data and begins the analysis process based on the identified format. The input is the drawing data received from the user, and the output is the analyzed layout information.

[0117] Step 3:

[0118] The server calculates the optimal cleaning route based on the analyzed layout data. The calculation uses "RouteOptimizer" and takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles. Specifically, the server takes the analyzed data as input and applies an algorithm to calculate the optimal route. The calculation results include the cleaning start point, movement path, and cleaning end point. The input is the analyzed layout data, and the output is the calculated cleaning route.

[0119] Step 4:

[0120] The server sends the calculated cleaning route to the automated machine. For communication, it uses "RobotCommunicator" to send specific cleaning instructions to the automated machine. Specifically, the server prepares cleaning route data and sends the data to the automated machine using a communication protocol. The input is the calculated cleaning route, and the output is the cleaning route information received by the automated machine.

[0121] Step 5:

[0122] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with "AutoCleanPro" and cleans efficiently according to the specified route. Specifically, the automatic machine starts cleaning and cleans each area as it moves along the route. The input is the cleaning route information received from the server, and the output is the completion of cleaning.

[0123] Step 6:

[0124] When cleaning is complete, the server notifies the user that the work is complete. Notification is sent using "NotificationManager." Specifically, the server generates cleaning completion information and sends a notification to the user's device. The input is a cleaning completion report from the automatic machine, and the output is a cleaning completion notification to the user.

[0125] (Application example 1)

[0126] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] In conventional cleaning systems, analyzing drawing data and calculating cleaning routes are often done manually, making it difficult to work efficiently. Even when cleaning robots fully automate their work, there is no way for users to grasp the progress in real time, making it difficult to manage the progress of cleaning. Furthermore, cleaning instructions cannot be intuitively given using a smartphone, which makes it inconvenient.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0129] In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to a cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for displaying the cleaning route and progress on a smartphone device and sending cleaning instructions, thereby enabling efficient real-time management of cleaning work and progress, and allowing the user to intuitively give cleaning instructions using their smartphone.

[0130] "Drawing data" is a data format that contains layout information for the area to be cleaned, and is available in formats such as PDF, CAD, and JPG.

[0131] A "user" is an individual or organization that uses the cleaning system to give cleaning instructions.

[0132] "Analyzing" means extracting necessary information from the received drawing data, such as the layout of rooms, the structure of passageways, and the presence or absence of obstacles.

[0133] The "area to be cleaned" refers to the area where cleaning work will be carried out, including the interior of the building, specific rooms, hallways, etc.

[0134] A "cleaning route" is a specific path from a cleaning start point to an end point that a cleaning robot should follow.

[0135] A "cleaning robot" is an autonomous mechanical device that automatically performs cleaning tasks according to a calculated cleaning route.

[0136] "Progress" refers to dynamic information such as the progress of the cleaning work and the current cleaning location.

[0137] A "smartphone device" is a portable information terminal that supports Internet connection and is easy for users to operate.

[0138] "Parsing logic" is an analysis algorithm that performs different processing depending on the format of the drawing data.

[0139] "Room layout" is information indicating the relative positions and sizes of each room within a building.

[0140] "Corridor location" is information indicating the location of a passageway within a building.

[0141] An "obstacle" is an object or structure that blocks the movement of the cleaning robot during cleaning work.

[0142] The present invention is an automated system for improving the efficiency of cleaning work in a factory. A specific embodiment of this system will be described below.

[0143] System configuration

[0144] The system consists of a user, a server, a cleaning robot, and a smartphone device.

[0145] Receiving drawing data

[0146] Users upload blueprint data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device, either through a dedicated application or a web interface.

[0147] Drawing data analysis

[0148] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0149] Cleaning route calculation

[0150] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0151] Display of cleaning route

[0152] The server sends the calculated cleaning route to the smartphone device, where the user can visually check the route in the application.

[0153] Send cleaning instructions

[0154] After the user confirms the cleaning route, they send cleaning instructions to the cleaning robot via smartphone, and the cleaning robot is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0155] Cleaning by cleaning robot

[0156] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[0157] Cleaning completion notification

[0158] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0159] Hardware and software used

[0160] Hardware: smartphones, cleaning robots

[0161] Software: Flask, PIL (Python Imaging Library), pytesseract, dedicated smartphone application

[0162] Specific examples

[0163] As an example, consider the case where a user cleans the first floor of a factory. The user uploads a CAD file of the factory floor to the system using a smartphone. The server receives this file and analyzes it to extract the layout of rooms and the structure of corridors. The server then calculates the optimal cleaning route based on the extracted information and displays this route on the smartphone. The user checks the displayed route and sends cleaning instructions to the cleaning robot via the smartphone app. The cleaning robot automatically cleans the factory floor according to the specified route, and a notification is sent to the smartphone when cleaning is complete.

[0164] Prompt Sentence Examples

[0165] "We would like to clean the first floor of a factory. Please analyze the following drawing data and calculate the optimal cleaning route. Also, please be able to check the progress of the cleaning after it starts. Drawing data: 'factory_floor_plan.jpg'"

[0166] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0167] Step 1:

[0168] The user uploads drawing data (e.g., PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device. The uploaded drawing data is sent to the server. The input here is the drawing data, and the output is the transmission of the drawing data to the server.

[0169] Step 2:

[0170] The server parses the received drawing data. Different parsing logic is applied depending on the data format. The input is the drawing data received by the server, and the output is information about the parsed area (e.g., room layout, corridor location, presence or absence of obstacles).

[0171] Step 3:

[0172] The server calculates the optimal cleaning route based on the analyzed information. This calculation takes into account the layout of the rooms, the location of the hallways, and the presence or absence of obstacles. The input is the analyzed area information, and the output is the calculated cleaning route. Specifically, an optimization calculation is performed using an algorithm.

[0173] Step 4:

[0174] The server sends the calculated cleaning route to the smartphone device for visual confirmation by the user, where the input is the calculated cleaning route and the output is sending it to the smartphone device and displaying the route.

[0175] Step 5:

[0176] The user checks the displayed cleaning route through the smartphone application. After checking, the user sends cleaning instructions to the cleaning robot from the smartphone. The input is the user's confirmation and cleaning instructions, and the output is the cleaning instructions sent to the cleaning robot.

[0177] Step 6:

[0178] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area. The input is the cleaning route information for the cleaning robot, and the output is the behavior of the cleaning robot and the progress of the cleaning task.

[0179] Step 7:

[0180] When cleaning is complete, the cleaning robot sends a notification of completion to the server. The server receives this notification and sends it to the user's smartphone. The input is the cleaning completion notification from the cleaning robot, and the output is the cleaning completion notification to the user.

[0181] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0182] The present invention provides an automated system for improving the efficiency of cleaning work in buildings and facilities, and further includes a function for recognizing the user's emotions and adapting operations accordingly. Specific embodiments of this system are described below.

[0183] System configuration

[0184] Receiving drawing data

[0185] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0186] Drawing data analysis

[0187] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0188] Cleaning route calculation

[0189] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0190] Sending cleaning routes to the robot

[0191] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0192] Cleaning by cleaning robot

[0193] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot follows the specified route and cleans each area efficiently. The robot's operation is monitored in real time and its progress is reported to the server.

[0194] Cleaning completion notification

[0195] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0196] Emotion recognition engine configuration

[0197] Emotion engine integration

[0198] The server has a built-in emotion recognition engine that analyzes the user's voice and image input to recognize the user's emotions. This emotion recognition is performed using technologies such as voice tone, language analysis, and facial expression recognition.

[0199] Adapting emotion-based operations

[0200] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on the emotional information. For example, if the user is in a hurry, the cleaning speed can be increased, but if the user is relaxed, the robot will continue cleaning at a standard speed.

[0201] Specific examples

[0202] As an example, consider a situation where a user wants to clean a floor of an office building and the system needs to recognize emotions from the user's voice. The user uploads a CAD file of the ninth floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[0203] If a user says, "I want to finish cleaning quickly," the emotion engine will recognize the urgency from the tone and content of the words and instruct the cleaning robot to speed up its operation. The cleaning robot will follow the instruction and clean the floor at a faster pace than usual. Once cleaning is complete, the server will notify the user that the cleaning is complete. The user can receive this notification and instruct the robot on the next task.

[0204] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and performs optimal operations based on emotions, further enabling managers to reduce costs and improve work efficiency.

[0205] The processing flow will be explained below.

[0206] Step 1:

[0207] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0208] Step 2:

[0209] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[0210] Step 3:

[0211] The server analyzes the received drawing data. Different parsing logic is used depending on the data format. For example, PDFs use parsing logic specific to PDFs, and CAD uses parsing logic specific to CADs. Once the analysis is complete, information such as the layout of rooms, the location of corridors, and whether or not there are obstacles is extracted.

[0212] Step 4:

[0213] The server then calculates the optimal cleaning route based on the analyzed layout data. This calculation is designed to allow the cleaning robot to move efficiently, setting the shortest path based on the layout of rooms, the location of hallways, and the presence or absence of obstacles.

[0214] Step 5:

[0215] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start point, end point, and movement instructions for each segment.

[0216] Step 6:

[0217] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[0218] Step 7:

[0219] The server uses an emotion engine that analyzes the user's voice and image inputs to recognize the user's emotions, for example, if the user is in a hurry or frustrated.

[0220] Step 8:

[0221] The server dynamically adjusts the cleaning robot's behavior based on the recognized emotion information, such as increasing the cleaning speed if the user is in a hurry, or maintaining a standard speed if the user is relaxed.

[0222] Step 9:

[0223] When the cleaning is complete, the server notifies the user that the cleaning is complete. This notification is given in a way that reflects the user's emotions. For example, if the user is feeling stressed, an encouraging message may be added.

[0224] Step 10:

[0225] The user receives a notification when cleaning is complete and can give instructions for the next cleaning job to the system. The user can then upload the drawing data again to clean the next area.

[0226] Example 2

[0227] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0228] While conventional cleaning systems are capable of calculating efficient cleaning routes and avoiding obstacles, they do not adapt their behavior to the user's emotions. As a result, they are not able to respond flexibly to situations such as emergencies or when the user wants to relax. This results in issues such as not maximizing the efficiency of cleaning work and reducing user satisfaction.

[0229] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for recognizing the user's emotions and adjusting the operation of the cleaning robot based on the information. This makes it possible to perform optimal cleaning work according to the user's emotions and situation.

[0230] "Drawing data" refers to digital files that represent the layout and structure of a building or facility, and includes formats such as PDF, CAD, and JPG.

[0231] The "means of receiving" is an interface for receiving input data from the user on the server side, such as a web form or API.

[0232] The "means of analysis" refers to the technology for digitally analyzing received drawing data, and uses various parsing logic such as Adobe Acrobat API, Autodesk API, and image recognition libraries.

[0233] "Means of extraction" refers to the technology used to extract useful information from the analyzed data, such as room layout, corridor location, and the presence of obstacles.

[0234] "Means for automatically calculating cleaning routes" refers to technology that calculates the optimal cleaning route based on the extracted information, and uses a pathfinding algorithm (e.g., A algorithm).

[0235] "Means for sending" refers to a communication means for transmitting the calculated cleaning route to the cleaning robot, and includes REST API and MQTT protocol.

[0236] A "cleaning robot" refers to a mechanical device that automatically performs cleaning tasks based on a programmed route.

[0237] "Means for executing" refers to the technology that enables the cleaning robot to automatically perform cleaning tasks according to a programmed route.

[0238] "Means of notification" refers to technologies used to notify users of the progress and completion status of cleaning work, including email and in-app notifications.

[0239] "Means for recognizing emotions" refers to technology that analyzes the user's voice and image input and identifies emotions, and uses voice recognition APIs and facial expression recognition technology.

[0240] "Means for adjusting behavior" refers to technology that varies the speed and performance of the cleaning robot's movements based on recognized emotional information.

[0241] The present invention provides an automated system that improves the efficiency of cleaning work in buildings and facilities, and further has the function of recognizing the user's emotions and adapting operations accordingly. A specific embodiment of this system is described below.

[0242] Receiving drawing data

[0243] Users upload drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface. Drawing data uploading is done through a dedicated web interface, and users can use a file selection dialog by clicking the "Drawing Upload" button. Drawing data uploaded by users is sent to the server.

[0244] Drawing data analysis

[0245] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. For example, PDF data is analyzed using Adobe Acrobat API, CAD data using Autodesk API, and JPG data using image recognition libraries such as OpenCV. Important information such as room layout, corridor locations, and the presence or absence of obstacles is extracted from the analyzed data.

[0246] Cleaning route calculation

[0247] The server calculates the optimal cleaning route based on the analyzed layout data. This calculation uses a pathfinding algorithm, specifically the A algorithm. The cleaning route calculation takes into account the layout of rooms, the location of corridors, and the presence or absence of obstacles. The server sends the analyzed data to the route calculation module, which obtains the optimal route.

[0248] Sending cleaning routes to the robot

[0249] The server sends the calculated cleaning route to the cleaning robot. This communication uses REST API and MQTT protocol. The server converts the route information into JSON format and sends it to the cleaning robot's endpoint.

[0250] Cleaning by cleaning robot

[0251] The terminal (cleaning robot) starts cleaning based on the received cleaning route information. The robot uses LIDAR and camera modules to avoid obstacles in real time and cleans efficiently by following the specified route.

[0252] Cleaning completion notification

[0253] When the cleaning is complete, the server notifies the user that the cleaning work is complete. This notification can be sent via email or in-app notification, for example, by generating a message informing the user that the document has been completed and sending it to the user's registered email address.

[0254] Emotion recognition engine configuration

[0255] The server is equipped with an emotion recognition engine that analyzes voice and image input from the user to recognize the user's emotions. Emotion recognition uses Google® Cloud Speech-to-Text API and Microsoft® Cognitive Services. Voice data is acquired in real time, converted into text, and the emotions are analyzed. Similarly, image analysis technology is used to recognize facial expressions using image data.

[0256] Adapting emotion-based operations

[0257] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on that emotional information. For example, if the user wants to finish cleaning quickly, the server instructs the cleaning robot to increase its speed. If the user is relaxed, the robot continues cleaning at a normal speed. The server sets the cleaning robot's operating parameters based on the data obtained from the emotion recognition engine and transmits the settings to the robot in real time.

[0258] Specific examples

[0259] As a concrete example, consider a user cleaning a floor of an office building. The user uploads a CAD file of the ninth floor to the system. The server analyzes this file and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and transmits this route to the cleaning robot.

[0260] If the user says, "I want to hurry up and finish cleaning," the server uses the Google Cloud Speech-to-Text API to recognize emotions from the tone and content of the words and instructs the cleaning robot to increase its speed. The cleaning robot follows the instruction and cleans the floor at a faster pace than usual. When cleaning is complete, the server sends a notification to the user. The user can then instruct the robot on the next task.

[0261] Example prompts to input to the generative AI model:

[0262] "Please describe in detail a scenario in which a cleaning system for an office building requires a user to hurry up and complete the cleaning. Explain the process of using voice recognition to analyze the user's emotions and adjust the cleaning robot's speed based on those emotions."

[0263] This system allows users to efficiently perform cleaning tasks without specialized knowledge, and also optimizes operations based on the user's emotions, allowing managers to reduce costs and improve work efficiency.

[0264] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0265] Step 1:

[0266] The user uploads drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned from the system interface. The input is the drawing data file from the user, and the output is the transmission of the drawing data to the server. The user accesses the web interface, clicks the "Drawing Upload" button to open a file selection dialog, and selects the drawing data. The selected data is sent to the server and saved.

[0267] Step 2:

[0268] The server analyzes the drawing data received from the user. The input is the received drawing data (PDF, CAD, JPG, etc.), and the output is analyzed area information (room layout, hallway location, presence or absence of obstacles). The server analyzes the data using Adobe Acrobat API, Autodesk API, OpenCV, etc., and extracts the necessary information. For example, in the case of PDF data, the Adobe Acrobat API is used to check physical structure information, and in the case of CAD data, the Autodesk API is used to analyze room layout and structure.

[0269] Step 3:

[0270] The server calculates the optimal cleaning route based on the analyzed area information. The input is the analyzed area information, and the output is the optimal cleaning route (movement instructions in JSON format). This route calculation uses algorithm A, which derives the shortest route from the area information, taking into account the layout of rooms, the location of hallways, and the presence or absence of obstacles. Specifically, the data is sent to a route calculation module inside the server, which calculates the optimal route within the analyzed area.

[0271] Step 4:

[0272] The server sends the calculated cleaning route to the cleaning robot. The input is the calculated cleaning route, and the output is the route information received by the cleaning robot. The server converts the route information into JSON format and sends it by specifying the endpoint of the cleaning robot. REST API and MQTT protocol are used as communication protocols.

[0273] Step 5:

[0274] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The input is the cleaning route information sent from the server, and the output is the progress of the cleaning work. The cleaning robot uses LIDAR and camera modules to clean efficiently by following the specified route while avoiding obstacles in real time. The robot's progress and data are periodically reported to the server.

[0275] Step 6:

[0276] When the cleaning task is completed, the server sends a cleaning completion notification to the user. The input is the task completion report from the cleaning robot, and the output is the completion notification (email or in-app notification) to the user. The server generates a completion notification based on the completion report received from the cleaning robot and sends it to the user. For example, in the case of an email notification, the completion message is sent to the user's registered email address.

[0277] Step 7:

[0278] The server uses an emotion recognition engine to analyze the user's voice and image inputs and recognize emotions. The input is the user's voice data or image data, and the output is recognized emotional information. The server uses Google Cloud Speech-to-Text API or Microsoft Cognitive Services to convert the voice data into text and detect emotions based on that. Similarly, image analysis technology is used for facial expression recognition.

[0279] Step 8:

[0280] The server dynamically adjusts the behavior of the cleaning robot based on the recognized emotional information. The input is the recognized emotional information, and the output is the adjusted behavior parameters of the cleaning robot. For example, if the user says, "I want to finish cleaning quickly," the server recognizes the urgency and instructs the cleaning robot to increase its speed. This instruction is sent to the robot in real time, and the robot adjusts its behavior accordingly.

[0281] (Application example 2)

[0282] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0283] Efficient cleaning of large areas is an important task in many modern facilities and factories. However, conventional cleaning systems often rely on manual user control to set cleaning routes and adjust the cleaning robot's behavior, resulting in reduced work efficiency and the risk of human error. Furthermore, the robot's behavior cannot be flexibly adjusted according to the user's emotional state, making it difficult to adapt to situations requiring a rapid response or specific circumstances.

[0284] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, means for recognizing the user's emotion using an emotion recognition engine, and means for dynamically adjusting the operating speed and work route of the cleaning robot based on the recognized emotion. This enables flexible cleaning work to be performed in accordance with the user's emotional state and efficient cleaning work that does not rely on conventional manual operation.

[0285] "Drawing data" refers to data that contains layout information for facilities and areas, and includes different formats such as PDF, CAD, and JPG.

[0286] A "cleaning target area" is a specific area where the cleaning robot should work.

[0287] A "cleaning route" is the path that a cleaning robot should follow within the area to be cleaned.

[0288] A "cleaning robot" is a device that autonomously performs cleaning tasks along a designated route.

[0289] An "emotion recognition engine" is a system for recognizing emotions from a user's voice and image, using voice tone, facial expressions, language analysis, etc.

[0290] The "server" is a central processing unit that analyzes drawing data, calculates cleaning routes, sends instructions to cleaning robots, and recognizes emotions.

[0291] A "user" is a person who operates the system and issues instructions to the cleaning robot.

[0292] "Parsing logic" is an analytical method for analyzing drawing data and extracting necessary information.

[0293] The "operation speed" is the speed at which the cleaning robot moves along the cleaning route.

[0294] "Dynamic adjustment" refers to adaptively changing operating parameters based on real-time changing conditions.

[0295] This invention is an automated system for improving the efficiency of cleaning work in facilities and factories, and is particularly equipped with a function to recognize user emotions and adapt operations. This system analyzes drawing data, calculates cleaning routes, controls cleaning robots, recognizes user emotions using an emotion recognition engine, and adjusts operations.

[0296] The system program is configured as follows:

[0297] Receiving and analyzing drawing data

[0298] Users upload drawing data (e.g., PDF, CAD, JPG) showing the layout of a facility or factory to the server. The server receives this data, parses it, and extracts information about the area to be cleaned. The parsing logic used varies depending on the format of the drawing data.

[0299] Calculate and send cleaning routes

[0300] The server calculates the optimal cleaning route based on the extracted information and transmits it to the cleaning robot, taking into account factors such as the layout of rooms within the area to be cleaned, the location of hallways, and whether there are any obstacles.

[0301] Emotion Recognition and Behavior Regulation

[0302] The server has a built-in emotion recognition engine that analyzes the user's voice and image inputs to recognize emotions. The "speech_recognition" library is used for voice recognition technology, and the "OpenCV" library is used for facial expression recognition. The emotion recognition engine analyzes the user's voice tone, language analysis, and facial expressions to identify emotions. Based on the recognized emotions, the server dynamically adjusts the cleaning robot's operating speed and work route.

[0303] Hardware or software used

[0304] Server: Serves as the central processing unit, analyzing drawing data, calculating cleaning routes, controlling the cleaning robot, and performing emotion recognition.

[0305] Cleaning robot: An automated device that follows a designated route to perform cleaning tasks.

[0306] Emotion Recognition Engine: Software for analyzing voice tone and facial expressions. Libraries used include speech_recognition and OpenCV.

[0307] Specific examples

[0308] For example, when a user wants to clean a factory floor, they open a "cleaning app" on their smartphone and upload the factory layout data. Then, the user can voice-input "please hurry up and finish cleaning." The emotion recognition engine recognizes the urgency of this input. The server then responds by sending a command to speed up the cleaning robot's movements. Once the cleaning task is complete, the server notifies the user.

[0309] Example prompt sentence:

[0310] The user opens the "cleaning app" on their smartphone, uploads the layout data of the factory, and then says, "I'm busy, please hurry." The smartphone app analyzes this instruction using its emotion engine and sends a command to the cleaning robot to increase its speed.

[0311] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0312] Step 1:

[0313] The user opens the "cleaning app" on their smartphone and uploads the factory layout data (PDF, CAD, JPG, etc.). At this time, the user operates the app's interface to specify the drawing data. The input is the layout data file specified by the user through the app, and the output is that data sent to the server.

[0314] Step 2:

[0315] The server analyzes the drawing data received from the user. It applies parsing logic (analysis method) to the received drawing data and extracts layout information. It reads the drawing data in stages and analyzes information such as room layout, corridor location, and the presence or absence of obstacles. It receives drawing data as input and obtains extracted layout information as output.

[0316] Step 3:

[0317] The server calculates the optimal cleaning route based on the extracted layout information. The calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. A computer algorithm is used to determine the shortest path and most efficient cleaning route. The layout information is used as input, and the optimal cleaning route data is generated as output.

[0318] Step 4:

[0319] The server sends the calculated cleaning route to the cleaning robot. The cleaning route data is sent to the cleaning robot via the network. At this time, the server organizes the route information and converts it into a format that the cleaning robot can understand. The server receives the cleaning route data as input and sends this data to the robot as output.

[0320] Step 5:

[0321] The cleaning robot starts cleaning work based on the received cleaning route information. It follows the predetermined route and performs cleaning in each area. The robot's movement is controlled by a pre-programmed algorithm. It uses the cleaning route information as input and performs the actual cleaning work as output.

[0322] Step 6:

[0323] The server monitors the cleaning progress in real time and notifies the user. It evaluates the robot's progress based on the data received from sensors and cameras and notifies the user with progress updates. It receives status data from the cleaning robot as input and sends progress notifications to the user as output.

[0324] Step 7:

[0325] The server uses an emotion recognition engine to recognize the user's emotions. It uses the smartphone's microphone and camera to analyze voice and facial expressions to identify the user's emotional state. This analysis uses libraries such as speech_recognition and OpenCV. It receives the user's voice and image data as input and extracts emotional information as output.

[0326] Step 8:

[0327] The server dynamically adjusts the cleaning robot's operating speed and work route based on the recognized emotion information. For example, it increases the cleaning speed if the user is in a hurry, and decreases the speed if the user is instructed to clean slowly. It receives emotion information as input and sends instructions to adjust the robot's operating parameters as output.

[0328] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0329] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0330] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0331] [Second embodiment]

[0332] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0333] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0334] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0335] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0336] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0337] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0338] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0339] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0340] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0341] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0342] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0343] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0344] The present invention is an automated system for improving the efficiency of cleaning work in buildings and facilities. Specific embodiments of this system are described below.

[0345] System configuration

[0346] Receiving drawing data

[0347] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system. This can be easily done using a dedicated application or a web interface.

[0348] Drawing data analysis

[0349] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0350] Cleaning route calculation

[0351] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0352] Sending cleaning routes to the robot

[0353] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0354] Cleaning by cleaning robot

[0355] The terminal (cleaning robot) automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[0356] Cleaning completion notification

[0357] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0358] Specific examples

[0359] For example, consider a user who wants to clean the ninth floor of an office building. The user uploads a CAD file of the office floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[0360] The cleaning robot automatically cleans the ninth office floor according to the designated route. When cleaning is complete, the server sends a notification to the user, who can then instruct the robot to clean other floors as needed.

[0361] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and managers can reduce costs and improve work efficiency.

[0362] The processing flow will be explained below.

[0363] Step 1:

[0364] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0365] Step 2:

[0366] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[0367] Step 3:

[0368] The server analyzes the received drawing data, applying parsing logic specific to the specific file format (PDF, CAD, JPG, etc.) to extract detailed layout information, including room layout, corridor location, and the presence or absence of obstacles.

[0369] Step 4:

[0370] The server calculates the optimal cleaning route based on the analyzed layout data, taking into account the layout of rooms, the location of hallways, and the location of obstacles, and sets the shortest route that allows the cleaning robot to move efficiently.

[0371] Step 5:

[0372] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start and end points and movement instructions for each segment.

[0373] Step 6:

[0374] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[0375] Step 7:

[0376] The server receives progress reports from the cleaning robot and uses that data to confirm completion of the cleaning task.

[0377] Step 8:

[0378] The server notifies the user that the cleaning is complete, and the user receives the notification and can then instruct the next step.

[0379] Example 1

[0380] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0381] Traditional cleaning work in buildings and facilities requires manual operation, resulting in problems of inefficiency and inaccuracy. In particular, when cleaning large areas or complex layouts, cleaning planning and execution are often inadequate, consuming a significant amount of time and effort. To solve these problems, a system is needed that automates cleaning work and cleans efficiently and accurately.

[0382] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0383] In this invention, the server includes a means for receiving drawing data from a user, a means for analyzing the received drawing data to extract information on the area to be cleaned, a means for automatically calculating a cleaning route based on the extracted information, and a means for transmitting the information to an automated machine. This allows a user to automatically calculate an optimal cleaning route without specialized knowledge and have the automated machine perform the cleaning work. By notifying the user of the progress of the cleaning work in real time, overall management efficiency can also be improved.

[0384] "Drawing data" is a digital file (PDF, CAD, JPG, etc.) that contains information showing the layout of a building or facility.

[0385] A "user" is a person or organization that operates the system to upload drawing data of the area to be cleaned and manages the execution of the cleaning.

[0386] The "analysis means" is a function that processes the drawing data received by the server and extracts information such as the layout of rooms, the location of corridors, and whether or not there are obstacles.

[0387] A "cleaning route" is a path along which a cleaning robot moves efficiently within an area to be cleaned.

[0388] An "automated machine" is a robot or cleaning device that automatically performs cleaning tasks based on a calculated cleaning route.

[0389] The "notification means" is a function that notifies the user of the progress and completion of the cleaning work.

[0390] An "analysis method" is a technique for analyzing data in different ways depending on the format of the drawing data.

[0391] An "obstacle" is an object or situation that impedes the robot's movement within the area to be cleaned.

[0392] The present invention relates to a system for streamlining and automating cleaning work in buildings and facilities. This system involves receiving and analyzing drawing data, calculating optimal cleaning routes, and issuing instructions to automated machines, aiming to improve the efficiency and management of cleaning work.

[0393] System configuration

[0394] Receiving drawing data

[0395] Users upload blueprints of the areas they wish to clean to the system. The blueprints can be in a variety of formats, including PDF, CAD, and JPG, and can be easily uploaded using a dedicated application or web interface. For example, they can use software or web applications such as "CleanMaster."

[0396] Drawing data analysis

[0397] The server analyzes the drawing data received from the user. Different analysis methods are used depending on the data format. For example, PDF Parser is used to analyze PDFs, and CADAnalyzer is used to analyze CAD data. The server uses this software to extract information such as room layout, aisle locations, and the presence or absence of obstacles.

[0398] Cleaning route calculation

[0399] The server calculates the optimal cleaning route based on the analyzed layout data. It takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles, and generates a route that includes specific movement instructions from the start point to the end point of the cleaning. Software called "Route Optimizer" is used to calculate the route.

[0400] Sending cleaning routes to automated machines

[0401] The server sends the calculated cleaning route to the automated machine (cleaning robot). Software called "RobotCommunicator" is used for communication. The information sent includes the location where cleaning should start, the route to follow, and the location where cleaning should end.

[0402] Carrying out cleaning tasks with automated machines

[0403] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with software called "AutoCleanPro" that efficiently performs work by following the specified route.

[0404] Cleaning completion notification

[0405] When the cleaning is complete, the server notifies the user using the "NotificationManager." The user can receive this notification, understand the progress of the cleaning work in real time, and issue instructions for the next work if necessary.

[0406] Specific examples

[0407] For example, if a user wants to clean the 9th floor of an office building, the system will operate as follows:

[0408] 1. A user uses the "CleanMaster" application to upload a CAD file of an office floor to the system.

[0409] 2. The server uses "CADAnalyzer" to analyze this file and extract the room layout and corridor structure.

[0410] 3. The server uses "Route Optimizer" to calculate the optimal cleaning route based on the analyzed information and sends this route information to the cleaning robot.

[0411] 4. The terminal (automatic machine) uses "AutoCleanPro" to automatically clean the 9th office floor according to the specified route.

[0412] 5. When the cleaning is complete, the server uses "NotificationManager" to send a notification to the user's smartphone saying "Cleaning of the 9th floor is complete."

[0413] Prompt Sentence Examples

[0414] "Please provide the specific processing steps for a system that calculates the optimal cleaning route for cleaning the ninth floor of an office building and sends the results to an automated machine. Please provide a detailed explanation, including the names of the specific hardware and software."

[0415] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and managers can reduce costs and improve work efficiency.

[0416] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0417] Step 1:

[0418] Users upload blueprint data of the area they wish to clean to the system. The blueprint data can be in a variety of formats, including PDF, CAD, and JPG, and can be uploaded using a dedicated application or web interface. Specifically, users open the "CleanMaster" application, click the blueprint data upload button, and select a file from their local device. The input is the blueprint data file selected by the user, which is sent to the server. The output is the blueprint data received by the server.

[0419] Step 2:

[0420] The server analyzes the drawing data received from the user. First, it identifies the data format and applies the appropriate analysis method. For example, it uses "PDFParser" for PDF and "CADAnalyzer" for CAD. From the analyzed data, it extracts information such as the layout of rooms, the location of aisles, and the presence or absence of obstacles. Specifically, the server reads the drawing data and begins the analysis process based on the identified format. The input is the drawing data received from the user, and the output is the analyzed layout information.

[0421] Step 3:

[0422] The server calculates the optimal cleaning route based on the analyzed layout data. The calculation uses "RouteOptimizer" and takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles. Specifically, the server takes the analyzed data as input and applies an algorithm to calculate the optimal route. The calculation results include the cleaning start point, movement path, and cleaning end point. The input is the analyzed layout data, and the output is the calculated cleaning route.

[0423] Step 4:

[0424] The server sends the calculated cleaning route to the automated machine. For communication, it uses "RobotCommunicator" to send specific cleaning instructions to the automated machine. Specifically, the server prepares cleaning route data and sends the data to the automated machine using a communication protocol. The input is the calculated cleaning route, and the output is the cleaning route information received by the automated machine.

[0425] Step 5:

[0426] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with "AutoCleanPro" and cleans efficiently according to the specified route. Specifically, the automatic machine starts cleaning and cleans each area as it moves along the route. The input is the cleaning route information received from the server, and the output is the completion of cleaning.

[0427] Step 6:

[0428] When cleaning is complete, the server notifies the user that the work is complete. Notification is sent using "NotificationManager." Specifically, the server generates cleaning completion information and sends a notification to the user's device. The input is a cleaning completion report from the automatic machine, and the output is a cleaning completion notification to the user.

[0429] (Application example 1)

[0430] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0431] In conventional cleaning systems, analyzing drawing data and calculating cleaning routes are often done manually, making it difficult to work efficiently. Even when cleaning robots fully automate their work, there is no way for users to grasp the progress in real time, making it difficult to manage the progress of cleaning. Furthermore, cleaning instructions cannot be intuitively given using a smartphone, which makes it inconvenient.

[0432] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0433] In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to a cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for displaying the cleaning route and progress on a smartphone device and sending cleaning instructions, thereby enabling efficient real-time management of cleaning work and progress, and allowing the user to intuitively give cleaning instructions using their smartphone.

[0434] "Drawing data" is a data format that contains layout information for the area to be cleaned, and is available in formats such as PDF, CAD, and JPG.

[0435] A "user" is an individual or organization that uses the cleaning system to give cleaning instructions.

[0436] "Analyzing" means extracting necessary information from the received drawing data, such as the layout of rooms, the structure of passageways, and the presence or absence of obstacles.

[0437] The "area to be cleaned" refers to the area where cleaning work will be carried out, including the interior of the building, specific rooms, hallways, etc.

[0438] A "cleaning route" is a specific path from a cleaning start point to an end point that a cleaning robot should follow.

[0439] A "cleaning robot" is an autonomous mechanical device that automatically performs cleaning tasks according to a calculated cleaning route.

[0440] "Progress" refers to dynamic information such as the progress of the cleaning work and the current cleaning location.

[0441] A "smartphone device" is a portable information terminal that supports Internet connection and is easy for users to operate.

[0442] "Parsing logic" is an analysis algorithm that performs different processing depending on the format of the drawing data.

[0443] "Room layout" is information indicating the relative positions and sizes of each room within a building.

[0444] "Corridor location" is information indicating the location of a passageway within a building.

[0445] An "obstacle" is an object or structure that blocks the movement of the cleaning robot during cleaning work.

[0446] The present invention is an automated system for improving the efficiency of cleaning work in a factory. A specific embodiment of this system will be described below.

[0447] System configuration

[0448] The system consists of a user, a server, a cleaning robot, and a smartphone device.

[0449] Receiving drawing data

[0450] Users upload blueprint data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device, either through a dedicated application or a web interface.

[0451] Drawing data analysis

[0452] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0453] Cleaning route calculation

[0454] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0455] Display of cleaning route

[0456] The server sends the calculated cleaning route to the smartphone device, where the user can visually check the route in the application.

[0457] Send cleaning instructions

[0458] After the user confirms the cleaning route, they send cleaning instructions to the cleaning robot via smartphone, and the cleaning robot is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0459] Cleaning by cleaning robot

[0460] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[0461] Cleaning completion notification

[0462] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0463] Hardware and software used

[0464] Hardware: smartphones, cleaning robots

[0465] Software: Flask, PIL (Python Imaging Library), pytesseract, dedicated smartphone application

[0466] Specific examples

[0467] As an example, consider the case where a user cleans the first floor of a factory. The user uploads a CAD file of the factory floor to the system using a smartphone. The server receives this file and analyzes it to extract the layout of rooms and the structure of corridors. The server then calculates the optimal cleaning route based on the extracted information and displays this route on the smartphone. The user checks the displayed route and sends cleaning instructions to the cleaning robot via the smartphone app. The cleaning robot automatically cleans the factory floor according to the specified route, and a notification is sent to the smartphone when cleaning is complete.

[0468] Prompt Sentence Examples

[0469] "We would like to clean the first floor of a factory. Please analyze the following drawing data and calculate the optimal cleaning route. Also, please be able to check the progress of the cleaning after it starts. Drawing data: 'factory_floor_plan.jpg'"

[0470] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0471] Step 1:

[0472] The user uploads drawing data (e.g., PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device. The uploaded drawing data is sent to the server. The input here is the drawing data, and the output is the transmission of the drawing data to the server.

[0473] Step 2:

[0474] The server parses the received drawing data. Different parsing logic is applied depending on the data format. The input is the drawing data received by the server, and the output is information about the parsed area (e.g., room layout, corridor location, presence or absence of obstacles).

[0475] Step 3:

[0476] The server calculates the optimal cleaning route based on the analyzed information. This calculation takes into account the layout of the rooms, the location of the hallways, and the presence or absence of obstacles. The input is the analyzed area information, and the output is the calculated cleaning route. Specifically, an optimization calculation is performed using an algorithm.

[0477] Step 4:

[0478] The server sends the calculated cleaning route to the smartphone device for visual confirmation by the user, where the input is the calculated cleaning route and the output is sending it to the smartphone device and displaying the route.

[0479] Step 5:

[0480] The user checks the displayed cleaning route through the smartphone application. After checking, the user sends cleaning instructions to the cleaning robot from the smartphone. The input is the user's confirmation and cleaning instructions, and the output is the cleaning instructions sent to the cleaning robot.

[0481] Step 6:

[0482] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area. The input is the cleaning route information for the cleaning robot, and the output is the behavior of the cleaning robot and the progress of the cleaning task.

[0483] Step 7:

[0484] When cleaning is complete, the cleaning robot sends a notification of completion to the server. The server receives this notification and sends it to the user's smartphone. The input is the cleaning completion notification from the cleaning robot, and the output is the cleaning completion notification to the user.

[0485] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0486] The present invention provides an automated system for improving the efficiency of cleaning work in buildings and facilities, and further includes a function for recognizing the user's emotions and adapting operations accordingly. Specific embodiments of this system are described below.

[0487] System configuration

[0488] Receiving drawing data

[0489] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0490] Drawing data analysis

[0491] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0492] Cleaning route calculation

[0493] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0494] Sending cleaning routes to the robot

[0495] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0496] Cleaning by cleaning robot

[0497] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot follows the specified route and cleans each area efficiently. The robot's operation is monitored in real time and its progress is reported to the server.

[0498] Cleaning completion notification

[0499] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0500] Emotion recognition engine configuration

[0501] Emotion engine integration

[0502] The server has a built-in emotion recognition engine that analyzes the user's voice and image input to recognize the user's emotions. This emotion recognition is performed using technologies such as voice tone, language analysis, and facial expression recognition.

[0503] Adapting emotion-based operations

[0504] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on the emotional information. For example, if the user is in a hurry, the cleaning speed can be increased, but if the user is relaxed, the robot will continue cleaning at a standard speed.

[0505] Specific examples

[0506] As an example, consider a situation where a user wants to clean a floor of an office building and the system needs to recognize emotions from the user's voice. The user uploads a CAD file of the ninth floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[0507] If a user says, "I want to finish cleaning quickly," the emotion engine will recognize the urgency from the tone and content of the words and instruct the cleaning robot to speed up its operation. The cleaning robot will follow the instruction and clean the floor at a faster pace than usual. Once cleaning is complete, the server will notify the user that the cleaning is complete. The user can receive this notification and instruct the robot on the next task.

[0508] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and performs optimal operations based on emotions, further enabling managers to reduce costs and improve work efficiency.

[0509] The processing flow will be explained below.

[0510] Step 1:

[0511] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0512] Step 2:

[0513] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[0514] Step 3:

[0515] The server analyzes the received drawing data. Different parsing logic is used depending on the data format. For example, PDFs use parsing logic specific to PDFs, and CAD uses parsing logic specific to CADs. Once the analysis is complete, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0516] Step 4:

[0517] The server calculates the optimal cleaning route based on the analyzed layout data. This calculation is designed to allow the cleaning robot to move efficiently, setting the shortest path based on the layout of rooms, the location of hallways, and the presence or absence of obstacles.

[0518] Step 5:

[0519] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start point, end point, and movement instructions for each segment.

[0520] Step 6:

[0521] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[0522] Step 7:

[0523] The server uses an emotion engine that analyzes the user's voice and image inputs to recognize the user's emotions, for example, if the user is in a hurry or frustrated.

[0524] Step 8:

[0525] The server dynamically adjusts the cleaning robot's behavior based on the recognized emotion information, such as increasing the cleaning speed if the user is in a hurry, or maintaining a standard speed if the user is relaxed.

[0526] Step 9:

[0527] When the cleaning is complete, the server notifies the user that the cleaning is complete. This notification is given in a way that reflects the user's emotions. For example, if the user is feeling stressed, an encouraging message may be added.

[0528] Step 10:

[0529] The user receives a notification when cleaning is complete and can give instructions for the next cleaning job to the system. The user can then upload the drawing data again to clean the next area.

[0530] Example 2

[0531] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0532] While conventional cleaning systems are capable of calculating efficient cleaning routes and avoiding obstacles, they do not adapt their behavior to the user's emotions. As a result, they are not able to respond flexibly to situations such as emergencies or when the user wants to relax. This results in issues such as not maximizing the efficiency of cleaning work and reducing user satisfaction.

[0533] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for recognizing the user's emotions and adjusting the operation of the cleaning robot based on the information. This makes it possible to perform optimal cleaning work according to the user's emotions and situation.

[0534] "Drawing data" refers to digital files that represent the layout and structure of a building or facility, and includes formats such as PDF, CAD, and JPG.

[0535] The "means of receiving" is an interface for receiving input data from the user on the server side, such as a web form or API.

[0536] The "means of analysis" refers to the technology for digitally analyzing received drawing data, and uses various parsing logic such as Adobe Acrobat API, Autodesk API, and image recognition libraries.

[0537] "Means of extraction" refers to the technology used to extract useful information from the analyzed data, such as room layout, corridor location, and the presence of obstacles.

[0538] "Means for automatically calculating cleaning routes" refers to technology that calculates the optimal cleaning route based on the extracted information, and uses a pathfinding algorithm (e.g., A algorithm).

[0539] "Means for sending" refers to a communication means for transmitting the calculated cleaning route to the cleaning robot, and includes REST API and MQTT protocol.

[0540] A "cleaning robot" refers to a mechanical device that automatically performs cleaning tasks based on a programmed route.

[0541] "Means for executing" refers to the technology that enables the cleaning robot to automatically perform cleaning tasks according to a programmed route.

[0542] "Means of notification" refers to technologies used to notify users of the progress and completion status of cleaning work, including email and in-app notifications.

[0543] "Means for recognizing emotions" refers to technology that analyzes the user's voice and image input and identifies emotions, and uses voice recognition APIs and facial expression recognition technology.

[0544] "Means for adjusting behavior" refers to technology that varies the speed and performance of the cleaning robot's movements based on recognized emotional information.

[0545] The present invention provides an automated system that improves the efficiency of cleaning work in buildings and facilities, and further has the function of recognizing the user's emotions and adapting operations accordingly. A specific embodiment of this system is described below.

[0546] Receiving drawing data

[0547] Users upload drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface. Drawing data uploading is done through a dedicated web interface, and users can use a file selection dialog by clicking the "Drawing Upload" button. Drawing data uploaded by users is sent to the server.

[0548] Drawing data analysis

[0549] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. For example, PDF data is analyzed using Adobe Acrobat API, CAD data using Autodesk API, and JPG data using image recognition libraries such as OpenCV. Important information such as room layout, corridor locations, and the presence or absence of obstacles is extracted from the analyzed data.

[0550] Cleaning route calculation

[0551] The server calculates the optimal cleaning route based on the analyzed layout data. This calculation uses a pathfinding algorithm, specifically the A algorithm. The cleaning route calculation takes into account the layout of rooms, the location of corridors, and the presence or absence of obstacles. The server sends the analyzed data to the route calculation module, which obtains the optimal route.

[0552] Sending cleaning routes to the robot

[0553] The server sends the calculated cleaning route to the cleaning robot. This communication uses REST API and MQTT protocol. The server converts the route information into JSON format and sends it to the cleaning robot's endpoint.

[0554] Cleaning by cleaning robot

[0555] The terminal (cleaning robot) starts cleaning based on the received cleaning route information. The robot uses LIDAR and camera modules to avoid obstacles in real time and cleans efficiently by following the specified route.

[0556] Cleaning completion notification

[0557] When the cleaning is complete, the server notifies the user that the cleaning work is complete. This notification can be sent via email or in-app notification, for example, by generating a message informing the user that the document has been completed and sending it to the user's registered email address.

[0558] Emotion recognition engine configuration

[0559] The server is equipped with an emotion recognition engine that analyzes voice and image input from the user to recognize the user's emotions. Emotion recognition uses Google Cloud Speech-to-Text API and Microsoft Cognitive Services. Voice data is acquired in real time, converted into text, and analyzed for emotion. Similarly, image analysis technology is used to recognize facial expressions using image data.

[0560] Adapting emotion-based operations

[0561] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on that emotional information. For example, if the user wants to finish cleaning quickly, the server instructs the cleaning robot to increase its speed. If the user is relaxed, the robot continues cleaning at a normal speed. The server sets the cleaning robot's operating parameters based on the data obtained from the emotion recognition engine and transmits the settings to the robot in real time.

[0562] Specific examples

[0563] As a concrete example, consider a user cleaning a floor of an office building. The user uploads a CAD file of the ninth floor to the system. The server analyzes this file and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and transmits this route to the cleaning robot.

[0564] If the user says, "I want to hurry up and finish cleaning," the server uses the Google Cloud Speech-to-Text API to recognize emotions from the tone and content of the words and instructs the cleaning robot to increase its speed. The cleaning robot follows the instruction and cleans the floor at a faster pace than usual. When cleaning is complete, the server sends a notification to the user. The user can then instruct the robot on the next task.

[0565] Example prompts to input to a generative AI model:

[0566] "Please describe in detail a scenario in which a cleaning system for an office building requires a user to hurry up and complete the cleaning. Explain the process of using voice recognition to analyze the user's emotions and adjust the cleaning robot's speed based on those emotions."

[0567] This system allows users to efficiently perform cleaning tasks without specialized knowledge, and also optimizes operations based on the user's emotions, allowing managers to reduce costs and improve work efficiency.

[0568] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0569] Step 1:

[0570] The user uploads drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned from the system interface. The input is the drawing data file from the user, and the output is the transmission of the drawing data to the server. The user accesses the web interface, clicks the "Drawing Upload" button to open a file selection dialog, and selects the drawing data. The selected data is sent to the server and saved.

[0571] Step 2:

[0572] The server analyzes the drawing data received from the user. The input is the received drawing data (PDF, CAD, JPG, etc.), and the output is analyzed area information (room layout, hallway location, presence or absence of obstacles). The server analyzes the data using Adobe Acrobat API, Autodesk API, OpenCV, etc., and extracts the necessary information. For example, in the case of PDF data, the Adobe Acrobat API is used to check physical structure information, and in the case of CAD data, the Autodesk API is used to analyze room layout and structure.

[0573] Step 3:

[0574] The server calculates the optimal cleaning route based on the analyzed area information. The input is the analyzed area information, and the output is the optimal cleaning route (movement instructions in JSON format). This route calculation uses algorithm A, which derives the shortest route from the area information, taking into account the layout of rooms, the location of hallways, and the presence or absence of obstacles. Specifically, the data is sent to a route calculation module inside the server, which calculates the optimal route within the analyzed area.

[0575] Step 4:

[0576] The server sends the calculated cleaning route to the cleaning robot. The input is the calculated cleaning route, and the output is the route information received by the cleaning robot. The server converts the route information into JSON format and sends it by specifying the endpoint of the cleaning robot. REST API and MQTT protocol are used as communication protocols.

[0577] Step 5:

[0578] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The input is the cleaning route information sent from the server, and the output is the progress of the cleaning work. The cleaning robot uses LIDAR and camera modules to clean efficiently by following the specified route while avoiding obstacles in real time. The robot's progress and data are periodically reported to the server.

[0579] Step 6:

[0580] When the cleaning task is completed, the server sends a cleaning completion notification to the user. The input is the task completion report from the cleaning robot, and the output is the completion notification (email or in-app notification) to the user. The server generates a completion notification based on the completion report received from the cleaning robot and sends it to the user. For example, in the case of an email notification, the completion message is sent to the user's registered email address.

[0581] Step 7:

[0582] The server uses an emotion recognition engine to analyze the user's voice and image inputs and recognize emotions. The input is the user's voice data or image data, and the output is recognized emotional information. The server uses Google Cloud Speech-to-Text API or Microsoft Cognitive Services to convert the voice data into text and detect emotions based on that. Similarly, image analysis technology is used for facial expression recognition.

[0583] Step 8:

[0584] The server dynamically adjusts the behavior of the cleaning robot based on the recognized emotional information. The input is the recognized emotional information, and the output is the adjusted behavior parameters of the cleaning robot. For example, if the user says, "I want to finish cleaning quickly," the server recognizes the urgency and instructs the cleaning robot to increase its speed. This instruction is sent to the robot in real time, and the robot adjusts its behavior accordingly.

[0585] (Application example 2)

[0586] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0587] Efficient cleaning of large areas is an important task in many modern facilities and factories. However, conventional cleaning systems often rely on manual user control to set cleaning routes and adjust the cleaning robot's behavior, resulting in reduced work efficiency and the risk of human error. Furthermore, the robot's behavior cannot be flexibly adjusted according to the user's emotional state, making it difficult to adapt to situations requiring a rapid response or specific circumstances.

[0588] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, means for recognizing the user's emotion using an emotion recognition engine, and means for dynamically adjusting the operating speed and work route of the cleaning robot based on the recognized emotion. This enables flexible cleaning work to be performed in accordance with the user's emotional state and efficient cleaning work that does not rely on conventional manual operation.

[0589] "Drawing data" refers to data that contains layout information for facilities and areas, and includes different formats such as PDF, CAD, and JPG.

[0590] A "cleaning target area" is a specific area where the cleaning robot should work.

[0591] A "cleaning route" is the path that a cleaning robot should follow within the area to be cleaned.

[0592] A "cleaning robot" is a device that autonomously performs cleaning tasks along a designated route.

[0593] An "emotion recognition engine" is a system for recognizing emotions from a user's voice and image, using voice tone, facial expressions, language analysis, etc.

[0594] The "server" is a central processing unit that analyzes drawing data, calculates cleaning routes, sends instructions to cleaning robots, and recognizes emotions.

[0595] A "user" is a person who operates the system and issues instructions to the cleaning robot.

[0596] "Parsing logic" is an analytical method for analyzing drawing data and extracting necessary information.

[0597] The "operation speed" is the speed at which the cleaning robot moves along the cleaning route.

[0598] "Dynamic adjustment" refers to adaptively changing operating parameters based on real-time changing conditions.

[0599] This invention is an automated system for improving the efficiency of cleaning work in facilities and factories, and is particularly equipped with a function to recognize user emotions and adapt operations. This system analyzes drawing data, calculates cleaning routes, controls cleaning robots, recognizes user emotions using an emotion recognition engine, and adjusts operations.

[0600] The system program is configured as follows:

[0601] Receiving and analyzing drawing data

[0602] Users upload drawing data (e.g., PDF, CAD, JPG) showing the layout of a facility or factory to the server. The server receives this data, parses it, and extracts information about the area to be cleaned. The parsing logic used varies depending on the format of the drawing data.

[0603] Calculate and send cleaning routes

[0604] The server calculates the optimal cleaning route based on the extracted information and transmits it to the cleaning robot, taking into account factors such as the layout of rooms within the area to be cleaned, the location of hallways, and whether there are any obstacles.

[0605] Emotion Recognition and Behavior Regulation

[0606] The server has a built-in emotion recognition engine that analyzes the user's voice and image inputs to recognize emotions. The "speech_recognition" library is used for voice recognition technology, and the "OpenCV" library is used for facial expression recognition. The emotion recognition engine analyzes the user's voice tone, language analysis, and facial expressions to identify emotions. Based on the recognized emotions, the server dynamically adjusts the cleaning robot's operating speed and work route.

[0607] Hardware or software used

[0608] Server: Serves as the central processing unit, analyzing drawing data, calculating cleaning routes, controlling the cleaning robot, and performing emotion recognition.

[0609] Cleaning robot: An automated device that follows a designated route to perform cleaning tasks.

[0610] Emotion Recognition Engine: Software for analyzing voice tone and facial expressions. Libraries used include speech_recognition and OpenCV.

[0611] Specific examples

[0612] For example, when a user wants to clean a factory floor, they open a "cleaning app" on their smartphone and upload the factory layout data. Then, the user can voice-input "please hurry up and finish cleaning." The emotion recognition engine recognizes the urgency of this input. The server then responds by sending a command to speed up the cleaning robot's movements. Once the cleaning task is complete, the server notifies the user.

[0613] Example prompt sentence:

[0614] The user opens the "cleaning app" on their smartphone, uploads the layout data of the factory, and then says, "I'm busy, please hurry." The smartphone app analyzes this instruction using its emotion engine and sends a command to the cleaning robot to increase its speed.

[0615] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0616] Step 1:

[0617] The user opens the "cleaning app" on their smartphone and uploads the factory layout data (PDF, CAD, JPG, etc.). At this time, the user operates the app's interface to specify the drawing data. The input is the layout data file specified by the user through the app, and the output is that data sent to the server.

[0618] Step 2:

[0619] The server analyzes the drawing data received from the user. It applies parsing logic (analysis method) to the received drawing data and extracts layout information. It reads the drawing data in stages and analyzes information such as room layout, corridor location, and the presence or absence of obstacles. It receives drawing data as input and obtains extracted layout information as output.

[0620] Step 3:

[0621] The server calculates the optimal cleaning route based on the extracted layout information. The calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. A computer algorithm is used to determine the shortest path and most efficient cleaning route. The layout information is used as input, and the optimal cleaning route data is generated as output.

[0622] Step 4:

[0623] The server sends the calculated cleaning route to the cleaning robot. The cleaning route data is sent to the cleaning robot via the network. At this time, the server organizes the route information and converts it into a format that the cleaning robot can understand. The server receives the cleaning route data as input and sends this data to the robot as output.

[0624] Step 5:

[0625] The cleaning robot starts cleaning work based on the received cleaning route information. It follows the predetermined route and performs cleaning in each area. The robot's movement is controlled by a pre-programmed algorithm. It uses the cleaning route information as input and performs the actual cleaning work as output.

[0626] Step 6:

[0627] The server monitors the cleaning progress in real time and notifies the user. It evaluates the robot's progress based on the data received from sensors and cameras and notifies the user with progress updates. It receives status data from the cleaning robot as input and sends progress notifications to the user as output.

[0628] Step 7:

[0629] The server uses an emotion recognition engine to recognize the user's emotions. It uses the smartphone's microphone and camera to analyze voice and facial expressions to identify the user's emotional state. This analysis uses libraries such as speech_recognition and OpenCV. It receives the user's voice and image data as input and extracts emotional information as output.

[0630] Step 8:

[0631] The server dynamically adjusts the cleaning robot's operating speed and work route based on the recognized emotion information. For example, it increases the cleaning speed if the user is in a hurry, and decreases the speed if the user is instructed to clean slowly. It receives emotion information as input and sends instructions to adjust the robot's operating parameters as output.

[0632] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0633] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0634] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0635] [Third embodiment]

[0636] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0637] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0638] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0639] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0640] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0641] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0642] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0643] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0644] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0645] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0646] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0647] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0648] The present invention is an automated system for improving the efficiency of cleaning work in buildings and facilities. Specific embodiments of this system are described below.

[0649] System configuration

[0650] Receiving drawing data

[0651] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system. This can be easily done using a dedicated application or a web interface.

[0652] Drawing data analysis

[0653] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0654] Cleaning route calculation

[0655] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0656] Sending cleaning routes to the robot

[0657] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0658] Cleaning by cleaning robot

[0659] The terminal (cleaning robot) automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[0660] Cleaning completion notification

[0661] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0662] Specific examples

[0663] For example, consider a user who wants to clean the ninth floor of an office building. The user uploads a CAD file of the office floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[0664] The cleaning robot automatically cleans the ninth office floor according to the designated route. When cleaning is complete, the server sends a notification to the user, who can then instruct the robot to clean other floors as needed.

[0665] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and managers can reduce costs and improve work efficiency.

[0666] The processing flow will be explained below.

[0667] Step 1:

[0668] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0669] Step 2:

[0670] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[0671] Step 3:

[0672] The server analyzes the received drawing data, applying parsing logic specific to the specific file format (PDF, CAD, JPG, etc.) to extract detailed layout information, including room layout, corridor location, and the presence or absence of obstacles.

[0673] Step 4:

[0674] The server calculates the optimal cleaning route based on the analyzed layout data, taking into account the layout of rooms, the location of hallways, and the location of obstacles, and sets the shortest route that allows the cleaning robot to move efficiently.

[0675] Step 5:

[0676] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start and end points and movement instructions for each segment.

[0677] Step 6:

[0678] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[0679] Step 7:

[0680] The server receives progress reports from the cleaning robot and uses that data to confirm completion of the cleaning task.

[0681] Step 8:

[0682] The server notifies the user that the cleaning is complete, and the user receives the notification and can then instruct the next step.

[0683] Example 1

[0684] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0685] Traditional cleaning work in buildings and facilities requires manual operation, resulting in problems of inefficiency and inaccuracy. In particular, when cleaning large areas or complex layouts, cleaning planning and execution are often inadequate, consuming a significant amount of time and effort. To solve these problems, a system is needed that automates cleaning work and cleans efficiently and accurately.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0687] In this invention, the server includes a means for receiving drawing data from a user, a means for analyzing the received drawing data to extract information on the area to be cleaned, a means for automatically calculating a cleaning route based on the extracted information, and a means for transmitting the information to an automated machine. This allows a user to automatically calculate an optimal cleaning route without specialized knowledge and have the automated machine perform the cleaning work. By notifying the user of the progress of the cleaning work in real time, overall management efficiency can also be improved.

[0688] "Drawing data" is a digital file (PDF, CAD, JPG, etc.) that contains information showing the layout of a building or facility.

[0689] A "user" is a person or organization that operates the system to upload drawing data of the area to be cleaned and manages the execution of the cleaning.

[0690] The "analysis means" is a function that processes the drawing data received by the server and extracts information such as the layout of rooms, the location of corridors, and whether or not there are obstacles.

[0691] A "cleaning route" is a path along which a cleaning robot moves efficiently within an area to be cleaned.

[0692] An "automated machine" is a robot or cleaning device that automatically performs cleaning tasks based on a calculated cleaning route.

[0693] The "notification means" is a function that notifies the user of the progress and completion of the cleaning work.

[0694] An "analysis method" is a technique for analyzing data in different ways depending on the format of the drawing data.

[0695] An "obstacle" is an object or situation that impedes the robot's movement within the area to be cleaned.

[0696] The present invention relates to a system for streamlining and automating cleaning work in buildings and facilities. This system involves receiving and analyzing drawing data, calculating optimal cleaning routes, and issuing instructions to automated machines, aiming to improve the efficiency and management of cleaning work.

[0697] System configuration

[0698] Receiving drawing data

[0699] Users upload blueprints of the areas they wish to clean to the system. The blueprints can be in a variety of formats, including PDF, CAD, and JPG, and can be easily uploaded using a dedicated application or web interface. For example, they can use software or web applications such as "CleanMaster."

[0700] Drawing data analysis

[0701] The server analyzes the drawing data received from the user. Different analysis methods are used depending on the data format. For example, PDF Parser is used to analyze PDFs, and CADAnalyzer is used to analyze CAD data. The server uses this software to extract information such as room layout, aisle locations, and the presence or absence of obstacles.

[0702] Cleaning route calculation

[0703] The server calculates the optimal cleaning route based on the analyzed layout data. It takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles, and generates a route that includes specific movement instructions from the start point to the end point of the cleaning. Software called "Route Optimizer" is used to calculate the route.

[0704] Sending cleaning routes to automated machines

[0705] The server sends the calculated cleaning route to the automated machine (cleaning robot). Software called "RobotCommunicator" is used for communication. The information sent includes the location where cleaning should start, the route to follow, and the location where cleaning should end.

[0706] Carrying out cleaning tasks with automated machines

[0707] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with software called "AutoCleanPro" that efficiently performs work by following the specified route.

[0708] Cleaning completion notification

[0709] When the cleaning is complete, the server notifies the user using the "NotificationManager." The user can receive this notification, understand the progress of the cleaning work in real time, and issue instructions for the next work if necessary.

[0710] Specific examples

[0711] For example, if a user wants to clean the 9th floor of an office building, the system will operate as follows:

[0712] 1. A user uses the "CleanMaster" application to upload a CAD file of an office floor to the system.

[0713] 2. The server uses "CADAnalyzer" to analyze this file and extract the room layout and corridor structure.

[0714] 3. The server uses "Route Optimizer" to calculate the optimal cleaning route based on the analyzed information and sends this route information to the cleaning robot.

[0715] 4. The terminal (automatic machine) uses "AutoCleanPro" to automatically clean the 9th office floor according to the specified route.

[0716] 5. When the cleaning is complete, the server uses "NotificationManager" to send a notification to the user's smartphone saying "Cleaning of the 9th floor is complete."

[0717] Prompt Sentence Examples

[0718] "Please provide the specific processing steps for a system that calculates the optimal cleaning route for cleaning the ninth floor of an office building and sends the results to an automated machine. Please provide a detailed explanation, including the names of the specific hardware and software."

[0719] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and managers can reduce costs and improve work efficiency.

[0720] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0721] Step 1:

[0722] Users upload blueprint data of the area they wish to clean to the system. The blueprint data can be in a variety of formats, including PDF, CAD, and JPG, and can be uploaded using a dedicated application or web interface. Specifically, users open the "CleanMaster" application, click the blueprint data upload button, and select a file from their local device. The input is the blueprint data file selected by the user, which is sent to the server. The output is the blueprint data received by the server.

[0723] Step 2:

[0724] The server analyzes the drawing data received from the user. First, it identifies the data format and applies the appropriate analysis method. For example, it uses "PDFParser" for PDF and "CADAnalyzer" for CAD. From the analyzed data, it extracts information such as the layout of rooms, the location of aisles, and the presence or absence of obstacles. Specifically, the server reads the drawing data and begins the analysis process based on the identified format. The input is the drawing data received from the user, and the output is the analyzed layout information.

[0725] Step 3:

[0726] The server calculates the optimal cleaning route based on the analyzed layout data. The calculation uses "RouteOptimizer" and takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles. Specifically, the server takes the analyzed data as input and applies an algorithm to calculate the optimal route. The calculation results include the cleaning start point, movement path, and cleaning end point. The input is the analyzed layout data, and the output is the calculated cleaning route.

[0727] Step 4:

[0728] The server sends the calculated cleaning route to the automated machine. For communication, it uses "RobotCommunicator" to send specific cleaning instructions to the automated machine. Specifically, the server prepares cleaning route data and sends the data to the automated machine using a communication protocol. The input is the calculated cleaning route, and the output is the cleaning route information received by the automated machine.

[0729] Step 5:

[0730] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with "AutoCleanPro" and cleans efficiently according to the specified route. Specifically, the automatic machine starts cleaning and cleans each area as it moves along the route. The input is the cleaning route information received from the server, and the output is the completion of cleaning.

[0731] Step 6:

[0732] When cleaning is complete, the server notifies the user that the work is complete. Notification is sent using "NotificationManager." Specifically, the server generates cleaning completion information and sends a notification to the user's device. The input is a cleaning completion report from the automatic machine, and the output is a cleaning completion notification to the user.

[0733] (Application example 1)

[0734] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0735] In conventional cleaning systems, analyzing drawing data and calculating cleaning routes are often done manually, making it difficult to work efficiently. Even when cleaning robots fully automate their work, there is no way for users to grasp the progress in real time, making it difficult to manage the progress of cleaning. Furthermore, cleaning instructions cannot be intuitively given using a smartphone, which makes it inconvenient.

[0736] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0737] In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to a cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for displaying the cleaning route and progress on a smartphone device and sending cleaning instructions, thereby enabling efficient real-time management of cleaning work and progress, and allowing the user to intuitively give cleaning instructions using their smartphone.

[0738] "Drawing data" is a data format that contains layout information for the area to be cleaned, and is available in formats such as PDF, CAD, and JPG.

[0739] A "user" is an individual or organization that uses the cleaning system to give cleaning instructions.

[0740] "Analyzing" means extracting necessary information from the received drawing data, such as the layout of rooms, the structure of passageways, and the presence or absence of obstacles.

[0741] The "area to be cleaned" refers to the area where cleaning work will be carried out, including the interior of the building, specific rooms, hallways, etc.

[0742] A "cleaning route" is a specific path from a cleaning start point to an end point that a cleaning robot should follow.

[0743] A "cleaning robot" is an autonomous mechanical device that automatically performs cleaning tasks according to a calculated cleaning route.

[0744] "Progress" refers to dynamic information such as the progress of the cleaning work and the current cleaning location.

[0745] A "smartphone device" is a portable information terminal that supports Internet connection and is easy for users to operate.

[0746] "Parsing logic" is an analysis algorithm that performs different processing depending on the format of the drawing data.

[0747] "Room layout" is information indicating the relative positions and sizes of each room within a building.

[0748] "Corridor location" is information indicating the location of a passageway within a building.

[0749] An "obstacle" is an object or structure that blocks the movement of the cleaning robot during cleaning work.

[0750] The present invention is an automated system for improving the efficiency of cleaning work in a factory. A specific embodiment of this system will be described below.

[0751] System configuration

[0752] The system consists of a user, a server, a cleaning robot, and a smartphone device.

[0753] Receiving drawing data

[0754] Users upload blueprint data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device, either through a dedicated application or a web interface.

[0755] Drawing data analysis

[0756] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0757] Cleaning route calculation

[0758] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0759] Display of cleaning route

[0760] The server sends the calculated cleaning route to the smartphone device, where the user can visually check the route in the application.

[0761] Send cleaning instructions

[0762] After the user confirms the cleaning route, they send cleaning instructions to the cleaning robot via smartphone, and the cleaning robot is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0763] Cleaning by cleaning robot

[0764] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[0765] Cleaning completion notification

[0766] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0767] Hardware and software used

[0768] Hardware: smartphones, cleaning robots

[0769] Software: Flask, PIL (Python Imaging Library), pytesseract, dedicated smartphone application

[0770] Specific examples

[0771] As an example, consider the case where a user cleans the first floor of a factory. The user uploads a CAD file of the factory floor to the system using a smartphone. The server receives this file and analyzes it to extract the layout of rooms and the structure of corridors. The server then calculates the optimal cleaning route based on the extracted information and displays this route on the smartphone. The user checks the displayed route and sends cleaning instructions to the cleaning robot via the smartphone app. The cleaning robot automatically cleans the factory floor according to the specified route, and a notification is sent to the smartphone when cleaning is complete.

[0772] Prompt Sentence Examples

[0773] "We would like to clean the first floor of a factory. Please analyze the following drawing data and calculate the optimal cleaning route. Also, please be able to check the progress of the cleaning after it starts. Drawing data: 'factory_floor_plan.jpg'"

[0774] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0775] Step 1:

[0776] The user uploads drawing data (e.g., PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device. The uploaded drawing data is sent to the server. The input here is the drawing data, and the output is the transmission of the drawing data to the server.

[0777] Step 2:

[0778] The server parses the received drawing data. Different parsing logic is applied depending on the data format. The input is the drawing data received by the server, and the output is information about the parsed area (e.g., room layout, corridor location, presence or absence of obstacles).

[0779] Step 3:

[0780] The server calculates the optimal cleaning route based on the analyzed information. This calculation takes into account the layout of the rooms, the location of the hallways, and the presence or absence of obstacles. The input is the analyzed area information, and the output is the calculated cleaning route. Specifically, an optimization calculation is performed using an algorithm.

[0781] Step 4:

[0782] The server sends the calculated cleaning route to the smartphone device for visual confirmation by the user, where the input is the calculated cleaning route and the output is sending it to the smartphone device and displaying the route.

[0783] Step 5:

[0784] The user checks the displayed cleaning route through the smartphone application. After checking, the user sends cleaning instructions to the cleaning robot from the smartphone. The input is the user's confirmation and cleaning instructions, and the output is the cleaning instructions sent to the cleaning robot.

[0785] Step 6:

[0786] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area. The input is the cleaning route information for the cleaning robot, and the output is the behavior of the cleaning robot and the progress of the cleaning task.

[0787] Step 7:

[0788] When cleaning is complete, the cleaning robot sends a notification of completion to the server. The server receives this notification and sends it to the user's smartphone. The input is the cleaning completion notification from the cleaning robot, and the output is the cleaning completion notification to the user.

[0789] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0790] The present invention provides an automated system for improving the efficiency of cleaning work in buildings and facilities, and further includes a function for recognizing the user's emotions and adapting operations accordingly. Specific embodiments of this system are described below.

[0791] System configuration

[0792] Receiving drawing data

[0793] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0794] Drawing data analysis

[0795] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0796] Cleaning route calculation

[0797] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0798] Sending cleaning routes to the robot

[0799] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0800] Cleaning by cleaning robot

[0801] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot follows the specified route and cleans each area efficiently. The robot's operation is monitored in real time and its progress is reported to the server.

[0802] Cleaning completion notification

[0803] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0804] Emotion recognition engine configuration

[0805] Emotion engine integration

[0806] The server has a built-in emotion recognition engine that analyzes the user's voice and image input to recognize the user's emotions. This emotion recognition is performed using technologies such as voice tone, language analysis, and facial expression recognition.

[0807] Adapting emotion-based operations

[0808] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on the emotional information. For example, if the user is in a hurry, the cleaning speed can be increased, but if the user is relaxed, the robot will continue cleaning at a standard speed.

[0809] Specific examples

[0810] As an example, consider a situation where a user wants to clean a floor of an office building and the system needs to recognize emotions from the user's voice. The user uploads a CAD file of the ninth floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[0811] If a user says, "I want to finish cleaning quickly," the emotion engine will recognize the urgency from the tone and content of the words and instruct the cleaning robot to speed up its operation. The cleaning robot will follow the instruction and clean the floor at a faster pace than usual. Once cleaning is complete, the server will notify the user that the cleaning is complete. The user can receive this notification and instruct the robot on the next task.

[0812] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and performs optimal operations based on emotions, further enabling managers to reduce costs and improve work efficiency.

[0813] The processing flow will be explained below.

[0814] Step 1:

[0815] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0816] Step 2:

[0817] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[0818] Step 3:

[0819] The server analyzes the received drawing data. Different parsing logic is used depending on the data format. For example, PDFs use parsing logic specific to PDFs, and CAD uses parsing logic specific to CADs. Once the analysis is complete, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0820] Step 4:

[0821] The server calculates the optimal cleaning route based on the analyzed layout data. This calculation is designed to allow the cleaning robot to move efficiently, setting the shortest path based on the layout of rooms, the location of hallways, and the presence or absence of obstacles.

[0822] Step 5:

[0823] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start point, end point, and movement instructions for each segment.

[0824] Step 6:

[0825] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[0826] Step 7:

[0827] The server uses an emotion engine that analyzes the user's voice and image inputs to recognize the user's emotions, for example, if the user is in a hurry or frustrated.

[0828] Step 8:

[0829] The server dynamically adjusts the cleaning robot's behavior based on the recognized emotion information, such as increasing the cleaning speed if the user is in a hurry, or maintaining a standard speed if the user is relaxed.

[0830] Step 9:

[0831] When the cleaning is complete, the server notifies the user that the cleaning is complete. This notification is given in a way that reflects the user's emotions. For example, if the user is feeling stressed, an encouraging message may be added.

[0832] Step 10:

[0833] The user receives a notification when cleaning is complete and can give instructions for the next cleaning job to the system. The user can then upload the drawing data again to clean the next area.

[0834] Example 2

[0835] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0836] While conventional cleaning systems are capable of calculating efficient cleaning routes and avoiding obstacles, they do not adapt their behavior to the user's emotions. As a result, they are not able to respond flexibly to situations such as emergencies or when the user wants to relax. This results in issues such as not maximizing the efficiency of cleaning work and reducing user satisfaction.

[0837] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for recognizing the user's emotions and adjusting the operation of the cleaning robot based on the information. This makes it possible to perform optimal cleaning work according to the user's emotions and situation.

[0838] "Drawing data" refers to digital files that represent the layout and structure of a building or facility, and includes formats such as PDF, CAD, and JPG.

[0839] The "means of receiving" is an interface for receiving input data from the user on the server side, such as a web form or API.

[0840] The "means of analysis" refers to the technology for digitally analyzing received drawing data, and uses various parsing logic such as Adobe Acrobat API, Autodesk API, and image recognition libraries.

[0841] "Means of extraction" refers to the technology used to extract useful information from the analyzed data, such as room layout, corridor location, and the presence of obstacles.

[0842] "Means for automatically calculating cleaning routes" refers to technology that calculates the optimal cleaning route based on the extracted information, and uses a pathfinding algorithm (e.g., A algorithm).

[0843] "Means for sending" refers to a communication means for transmitting the calculated cleaning route to the cleaning robot, and includes REST API and MQTT protocol.

[0844] A "cleaning robot" refers to a mechanical device that automatically performs cleaning tasks based on a programmed route.

[0845] "Means for executing" refers to the technology that enables the cleaning robot to automatically perform cleaning tasks according to a programmed route.

[0846] "Means of notification" refers to technologies used to notify users of the progress and completion status of cleaning work, including email and in-app notifications.

[0847] "Means for recognizing emotions" refers to technology that analyzes the user's voice and image input and identifies emotions, and uses voice recognition APIs and facial expression recognition technology.

[0848] "Means for adjusting behavior" refers to technology that varies the speed and performance of the cleaning robot's movements based on recognized emotional information.

[0849] The present invention provides an automated system that improves the efficiency of cleaning work in buildings and facilities, and further has the function of recognizing the user's emotions and adapting operations accordingly. A specific embodiment of this system is described below.

[0850] Receiving drawing data

[0851] Users upload drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface. Drawing data uploading is done through a dedicated web interface, and users can use a file selection dialog by clicking the "Drawing Upload" button. Drawing data uploaded by users is sent to the server.

[0852] Drawing data analysis

[0853] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. For example, PDF data is analyzed using Adobe Acrobat API, CAD data using Autodesk API, and JPG data using image recognition libraries such as OpenCV. Important information such as room layout, corridor locations, and the presence or absence of obstacles is extracted from the analyzed data.

[0854] Cleaning route calculation

[0855] The server calculates the optimal cleaning route based on the analyzed layout data. This calculation uses a pathfinding algorithm, specifically the A algorithm. The cleaning route calculation takes into account the layout of rooms, the location of corridors, and the presence or absence of obstacles. The server sends the analyzed data to the route calculation module, which obtains the optimal route.

[0856] Sending cleaning routes to the robot

[0857] The server sends the calculated cleaning route to the cleaning robot. This communication uses REST API and MQTT protocol. The server converts the route information into JSON format and sends it to the cleaning robot's endpoint.

[0858] Cleaning by cleaning robot

[0859] The terminal (cleaning robot) starts cleaning based on the received cleaning route information. The robot uses LIDAR and camera modules to avoid obstacles in real time and cleans efficiently by following the specified route.

[0860] Cleaning completion notification

[0861] When the cleaning is complete, the server notifies the user that the cleaning work is complete. This notification can be sent via email or in-app notification, for example, by generating a message informing the user that the document has been completed and sending it to the user's registered email address.

[0862] Emotion recognition engine configuration

[0863] The server is equipped with an emotion recognition engine that analyzes voice and image input from the user to recognize the user's emotions. Emotion recognition uses Google Cloud Speech-to-Text API and Microsoft Cognitive Services. Voice data is acquired in real time, converted into text, and analyzed for emotion. Similarly, image analysis technology is used to recognize facial expressions using image data.

[0864] Adapting emotion-based operations

[0865] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on that emotional information. For example, if the user wants to finish cleaning quickly, the server instructs the cleaning robot to increase its speed. If the user is relaxed, the robot continues cleaning at a normal speed. The server sets the cleaning robot's operating parameters based on the data obtained from the emotion recognition engine and transmits the settings to the robot in real time.

[0866] Specific examples

[0867] As a concrete example, consider a user cleaning a floor of an office building. The user uploads a CAD file of the ninth floor to the system. The server analyzes this file and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and transmits this route to the cleaning robot.

[0868] If the user says, "I want to hurry up and finish cleaning," the server uses the Google Cloud Speech-to-Text API to recognize emotions from the tone and content of the words and instructs the cleaning robot to increase its speed. The cleaning robot follows the instruction and cleans the floor at a faster pace than usual. When cleaning is complete, the server sends a notification to the user. The user can then instruct the robot on the next task.

[0869] Example prompts to input to a generative AI model:

[0870] "Please describe in detail a scenario in which a cleaning system for an office building requires a user to hurry up and complete the cleaning. Explain the process of using voice recognition to analyze the user's emotions and adjust the cleaning robot's speed based on those emotions."

[0871] This system allows users to efficiently perform cleaning tasks without specialized knowledge, and also optimizes operations based on the user's emotions, allowing managers to reduce costs and improve work efficiency.

[0872] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0873] Step 1:

[0874] The user uploads drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned from the system interface. The input is the drawing data file from the user, and the output is the transmission of the drawing data to the server. The user accesses the web interface, clicks the "Drawing Upload" button to open a file selection dialog, and selects the drawing data. The selected data is sent to the server and saved.

[0875] Step 2:

[0876] The server analyzes the drawing data received from the user. The input is the received drawing data (PDF, CAD, JPG, etc.), and the output is analyzed area information (room layout, hallway location, presence or absence of obstacles). The server analyzes the data using Adobe Acrobat API, Autodesk API, OpenCV, etc., and extracts the necessary information. For example, in the case of PDF data, the Adobe Acrobat API is used to check physical structure information, and in the case of CAD data, the Autodesk API is used to analyze room layout and structure.

[0877] Step 3:

[0878] The server calculates the optimal cleaning route based on the analyzed area information. The input is the analyzed area information, and the output is the optimal cleaning route (movement instructions in JSON format). This route calculation uses algorithm A, which derives the shortest route from the area information, taking into account the layout of rooms, the location of hallways, and the presence or absence of obstacles. Specifically, the data is sent to a route calculation module inside the server, which calculates the optimal route within the analyzed area.

[0879] Step 4:

[0880] The server sends the calculated cleaning route to the cleaning robot. The input is the calculated cleaning route, and the output is the route information received by the cleaning robot. The server converts the route information into JSON format and sends it by specifying the endpoint of the cleaning robot. REST API and MQTT protocol are used as communication protocols.

[0881] Step 5:

[0882] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The input is the cleaning route information sent from the server, and the output is the progress of the cleaning work. The cleaning robot uses LIDAR and camera modules to clean efficiently by following the specified route while avoiding obstacles in real time. The robot's progress and data are periodically reported to the server.

[0883] Step 6:

[0884] When the cleaning task is completed, the server sends a cleaning completion notification to the user. The input is the task completion report from the cleaning robot, and the output is the completion notification (email or in-app notification) to the user. The server generates a completion notification based on the completion report received from the cleaning robot and sends it to the user. For example, in the case of an email notification, the completion message is sent to the user's registered email address.

[0885] Step 7:

[0886] The server uses an emotion recognition engine to analyze the user's voice and image inputs and recognize emotions. The input is the user's voice data or image data, and the output is recognized emotional information. The server uses Google Cloud Speech-to-Text API or Microsoft Cognitive Services to convert the voice data into text and detect emotions based on that. Similarly, image analysis technology is used for facial expression recognition.

[0887] Step 8:

[0888] The server dynamically adjusts the behavior of the cleaning robot based on the recognized emotional information. The input is the recognized emotional information, and the output is the adjusted behavior parameters of the cleaning robot. For example, if the user says, "I want to finish cleaning quickly," the server recognizes the urgency and instructs the cleaning robot to increase its speed. This instruction is sent to the robot in real time, and the robot adjusts its behavior accordingly.

[0889] (Application example 2)

[0890] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0891] Efficient cleaning of large areas is an important task in many modern facilities and factories. However, conventional cleaning systems often rely on manual user control to set cleaning routes and adjust the cleaning robot's behavior, resulting in reduced work efficiency and the risk of human error. Furthermore, the robot's behavior cannot be flexibly adjusted according to the user's emotional state, making it difficult to adapt to situations requiring a rapid response or specific circumstances.

[0892] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, means for recognizing the user's emotion using an emotion recognition engine, and means for dynamically adjusting the operating speed and work route of the cleaning robot based on the recognized emotion. This enables flexible cleaning work to be performed in accordance with the user's emotional state and efficient cleaning work that does not rely on conventional manual operation.

[0893] "Drawing data" refers to data that contains layout information for facilities and areas, and includes different formats such as PDF, CAD, and JPG.

[0894] A "cleaning target area" is a specific area where the cleaning robot should work.

[0895] A "cleaning route" is the path that a cleaning robot should follow within the area to be cleaned.

[0896] A "cleaning robot" is a device that autonomously performs cleaning tasks along a designated route.

[0897] An "emotion recognition engine" is a system for recognizing emotions from a user's voice and image, using voice tone, facial expressions, language analysis, etc.

[0898] The "server" is a central processing unit that analyzes drawing data, calculates cleaning routes, sends instructions to cleaning robots, and recognizes emotions.

[0899] A "user" is a person who operates the system and issues instructions to the cleaning robot.

[0900] "Parsing logic" is an analytical method for analyzing drawing data and extracting necessary information.

[0901] The "operation speed" is the speed at which the cleaning robot moves along the cleaning route.

[0902] "Dynamic adjustment" refers to adaptively changing operating parameters based on real-time changing conditions.

[0903] This invention is an automated system for improving the efficiency of cleaning work in facilities and factories, and is particularly equipped with a function to recognize user emotions and adapt operations. This system analyzes drawing data, calculates cleaning routes, controls cleaning robots, recognizes user emotions using an emotion recognition engine, and adjusts operations.

[0904] The system program is configured as follows:

[0905] Receiving and analyzing drawing data

[0906] Users upload drawing data (e.g., PDF, CAD, JPG) showing the layout of a facility or factory to the server. The server receives this data, parses it, and extracts information about the area to be cleaned. The parsing logic used varies depending on the format of the drawing data.

[0907] Calculate and send cleaning routes

[0908] The server calculates the optimal cleaning route based on the extracted information and transmits it to the cleaning robot, taking into account factors such as the layout of rooms within the area to be cleaned, the location of hallways, and whether there are any obstacles.

[0909] Emotion Recognition and Behavior Regulation

[0910] The server has a built-in emotion recognition engine that analyzes the user's voice and image inputs to recognize emotions. The "speech_recognition" library is used for voice recognition technology, and the "OpenCV" library is used for facial expression recognition. The emotion recognition engine analyzes the user's voice tone, language analysis, and facial expressions to identify emotions. Based on the recognized emotions, the server dynamically adjusts the cleaning robot's operating speed and work route.

[0911] Hardware or software used

[0912] Server: Serves as the central processing unit, analyzing drawing data, calculating cleaning routes, controlling the cleaning robot, and performing emotion recognition.

[0913] Cleaning robot: An automated device that follows a designated route to perform cleaning tasks.

[0914] Emotion Recognition Engine: Software for analyzing voice tone and facial expressions. Libraries used include speech_recognition and OpenCV.

[0915] Specific examples

[0916] For example, when a user wants to clean a factory floor, they open a "cleaning app" on their smartphone and upload the factory layout data. Then, the user can voice-input "please hurry up and finish cleaning." The emotion recognition engine recognizes the urgency of this input. The server then responds by sending a command to speed up the cleaning robot's movements. Once the cleaning task is complete, the server notifies the user.

[0917] Example prompt sentence:

[0918] The user opens the "cleaning app" on their smartphone, uploads the layout data of the factory, and then says, "I'm busy, please hurry." The smartphone app analyzes this instruction using its emotion engine and sends a command to the cleaning robot to increase its speed.

[0919] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0920] Step 1:

[0921] The user opens the "cleaning app" on their smartphone and uploads the factory layout data (PDF, CAD, JPG, etc.). At this time, the user operates the app's interface to specify the drawing data. The input is the layout data file specified by the user through the app, and the output is that data sent to the server.

[0922] Step 2:

[0923] The server analyzes the drawing data received from the user. It applies parsing logic (analysis method) to the received drawing data and extracts layout information. It reads the drawing data in stages and analyzes information such as room layout, corridor location, and the presence or absence of obstacles. It receives drawing data as input and obtains extracted layout information as output.

[0924] Step 3:

[0925] The server calculates the optimal cleaning route based on the extracted layout information. The calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. A computer algorithm is used to determine the shortest path and most efficient cleaning route. The layout information is used as input, and the optimal cleaning route data is generated as output.

[0926] Step 4:

[0927] The server sends the calculated cleaning route to the cleaning robot. The cleaning route data is sent to the cleaning robot via the network. At this time, the server organizes the route information and converts it into a format that the cleaning robot can understand. The server receives the cleaning route data as input and sends this data to the robot as output.

[0928] Step 5:

[0929] The cleaning robot starts cleaning work based on the received cleaning route information. It follows the predetermined route and performs cleaning in each area. The robot's movement is controlled by a pre-programmed algorithm. It uses the cleaning route information as input and performs the actual cleaning work as output.

[0930] Step 6:

[0931] The server monitors the cleaning progress in real time and notifies the user. It evaluates the robot's progress based on the data received from sensors and cameras and notifies the user with progress updates. It receives status data from the cleaning robot as input and sends progress notifications to the user as output.

[0932] Step 7:

[0933] The server uses an emotion recognition engine to recognize the user's emotions. It uses the smartphone's microphone and camera to analyze voice and facial expressions to identify the user's emotional state. This analysis uses libraries such as speech_recognition and OpenCV. It receives the user's voice and image data as input and extracts emotional information as output.

[0934] Step 8:

[0935] The server dynamically adjusts the cleaning robot's operating speed and work route based on the recognized emotion information. For example, it increases the cleaning speed if the user is in a hurry, and decreases the speed if the user is instructed to clean slowly. It receives emotion information as input and sends instructions to adjust the robot's operating parameters as output.

[0936] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0937] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0938] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0939] [Fourth embodiment]

[0940] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0941] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0942] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0943] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0944] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0945] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0946] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0947] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0948] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0949] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0950] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0951] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0952] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0953] The present invention is an automated system for improving the efficiency of cleaning work in buildings and facilities. Specific embodiments of this system are described below.

[0954] System configuration

[0955] Receiving drawing data

[0956] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system. This can be easily done using a dedicated application or a web interface.

[0957] Drawing data analysis

[0958] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[0959] Cleaning route calculation

[0960] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[0961] Sending cleaning routes to the robot

[0962] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[0963] Cleaning by cleaning robot

[0964] The terminal (cleaning robot) automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[0965] Cleaning completion notification

[0966] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[0967] Specific examples

[0968] For example, consider a user who wants to clean the ninth floor of an office building. The user uploads a CAD file of the office floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[0969] The cleaning robot automatically cleans the ninth office floor according to the designated route. When cleaning is complete, the server sends a notification to the user, who can then instruct the robot to clean other floors as needed.

[0970] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and managers can reduce costs and improve work efficiency.

[0971] The processing flow will be explained below.

[0972] Step 1:

[0973] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[0974] Step 2:

[0975] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[0976] Step 3:

[0977] The server analyzes the received drawing data, applying parsing logic specific to the specific file format (PDF, CAD, JPG, etc.) to extract detailed layout information, including room layout, corridor location, and the presence or absence of obstacles.

[0978] Step 4:

[0979] The server calculates the optimal cleaning route based on the analyzed layout data, taking into account the layout of rooms, the location of hallways, and the location of obstacles, and sets the shortest route that allows the cleaning robot to move efficiently.

[0980] Step 5:

[0981] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start and end points and movement instructions for each segment.

[0982] Step 6:

[0983] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[0984] Step 7:

[0985] The server receives progress reports from the cleaning robot and uses that data to confirm completion of the cleaning task.

[0986] Step 8:

[0987] The server notifies the user that the cleaning is complete, and the user receives the notification and can then instruct the next step.

[0988] Example 1

[0989] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0990] Traditional cleaning work in buildings and facilities requires manual operation, resulting in problems of inefficiency and inaccuracy. In particular, when cleaning large areas or complex layouts, cleaning planning and execution are often inadequate, consuming a significant amount of time and effort. To solve these problems, a system is needed that automates cleaning work and cleans efficiently and accurately.

[0991] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0992] In this invention, the server includes a means for receiving drawing data from a user, a means for analyzing the received drawing data to extract information on the area to be cleaned, a means for automatically calculating a cleaning route based on the extracted information, and a means for transmitting the information to an automated machine. This allows a user to automatically calculate an optimal cleaning route without specialized knowledge and have the automated machine perform the cleaning work. By notifying the user of the progress of the cleaning work in real time, overall management efficiency can also be improved.

[0993] "Drawing data" is a digital file (PDF, CAD, JPG, etc.) that contains information showing the layout of a building or facility.

[0994] A "user" is a person or organization that operates the system to upload drawing data of the area to be cleaned and manages the execution of the cleaning.

[0995] The "analysis means" is a function that processes the drawing data received by the server and extracts information such as the layout of rooms, the location of corridors, and whether or not there are obstacles.

[0996] A "cleaning route" is a path along which a cleaning robot moves efficiently within an area to be cleaned.

[0997] An "automated machine" is a robot or cleaning device that automatically performs cleaning tasks based on a calculated cleaning route.

[0998] The "notification means" is a function that notifies the user of the progress and completion of the cleaning work.

[0999] An "analysis method" is a technique for analyzing data in different ways depending on the format of the drawing data.

[1000] An "obstacle" is an object or situation that impedes the robot's movement within the area to be cleaned.

[1001] The present invention relates to a system for streamlining and automating cleaning work in buildings and facilities. This system involves receiving and analyzing drawing data, calculating optimal cleaning routes, and issuing instructions to automated machines, aiming to improve the efficiency and management of cleaning work.

[1002] System configuration

[1003] Receiving drawing data

[1004] Users upload blueprints of the areas they wish to clean to the system. The blueprints can be in a variety of formats, including PDF, CAD, and JPG, and can be easily uploaded using a dedicated application or web interface. For example, they can use software or web applications such as "CleanMaster."

[1005] Drawing data analysis

[1006] The server analyzes the drawing data received from the user. Different analysis methods are used depending on the data format. For example, PDF Parser is used to analyze PDFs, and CADAnalyzer is used to analyze CAD data. The server uses this software to extract information such as room layout, aisle locations, and the presence or absence of obstacles.

[1007] Cleaning route calculation

[1008] The server calculates the optimal cleaning route based on the analyzed layout data. It takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles, and generates a route that includes specific movement instructions from the start point to the end point of the cleaning. Software called "Route Optimizer" is used to calculate the route.

[1009] Sending cleaning routes to automated machines

[1010] The server sends the calculated cleaning route to the automated machine (cleaning robot). Software called "RobotCommunicator" is used for communication. The information sent includes the location where cleaning should start, the route to follow, and the location where cleaning should end.

[1011] Carrying out cleaning tasks with automated machines

[1012] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with software called "AutoCleanPro" that efficiently performs work by following the specified route.

[1013] Cleaning completion notification

[1014] When the cleaning is complete, the server notifies the user using the "NotificationManager." The user can receive this notification, understand the progress of the cleaning work in real time, and issue instructions for the next work if necessary.

[1015] Specific examples

[1016] For example, if a user wants to clean the 9th floor of an office building, the system will operate as follows:

[1017] 1. A user uses the "CleanMaster" application to upload a CAD file of an office floor to the system.

[1018] 2. The server uses "CADAnalyzer" to analyze this file and extract the room layout and corridor structure.

[1019] 3. The server uses "Route Optimizer" to calculate the optimal cleaning route based on the analyzed information and sends this route information to the cleaning robot.

[1020] 4. The terminal (automatic machine) uses "AutoCleanPro" to automatically clean the 9th office floor according to the specified route.

[1021] 5. When the cleaning is complete, the server uses "NotificationManager" to send a notification to the user's smartphone saying "Cleaning of the 9th floor is complete."

[1022] Prompt Sentence Examples

[1023] "Please provide the specific processing steps for a system that calculates the optimal cleaning route for cleaning the ninth floor of an office building and sends the results to an automated machine. Please provide a detailed explanation, including the names of the specific hardware and software."

[1024] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and managers can reduce costs and improve work efficiency.

[1025] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1026] Step 1:

[1027] Users upload blueprint data of the area they wish to clean to the system. The blueprint data can be in a variety of formats, including PDF, CAD, and JPG, and can be uploaded using a dedicated application or web interface. Specifically, users open the "CleanMaster" application, click the blueprint data upload button, and select a file from their local device. The input is the blueprint data file selected by the user, which is sent to the server. The output is the blueprint data received by the server.

[1028] Step 2:

[1029] The server analyzes the drawing data received from the user. First, it identifies the data format and applies the appropriate analysis method. For example, it uses "PDFParser" for PDF and "CADAnalyzer" for CAD. From the analyzed data, it extracts information such as the layout of rooms, the location of aisles, and the presence or absence of obstacles. Specifically, the server reads the drawing data and begins the analysis process based on the identified format. The input is the drawing data received from the user, and the output is the analyzed layout information.

[1030] Step 3:

[1031] The server calculates the optimal cleaning route based on the analyzed layout data. The calculation uses "RouteOptimizer" and takes into account the layout of the rooms, the location of the aisles, and whether there are any obstacles. Specifically, the server takes the analyzed data as input and applies an algorithm to calculate the optimal route. The calculation results include the cleaning start point, movement path, and cleaning end point. The input is the analyzed layout data, and the output is the calculated cleaning route.

[1032] Step 4:

[1033] The server sends the calculated cleaning route to the automated machine. For communication, it uses "RobotCommunicator" to send specific cleaning instructions to the automated machine. Specifically, the server prepares cleaning route data and sends the data to the automated machine using a communication protocol. The input is the calculated cleaning route, and the output is the cleaning route information received by the automated machine.

[1034] Step 5:

[1035] The terminal (automated machine) automatically performs cleaning work based on the cleaning route information received from the server. The automatic machine is equipped with "AutoCleanPro" and cleans efficiently according to the specified route. Specifically, the automatic machine starts cleaning and cleans each area as it moves along the route. The input is the cleaning route information received from the server, and the output is the completion of cleaning.

[1036] Step 6:

[1037] When cleaning is complete, the server notifies the user that the work is complete. Notification is sent using "NotificationManager." Specifically, the server generates cleaning completion information and sends a notification to the user's device. The input is a cleaning completion report from the automatic machine, and the output is a cleaning completion notification to the user.

[1038] (Application example 1)

[1039] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1040] In conventional cleaning systems, analyzing drawing data and calculating cleaning routes are often done manually, making it difficult to work efficiently. Even when cleaning robots fully automate their work, there is no way for users to grasp the progress in real time, making it difficult to manage the progress of cleaning. Furthermore, cleaning instructions cannot be intuitively given using a smartphone, which makes it inconvenient.

[1041] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1042] In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to a cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for displaying the cleaning route and progress on a smartphone device and sending cleaning instructions, thereby enabling efficient real-time management of cleaning work and progress, and allowing the user to intuitively give cleaning instructions using their smartphone.

[1043] "Drawing data" is a data format that contains layout information for the area to be cleaned, and is available in formats such as PDF, CAD, and JPG.

[1044] A "user" is an individual or organization that uses the cleaning system to give cleaning instructions.

[1045] "Analyzing" means extracting necessary information from the received drawing data, such as the layout of rooms, the structure of passageways, and the presence or absence of obstacles.

[1046] The "area to be cleaned" refers to the area where cleaning work will be carried out, including the interior of the building, specific rooms, hallways, etc.

[1047] A "cleaning route" is a specific path from a cleaning start point to an end point that a cleaning robot should follow.

[1048] A "cleaning robot" is an autonomous mechanical device that automatically performs cleaning tasks according to a calculated cleaning route.

[1049] "Progress" refers to dynamic information such as the progress of the cleaning work and the current cleaning location.

[1050] A "smartphone device" is a portable information terminal that supports Internet connection and is easy for users to operate.

[1051] "Parsing logic" is an analysis algorithm that performs different processing depending on the format of the drawing data.

[1052] "Room layout" is information indicating the relative positions and sizes of each room within a building.

[1053] "Corridor location" is information indicating the location of a passageway within a building.

[1054] An "obstacle" is an object or structure that blocks the movement of the cleaning robot during cleaning work.

[1055] The present invention is an automated system for improving the efficiency of cleaning work in a factory. A specific embodiment of this system will be described below.

[1056] System configuration

[1057] The system consists of a user, a server, a cleaning robot, and a smartphone device.

[1058] Receiving drawing data

[1059] Users upload blueprint data (PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device, either through a dedicated application or a web interface.

[1060] Drawing data analysis

[1061] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[1062] Cleaning route calculation

[1063] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[1064] Display of cleaning route

[1065] The server sends the calculated cleaning route to the smartphone device, where the user can visually check the route in the application.

[1066] Send cleaning instructions

[1067] After the user confirms the cleaning route, they send cleaning instructions to the cleaning robot via smartphone, and the cleaning robot is then ready to start cleaning based on the route that matches the uploaded drawing data.

[1068] Cleaning by cleaning robot

[1069] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area.

[1070] Cleaning completion notification

[1071] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[1072] Hardware and software used

[1073] Hardware: smartphones, cleaning robots

[1074] Software: Flask, PIL (Python Imaging Library), pytesseract, dedicated smartphone application

[1075] Specific examples

[1076] As an example, consider the case where a user cleans the first floor of a factory. The user uploads a CAD file of the factory floor to the system using a smartphone. The server receives this file and analyzes it to extract the layout of rooms and the structure of corridors. The server then calculates the optimal cleaning route based on the extracted information and displays this route on the smartphone. The user checks the displayed route and sends cleaning instructions to the cleaning robot via the smartphone app. The cleaning robot automatically cleans the factory floor according to the specified route, and a notification is sent to the smartphone when cleaning is complete.

[1077] Prompt Sentence Examples

[1078] "We would like to clean the first floor of a factory. Please analyze the following drawing data and calculate the optimal cleaning route. Also, please be able to check the progress of the cleaning after it starts. Drawing data: 'factory_floor_plan.jpg'"

[1079] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1080] Step 1:

[1081] The user uploads drawing data (e.g., PDF, CAD, JPG, etc.) of the area to be cleaned to the system using a smartphone device. The uploaded drawing data is sent to the server. The input here is the drawing data, and the output is the transmission of the drawing data to the server.

[1082] Step 2:

[1083] The server parses the received drawing data. Different parsing logic is applied depending on the data format. The input is the drawing data received by the server, and the output is information about the parsed area (e.g., room layout, corridor location, presence or absence of obstacles).

[1084] Step 3:

[1085] The server calculates the optimal cleaning route based on the analyzed information. This calculation takes into account the layout of the rooms, the location of the hallways, and the presence or absence of obstacles. The input is the analyzed area information, and the output is the calculated cleaning route. Specifically, an optimization calculation is performed using an algorithm.

[1086] Step 4:

[1087] The server sends the calculated cleaning route to the smartphone device for visual confirmation by the user, where the input is the calculated cleaning route and the output is sending it to the smartphone device and displaying the route.

[1088] Step 5:

[1089] The user checks the displayed cleaning route through the smartphone application. After checking, the user sends cleaning instructions to the cleaning robot from the smartphone. The input is the user's confirmation and cleaning instructions, and the output is the cleaning instructions sent to the cleaning robot.

[1090] Step 6:

[1091] The cleaning robot automatically performs cleaning tasks based on the received cleaning route information. The robot follows the specified route and efficiently cleans each area. The input is the cleaning route information for the cleaning robot, and the output is the behavior of the cleaning robot and the progress of the cleaning task.

[1092] Step 7:

[1093] When cleaning is complete, the cleaning robot sends a notification of completion to the server. The server receives this notification and sends it to the user's smartphone. The input is the cleaning completion notification from the cleaning robot, and the output is the cleaning completion notification to the user.

[1094] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1095] The present invention provides an automated system for improving the efficiency of cleaning work in buildings and facilities, and further includes a function for recognizing the user's emotions and adapting operations accordingly. Specific embodiments of this system are described below.

[1096] System configuration

[1097] Receiving drawing data

[1098] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[1099] Drawing data analysis

[1100] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. From the analyzed data, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[1101] Cleaning route calculation

[1102] The server calculates the optimal cleaning route based on the analyzed layout data. This route calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. The calculated route includes specific movement instructions from the start point to the end point of the cleaning.

[1103] Sending cleaning routes to the robot

[1104] The server sends the calculated cleaning route to the cleaning robot, which is then ready to start cleaning based on the route that matches the uploaded drawing data.

[1105] Cleaning by cleaning robot

[1106] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot follows the specified route and cleans each area efficiently. The robot's operation is monitored in real time and its progress is reported to the server.

[1107] Cleaning completion notification

[1108] When the cleaning is complete, the server notifies the user, who can then follow the progress of the cleaning in real time and issue instructions for the next task if necessary.

[1109] Emotion recognition engine configuration

[1110] Emotion engine integration

[1111] The server has a built-in emotion recognition engine that analyzes the user's voice and image input to recognize the user's emotions. This emotion recognition is performed using technologies such as voice tone, language analysis, and facial expression recognition.

[1112] Adapting emotion-based operations

[1113] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on the emotional information. For example, if the user is in a hurry, the cleaning speed can be increased, but if the user is relaxed, the robot will continue cleaning at a standard speed.

[1114] Specific examples

[1115] As an example, consider a situation where a user wants to clean a floor of an office building and the system needs to recognize emotions from the user's voice. The user uploads a CAD file of the ninth floor to the system. The server receives this file, analyzes it, and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and sends this route to the cleaning robot.

[1116] If a user says, "I want to finish cleaning quickly," the emotion engine will recognize the urgency from the tone and content of the words and instruct the cleaning robot to speed up its operation. The cleaning robot will follow the instruction and clean the floor at a faster pace than usual. Once cleaning is complete, the server will notify the user that the cleaning is complete. The user can receive this notification and instruct the robot on the next task.

[1117] This system allows users to perform efficient cleaning work without requiring specialized knowledge, and performs optimal operations based on emotions, further enabling managers to reduce costs and improve work efficiency.

[1118] The processing flow will be explained below.

[1119] Step 1:

[1120] The user uploads the drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface, which sends the drawing data to the server.

[1121] Step 2:

[1122] The server receives the uploaded drawing data, which is first saved in storage and then prepared for analysis.

[1123] Step 3:

[1124] The server analyzes the received drawing data. Different parsing logic is used depending on the data format. For example, PDFs use parsing logic specific to PDFs, and CAD uses parsing logic specific to CADs. Once the analysis is complete, information such as the layout of rooms, the location of corridors, and the presence or absence of obstacles is extracted.

[1125] Step 4:

[1126] The server calculates the optimal cleaning route based on the analyzed layout data. This calculation is designed to allow the cleaning robot to move efficiently, setting the shortest path based on the layout of rooms, the location of hallways, and the presence or absence of obstacles.

[1127] Step 5:

[1128] The server sends the calculated cleaning route to the cleaning robot, which includes the cleaning start point, end point, and movement instructions for each segment.

[1129] Step 6:

[1130] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The robot moves along the specified route and cleans each area. The robot's operation is monitored in real time and its progress is reported to the server.

[1131] Step 7:

[1132] The server uses an emotion engine that analyzes the user's voice and image inputs to recognize the user's emotions, for example, if the user is in a hurry or frustrated.

[1133] Step 8:

[1134] The server dynamically adjusts the cleaning robot's behavior based on the recognized emotion information, such as increasing the cleaning speed if the user is in a hurry, or maintaining a standard speed if the user is relaxed.

[1135] Step 9:

[1136] When the cleaning is complete, the server notifies the user that the cleaning is complete. This notification is given in a way that reflects the user's emotions. For example, if the user is feeling stressed, an encouraging message may be added.

[1137] Step 10:

[1138] The user receives a notification when cleaning is complete and can give instructions for the next cleaning job to the system. The user can then upload the drawing data again to clean the next area.

[1139] Example 2

[1140] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1141] While conventional cleaning systems are capable of calculating efficient cleaning routes and avoiding obstacles, they do not adapt their behavior to the user's emotions. As a result, they are not able to respond flexibly to situations such as emergencies or when the user wants to relax. This results in issues such as not maximizing the efficiency of cleaning work and reducing user satisfaction.

[1142] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, and means for recognizing the user's emotions and adjusting the operation of the cleaning robot based on the information. This makes it possible to perform optimal cleaning work according to the user's emotions and situation.

[1143] "Drawing data" refers to digital files that represent the layout and structure of a building or facility, and includes formats such as PDF, CAD, and JPG.

[1144] The "means of receiving" is an interface for receiving input data from the user on the server side, such as a web form or API.

[1145] The "means of analysis" refers to the technology for digitally analyzing received drawing data, and uses various parsing logic such as Adobe Acrobat API, Autodesk API, and image recognition libraries.

[1146] "Means of extraction" refers to the technology used to extract useful information from the analyzed data, such as room layout, corridor location, and the presence of obstacles.

[1147] "Means for automatically calculating cleaning routes" refers to technology that calculates the optimal cleaning route based on the extracted information, and uses a pathfinding algorithm (e.g., A algorithm).

[1148] "Means for sending" refers to a communication means for transmitting the calculated cleaning route to the cleaning robot, and includes REST API and MQTT protocol.

[1149] A "cleaning robot" refers to a mechanical device that automatically performs cleaning tasks based on a programmed route.

[1150] "Means for executing" refers to the technology that enables the cleaning robot to automatically perform cleaning tasks according to a programmed route.

[1151] "Means of notification" refers to technologies used to notify users of the progress and completion status of cleaning work, including email and in-app notifications.

[1152] "Means for recognizing emotions" refers to technology that analyzes the user's voice and image input and identifies emotions, and uses voice recognition APIs and facial expression recognition technology.

[1153] "Means for adjusting behavior" refers to technology that varies the speed and performance of the cleaning robot's movements based on recognized emotional information.

[1154] The present invention provides an automated system that improves the efficiency of cleaning work in buildings and facilities, and further has the function of recognizing the user's emotions and adapting operations accordingly. A specific embodiment of this system is described below.

[1155] Receiving drawing data

[1156] Users upload drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned using the system interface. Drawing data uploading is done through a dedicated web interface, and users can use a file selection dialog by clicking the "Drawing Upload" button. Drawing data uploaded by users is sent to the server.

[1157] Drawing data analysis

[1158] The server analyzes the drawing data received from the user. Different parsing logic is used depending on the data format. For example, PDF data is analyzed using Adobe Acrobat API, CAD data using Autodesk API, and JPG data using image recognition libraries such as OpenCV. Important information such as room layout, corridor locations, and the presence or absence of obstacles is extracted from the analyzed data.

[1159] Cleaning route calculation

[1160] The server calculates the optimal cleaning route based on the analyzed layout data. This calculation uses a pathfinding algorithm, specifically the A algorithm. The cleaning route calculation takes into account the layout of rooms, the location of corridors, and the presence or absence of obstacles. The server sends the analyzed data to the route calculation module, which obtains the optimal route.

[1161] Sending cleaning routes to the robot

[1162] The server sends the calculated cleaning route to the cleaning robot. This communication uses REST API and MQTT protocol. The server converts the route information into JSON format and sends it to the cleaning robot's endpoint.

[1163] Cleaning by cleaning robot

[1164] The terminal (cleaning robot) starts cleaning based on the received cleaning route information. The robot uses LIDAR and camera modules to avoid obstacles in real time and cleans efficiently by following the specified route.

[1165] Cleaning completion notification

[1166] When the cleaning is complete, the server notifies the user that the cleaning work is complete. This notification can be sent via email or in-app notification, for example, by generating a message informing the user that the document has been completed and sending it to the user's registered email address.

[1167] Emotion recognition engine configuration

[1168] The server is equipped with an emotion recognition engine that analyzes voice and image input from the user to recognize the user's emotions. Emotion recognition uses Google Cloud Speech-to-Text API and Microsoft Cognitive Services. Voice data is acquired in real time, converted into text, and analyzed for emotion. Similarly, image analysis technology is used to recognize facial expressions using image data.

[1169] Adapting emotion-based operations

[1170] Once the user's emotions are recognized, the server dynamically adjusts the cleaning robot's behavior based on that emotional information. For example, if the user wants to finish cleaning quickly, the server instructs the cleaning robot to increase its speed. If the user is relaxed, the robot continues cleaning at a normal speed. The server sets the cleaning robot's operating parameters based on the data obtained from the emotion recognition engine and transmits the settings to the robot in real time.

[1171] Specific examples

[1172] As a concrete example, consider a user cleaning a floor of an office building. The user uploads a CAD file of the ninth floor to the system. The server analyzes this file and extracts the layout of the rooms and the structure of the corridors. The server then calculates the optimal cleaning route based on the extracted information and transmits this route to the cleaning robot.

[1173] If the user says, "I want to hurry up and finish cleaning," the server uses the Google Cloud Speech-to-Text API to recognize emotions from the tone and content of the words and instructs the cleaning robot to increase its speed. The cleaning robot follows the instruction and cleans the floor at a faster pace than usual. When cleaning is complete, the server sends a notification to the user. The user can then instruct the robot on the next task.

[1174] Example prompts to input to a generative AI model:

[1175] "Please describe in detail a scenario in which a cleaning system for an office building requires a user to hurry up and complete the cleaning. Explain the process of using voice recognition to analyze the user's emotions and adjust the cleaning robot's speed based on those emotions."

[1176] This system allows users to efficiently perform cleaning tasks without specialized knowledge, and also optimizes operations based on the user's emotions, allowing managers to reduce costs and improve work efficiency.

[1177] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1178] Step 1:

[1179] The user uploads drawing data (PDF, CAD, JPG, etc.) of the area to be cleaned from the system interface. The input is the drawing data file from the user, and the output is the transmission of the drawing data to the server. The user accesses the web interface, clicks the "Drawing Upload" button to open a file selection dialog, and selects the drawing data. The selected data is sent to the server and saved.

[1180] Step 2:

[1181] The server analyzes the drawing data received from the user. The input is the received drawing data (PDF, CAD, JPG, etc.), and the output is analyzed area information (room layout, hallway location, presence or absence of obstacles). The server analyzes the data using Adobe Acrobat API, Autodesk API, OpenCV, etc., and extracts the necessary information. For example, in the case of PDF data, the Adobe Acrobat API is used to check physical structure information, and in the case of CAD data, the Autodesk API is used to analyze room layout and structure.

[1182] Step 3:

[1183] The server calculates the optimal cleaning route based on the analyzed area information. The input is the analyzed area information, and the output is the optimal cleaning route (movement instructions in JSON format). This route calculation uses algorithm A, which derives the shortest route from the area information, taking into account the layout of rooms, the location of hallways, and the presence or absence of obstacles. Specifically, the data is sent to a route calculation module inside the server, which calculates the optimal route within the analyzed area.

[1184] Step 4:

[1185] The server sends the calculated cleaning route to the cleaning robot. The input is the calculated cleaning route, and the output is the route information received by the cleaning robot. The server converts the route information into JSON format and sends it by specifying the endpoint of the cleaning robot. REST API and MQTT protocol are used as communication protocols.

[1186] Step 5:

[1187] The terminal (cleaning robot) starts cleaning work based on the received cleaning route information. The input is the cleaning route information sent from the server, and the output is the progress of the cleaning work. The cleaning robot uses LIDAR and camera modules to clean efficiently by following the specified route while avoiding obstacles in real time. The robot's progress and data are periodically reported to the server.

[1188] Step 6:

[1189] When the cleaning task is completed, the server sends a cleaning completion notification to the user. The input is the task completion report from the cleaning robot, and the output is the completion notification (email or in-app notification) to the user. The server generates a completion notification based on the completion report received from the cleaning robot and sends it to the user. For example, in the case of an email notification, the completion message is sent to the user's registered email address.

[1190] Step 7:

[1191] The server uses an emotion recognition engine to analyze the user's voice and image inputs and recognize emotions. The input is the user's voice data or image data, and the output is recognized emotional information. The server uses Google Cloud Speech-to-Text API or Microsoft Cognitive Services to convert the voice data into text and detect emotions based on that. Similarly, image analysis technology is used for facial expression recognition.

[1192] Step 8:

[1193] The server dynamically adjusts the behavior of the cleaning robot based on the recognized emotional information. The input is the recognized emotional information, and the output is the adjusted behavior parameters of the cleaning robot. For example, if the user says, "I want to finish cleaning quickly," the server recognizes the urgency and instructs the cleaning robot to increase its speed. This instruction is sent to the robot in real time, and the robot adjusts its behavior accordingly.

[1194] (Application example 2)

[1195] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1196] Efficient cleaning of large areas is an important task in many modern facilities and factories. However, conventional cleaning systems often rely on manual user control to set cleaning routes and adjust the cleaning robot's behavior, resulting in reduced work efficiency and the risk of human error. Furthermore, the robot's behavior cannot be flexibly adjusted according to the user's emotional state, making it difficult to adapt to situations requiring a rapid response or specific circumstances.

[1197] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving drawing data from a user, means for analyzing the received drawing data and extracting information about the area to be cleaned, means for automatically calculating a cleaning route based on the extracted information, means for transmitting the calculated cleaning route to the cleaning robot, means for the cleaning robot to perform cleaning work according to the route, means for notifying the user of the progress of the cleaning work, means for recognizing the user's emotion using an emotion recognition engine, and means for dynamically adjusting the operating speed and work route of the cleaning robot based on the recognized emotion. This enables flexible cleaning work to be performed in accordance with the user's emotional state and efficient cleaning work that does not rely on conventional manual operation.

[1198] "Drawing data" refers to data that contains layout information for facilities and areas, and includes different formats such as PDF, CAD, and JPG.

[1199] A "cleaning target area" is a specific area where the cleaning robot should work.

[1200] A "cleaning route" is the path that a cleaning robot should follow within the area to be cleaned.

[1201] A "cleaning robot" is a device that autonomously performs cleaning tasks along a designated route.

[1202] An "emotion recognition engine" is a system for recognizing emotions from a user's voice and image, using voice tone, facial expressions, language analysis, etc.

[1203] The "server" is a central processing unit that analyzes drawing data, calculates cleaning routes, sends instructions to cleaning robots, and recognizes emotions.

[1204] A "user" is a person who operates the system and issues instructions to the cleaning robot.

[1205] "Parsing logic" is an analytical method for analyzing drawing data and extracting necessary information.

[1206] The "operation speed" is the speed at which the cleaning robot moves along the cleaning route.

[1207] "Dynamic adjustment" refers to adaptively changing operating parameters based on real-time changing conditions.

[1208] This invention is an automated system for improving the efficiency of cleaning work in facilities and factories, and is particularly equipped with a function to recognize user emotions and adapt operations. This system analyzes drawing data, calculates cleaning routes, controls cleaning robots, recognizes user emotions using an emotion recognition engine, and adjusts operations.

[1209] The system program is configured as follows:

[1210] Receiving and analyzing drawing data

[1211] Users upload drawing data (e.g., PDF, CAD, JPG) showing the layout of a facility or factory to the server. The server receives this data, parses it, and extracts information about the area to be cleaned. The parsing logic used varies depending on the format of the drawing data.

[1212] Calculate and send cleaning routes

[1213] The server calculates the optimal cleaning route based on the extracted information and transmits it to the cleaning robot, taking into account factors such as the layout of rooms within the area to be cleaned, the location of hallways, and whether there are any obstacles.

[1214] Emotion Recognition and Behavior Regulation

[1215] The server has a built-in emotion recognition engine that analyzes the user's voice and image inputs to recognize emotions. The "speech_recognition" library is used for voice recognition technology, and the "OpenCV" library is used for facial expression recognition. The emotion recognition engine analyzes the user's voice tone, language analysis, and facial expressions to identify emotions. Based on the recognized emotions, the server dynamically adjusts the cleaning robot's operating speed and work route.

[1216] Hardware or software used

[1217] Server: Serves as the central processing unit, analyzing drawing data, calculating cleaning routes, controlling the cleaning robot, and performing emotion recognition.

[1218] Cleaning robot: An automated device that follows a designated route to perform cleaning tasks.

[1219] Emotion Recognition Engine: Software for analyzing voice tone and facial expressions. Libraries used include speech_recognition and OpenCV.

[1220] Specific examples

[1221] For example, when a user wants to clean a factory floor, they open a "cleaning app" on their smartphone and upload the factory layout data. Then, the user can voice-input "please hurry up and finish cleaning." The emotion recognition engine recognizes the urgency of this input. The server then responds by sending a command to speed up the cleaning robot's movements. Once the cleaning task is complete, the server notifies the user.

[1222] Example prompt sentence:

[1223] The user opens the "cleaning app" on their smartphone, uploads the layout data of the factory, and then says, "I'm busy, please hurry." The smartphone app analyzes this instruction using its emotion engine and sends a command to the cleaning robot to increase its speed.

[1224] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1225] Step 1:

[1226] The user opens the "cleaning app" on their smartphone and uploads the factory layout data (PDF, CAD, JPG, etc.). At this time, the user operates the app's interface to specify the drawing data. The input is the layout data file specified by the user through the app, and the output is that data sent to the server.

[1227] Step 2:

[1228] The server analyzes the drawing data received from the user. It applies parsing logic (analysis method) to the received drawing data and extracts layout information. It reads the drawing data in stages and analyzes information such as room layout, corridor location, and the presence or absence of obstacles. It receives drawing data as input and obtains extracted layout information as output.

[1229] Step 3:

[1230] The server calculates the optimal cleaning route based on the extracted layout information. The calculation takes into account the layout of rooms, the location of hallways, and the presence of obstacles. A computer algorithm is used to determine the shortest path and most efficient cleaning route. The layout information is used as input, and the optimal cleaning route data is generated as output.

[1231] Step 4:

[1232] The server sends the calculated cleaning route to the cleaning robot. The cleaning route data is sent to the cleaning robot via the network. At this time, the server organizes the route information and converts it into a format that the cleaning robot can understand. The server receives the cleaning route data as input and sends this data to the robot as output.

[1233] Step 5:

[1234] The cleaning robot starts cleaning work based on the received cleaning route information. It follows the predetermined route and performs cleaning in each area. The robot's movement is controlled by a pre-programmed algorithm. It uses the cleaning route information as input and performs the actual cleaning work as output.

[1235] Step 6:

[1236] The server monitors the cleaning progress in real time and notifies the user. It evaluates the robot's progress based on the data received from sensors and cameras and notifies the user with progress updates. It receives status data from the cleaning robot as input and sends progress notifications to the user as output.

[1237] Step 7:

[1238] The server uses an emotion recognition engine to recognize the user's emotions. It uses the smartphone's microphone and camera to analyze voice and facial expressions to identify the user's emotional state. This analysis uses libraries such as speech_recognition and OpenCV. It receives the user's voice and image data as input and extracts emotional information as output.

[1239] Step 8:

[1240] The server dynamically adjusts the cleaning robot's operating speed and work route based on the recognized emotion information. For example, it increases the cleaning speed if the user is in a hurry, and decreases the speed if the user is instructed to clean slowly. It receives emotion information as input and sends instructions to adjust the robot's operating parameters as output.

[1241] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1242] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1243] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1244] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1245] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1246] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1247] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1248] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1249] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1250] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1251] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1252] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1253] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1254] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1255] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1256] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1257] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1258] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1259] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1260] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1261] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1262] The following is further disclosed regarding the above embodiment.

[1263] (Claim 1)

[1264] means for receiving drawing data from a user;

[1265] A means for analyzing the received drawing data and extracting information on the area to be cleaned;

[1266] means for automatically calculating a cleaning route based on the extracted information;

[1267] means for transmitting the calculated cleaning route to the cleaning robot;

[1268] A means for the cleaning robot to perform cleaning work according to a route;

[1269] means for notifying the user of the progress of the cleaning operation;

[1270] A system including:

[1271] (Claim 2)

[1272] 10. The system of claim 1, further comprising means for applying different parsing logic depending on the format of the drawing data.

[1273] (Claim 3)

[1274] 10. The system of claim 1, including means for taking into account room layout, hallway location, and the presence of obstacles when calculating an optimal cleaning route.

[1275] "Example 1"

[1276] (Claim 1)

[1277] means for receiving drawing data from a user;

[1278] A means for analyzing the received drawing data and extracting information on the area to be cleaned;

[1279] means for automatically calculating a cleaning route based on the extracted information;

[1280] means for transmitting the calculated cleaning route to the automated machine;

[1281] means for the automated machine to perform cleaning operations according to a route;

[1282] means for notifying the user of the progress of the cleaning operation;

[1283] A system including:

[1284] (Claim 2)

[1285] 10. The system of claim 1, further comprising means for applying different analysis techniques depending on the format of the drawing data.

[1286] (Claim 3)

[1287] 10. The system of claim 1, including means for taking into account room layout, aisle location, and presence of obstructions when calculating an optimal cleaning route.

[1288] "Application Example 1"

[1289] (Claim 1)

[1290] means for receiving drawing data from a user;

[1291] A means for analyzing the received drawing data and extracting information on the area to be cleaned;

[1292] means for automatically calculating a cleaning route based on the extracted information;

[1293] means for transmitting the calculated cleaning route to the cleaning robot;

[1294] A means for the cleaning robot to perform cleaning work according to a route;

[1295] means for notifying the user of the progress of the cleaning operation;

[1296] A means for displaying cleaning routes and progress and sending cleaning instructions to a smartphone device;

[1297] A system including:

[1298] (Claim 2)

[1299] 10. The system of claim 1, further comprising means for applying different parsing logic depending on the format of the drawing data.

[1300] (Claim 3)

[1301] 10. The system of claim 1, including means for taking into account room layout, hallway location, and the presence of obstacles when calculating an optimal cleaning route.

[1302] "Example 2: Combining Emotion Engines"

[1303] (Claim 1)

[1304] means for receiving drawing data from a user;

[1305] A means for analyzing the received drawing data and extracting information on the area to be cleaned;

[1306] means for automatically calculating a cleaning route based on the extracted information;

[1307] means for transmitting the calculated cleaning route to the cleaning robot;

[1308] A means for the cleaning robot to perform cleaning work according to a route;

[1309] means for notifying the user of the progress of the cleaning operation;

[1310] means for recognizing a user's emotion and adjusting the behavior of the cleaning robot based on that information;

[1311] A system including:

[1312] (Claim 2)

[1313] 10. The system of claim 1, further comprising means for applying different parsing logic depending on the format of the drawing data.

[1314] (Claim 3)

[1315] 10. The system of claim 1, including means for taking into account room layout, hallway location, and the presence of obstacles when calculating an optimal cleaning route.

[1316] (Claim 4)

[1317] 10. The system of claim 1, further comprising means for analyzing a user's voice and / or visual input to recognize emotions.

[1318] (Claim 5)

[1319] 10. The system of claim 1, further comprising means for adjusting an operating speed of the cleaning robot based on the recognized user emotion.

[1320] "Application example 2 when combining emotion engines"

[1321] (Claim 1)

[1322] means for receiving drawing data from a user;

[1323] A means for analyzing the received drawing data and extracting information on the area to be cleaned;

[1324] means for automatically calculating a cleaning route based on the extracted information;

[1325] means for transmitting the calculated cleaning route to the cleaning robot;

[1326] A means for the cleaning robot to perform cleaning work according to a route;

[1327] means for notifying the user of the progress of the cleaning operation;

[1328] means for recognizing a user's emotion using an emotion recognition engine;

[1329] A means for dynamically adjusting the operating speed and working route of the cleaning robot based on the recognized emotion;

[1330] A system including:

[1331] (Claim 2)

[1332] 10. The system of claim 1, further comprising means for applying different parsing logic depending on the format of the drawing data.

[1333] (Claim 3)

[1334] 10. The system of claim 1, including means for taking into account room layout, hallway location, and the presence of obstacles when calculating an optimal cleaning route. [Explanation of symbols]

[1335] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving drawing data from a user; A means for analyzing the received drawing data and extracting information on the area to be cleaned; means for automatically calculating a cleaning route based on the extracted information; means for transmitting the calculated cleaning route to the cleaning robot; A means for the cleaning robot to perform cleaning work according to a route; means for notifying the user of the progress of the cleaning operation; A system including:

2. 2. The system of claim 1, further comprising means for applying different parsing logic depending on the format of the drawing data.

3. 10. The system of claim 1, further comprising means for taking into account room layout, hallway location, and the presence of obstacles when calculating an optimal cleaning route.

Images