system

Abstract

Provide a system. A means for receiving drawing data as input and automatically generating a cleaning route; A means for analyzing the drawing data and identifying cleaning areas and obstacles; a means for presenting the automatically generated cleaning route to a user and allowing the user to adjust the route; means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Cleaning work in modern buildings and facilities is labor-intensive and poses significant challenges in terms of efficiency. In particular, in facilities with large floor areas or complex layouts, designing and implementing cleaning routes manually is difficult and often consumes time and effort. Furthermore, if cleaning staff lack specialized knowledge, it is difficult to optimize cleaning routes and achieve efficient work. This results in increased costs for managers and reduced work efficiency. [Means for solving the problem]

[0005] To address the above-mentioned issues, the present invention provides a system that receives drawing data as input and automatically generates a cleaning route. Specifically, the system includes a means for analyzing the drawing data and identifying cleaning areas and obstacles, thereby automatically generating an efficient cleaning route. The system also includes a means for presenting the automatically generated cleaning route to a user, allowing the user to adjust it using drag-and-drop operations. Furthermore, the system includes a means for transmitting the final cleaning route to a cleaning robot and starting cleaning work, thereby enabling the system to set an effective cleaning route and improve work efficiency without requiring specialized knowledge.

[0006] "Drawing data" refers to digital data that includes layout information for buildings and facilities, and includes formats such as PDF, CAD, and images.

[0007] "Input format" refers to the data format accepted by the system, including PDF, CAD, and image formats.

[0008] "Cleaning route" refers to the trajectory or path information that the cleaning robot should follow, including a route that is optimized taking into account the cleaning area and obstacles.

[0009] "Means for automatic generation" refers to algorithms or programs that analyze and generate cleaning routes from drawing data.

[0010] "Analysis" refers to the process of extracting necessary information from drawing data and identifying cleaning areas and obstacles.

[0011] "Cleaning area" refers to the location or area where cleaning is actually to take place.

[0012] "Obstacles" refer to objects or areas that the cleaning robot must avoid, such as furniture, pillars, and equipment.

[0013] "User" refers to the entity that operates the system and sets and adjusts the cleaning route.

[0014] "Presenting" refers to displaying system-generated information to the user.

[0015] "Adjustable" refers to the ability of the user to make changes and fine-tune the cleaning route presented to the system.

[0016] The term "drag-and-drop operation" refers to an operation method in which a user moves an object on the screen using an input device.

[0017] The "final cleaning route" refers to the final route information that has been adjusted by the user and is actually sent to the cleaning robot.

[0018] A "cleaning robot" refers to a mechanical device that automatically performs cleaning tasks and operates according to a cleaning route that has been transmitted to it.

[0019] "Transmission" refers to the system sending the generated or adjusted cleaning route as data to the cleaning robot.

[0020] "Start" refers to the cleaning robot starting to perform cleaning work according to the received cleaning route. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention is a system for streamlining cleaning work within buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. This system is particularly effective in facilities with large floors or complex layouts.

[0043] System configuration

[0044] 1. Inputting drawing data

[0045] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[0046] 2. Analysis of drawing data

[0047] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[0048] 3. Cleaning route generation

[0049] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[0050] 4. Present and coordinate cleaning routes

[0051] The generated cleaning route is presented to the user, who can check it on their device. If necessary, they can fine-tune the cleaning route using drag-and-drop operations. In this way, the user can make final adjustments to the route to suit the actual cleaning situation.

[0052] 5. Send the cleaning route and start cleaning

[0053] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route. This reduces the workload of cleaning staff and improves work efficiency.

[0054] Program processing

[0055] The program processing of this system will be explained in natural language below.

[0056] Uploading drawing data

[0057] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[0058] Drawing data analysis

[0059] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[0060] Cleaning route generation

[0061] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[0062] Present and coordinate cleaning routes

[0063] The user can view the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the user can adjust it using drag and drop. This allows the user to exclude areas that do not need cleaning or set specific areas for additional cleaning.

[0064] Cleaning work begins

[0065] The server sends the final cleaning route to the cleaning robot, which then starts cleaning based on this route. The robot follows the received route and efficiently cleans within the specified area. For example, the robot can clean the hallway and each room accurately.

[0066] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[0070] Step 2:

[0071] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and converts it into the appropriate format for use in the next analysis step.

[0072] Step 3:

[0073] The server begins analyzing the drawing data. Specifically, in the case of PDF or CAD data, it reads the drawing's layer information and identifies area information (rooms and corridors) and obstacles (furniture and pillars). In the case of image data, it uses image analysis algorithms to extract similar area information.

[0074] Step 4:

[0075] Based on the analysis results, the server automatically generates an optimal cleaning route that takes into account the cleaning area and obstacles. Specifically, it calculates a route that passes through each area only once and minimizes unnecessary movement.

[0076] Step 5:

[0077] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[0078] Step 6:

[0079] Users can fine-tune the proposed cleaning route by dragging and dropping. Users can exclude unnecessary areas or adjust to clean specific areas. Once adjustments are complete, users click the confirm button.

[0080] Step 7:

[0081] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[0082] Step 8:

[0083] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[0084] Step 9:

[0085] The cleaning robot will then start cleaning according to the received cleaning route. The cleaning robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] Conventional cleaning work requires a lot of time and effort, and it is difficult to efficiently set cleaning routes, especially in facilities with large floors or complex layouts. Furthermore, manual cleaning route setting by cleaning staff lacks flexibility and often results in inconsistent work quality. To solve these problems, a system was needed that could efficiently analyze drawing data and automatically generate and adjust cleaning routes.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for sending the final cleaning route to a cleaning device and starting cleaning work, means for sending a file to the server and displaying the upload progress, means for analyzing the drawing data based on layer information and coordinate data, means for converting the cleaning route into JSON format and sending it to the user's terminal, means for displaying a route preview and allowing adjustment by drag and drop, and means for sending the generated optimal route to the API endpoint of the cleaning device. This allows users to easily operate the system, enabling efficient cleaning route setting and automation of actual cleaning work.

[0091] "Drawing data" refers to digital data such as PDFs, blueprints, and image formats that contain layout information for buildings and facilities.

[0092] "Cleaning route" refers to the path that a cleaning device follows within the area to be cleaned, including a path optimized for efficient cleaning.

[0093] "Cleaning area" refers to the specific area, such as a room, hallway, or corridor, where cleaning work is performed.

[0094] "Obstacles" refers to objects such as furniture or pillars that impede the movement of cleaning equipment.

[0095] "Cleaning equipment" refers to robots and devices that perform cleaning tasks automatically.

[0096] "User" refers to the person who operates the system to set and adjust cleaning routes.

[0097] "Terminal" refers to an information device such as a computer or tablet that is operated by a user.

[0098] "Server" refers to a central computer system that performs processes such as analyzing drawing data and automatically generating cleaning routes.

[0099] "Upload" refers to the operation of sending data from a terminal to a server.

[0100] "Analysis" refers to the process of extracting necessary information from drawing data and identifying cleaning areas and obstacles.

[0101] "Drag and drop" refers to an operation in which a user moves an object on the screen using a mouse or touch operation.

[0102] "JSON format" refers to a lightweight data exchange format for structuring and storing data.

[0103] "API Endpoint" refers to a specific interface for communicating with a cleaning device.

[0104] "Preview" refers to the state in which the generated cleaning route can be visually displayed and confirmed.

[0105] The present invention is a system for improving the efficiency of cleaning work within buildings and facilities, receiving drawing data as input and automatically generating cleaning routes. This system is particularly effective in facilities with large floor areas or complex layouts. Specific embodiments of this system are described below.

[0106] The system consists of three main components: a server, a terminal, and cleaning equipment. Drawing data is provided in PDF, blueprint, image format, etc. The server uses analysis tools such as Adobe Acrobat SDK to extract the necessary information from the drawing data.

[0107] Uploading drawing data

[0108] The user uses the device to upload the drawing data of the area to be cleaned to the server. Specifically, the user opens the file selection window on the device, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The device sends the selected file to the server and displays the upload progress on the screen.

[0109] Drawing data analysis

[0110] The server analyzes the received drawing data and identifies the locations of rooms, passageways, and obstacles. The analysis uses Adobe Acrobat SDK and other tools, making full use of the PDF file's layer information and image processing technology to extract the necessary data. For example, room boundaries, passageway widths, and the locations of furniture and pillars can be identified.

[0111] Cleaning route generation

[0112] Based on the analysis results, the server automatically generates an optimal cleaning route. The server uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The result is converted into JSON format and sent to the user's device.

[0113] Present and coordinate cleaning routes

[0114] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," visually displays a preview of the route and allows the user to fine-tune the route using drag-and-drop operations. Users can exclude areas that do not require cleaning and set additional areas to be cleaned.

[0115] Cleaning work begins

[0116] The server sends the finalized cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts cleaning work based on the received route information. The cleaning device moves efficiently and cleans within the specified area.

[0117] Examples of specific examples and prompts

[0118] A specific example of this system is shown below.

[0119] Example of uploading drawing data

[0120] The user selects the PDF file of their office floor on the terminal and clicks the "Upload" button.

[0121] Specific examples of drawing data analysis

[0122] The server receives "Office Floor.pdf" and uses software called "PDF Analysis Tool" to analyze the PDF layers and identify the locations of rooms, corridors, and obstacles to be cleaned.

[0123] Example of cleaning route generation

[0124] The server uses an algorithm to calculate the shortest route from "Room A" to "Room B" and generates a route.

[0125] Examples of cleaning route presentation and adjustment

[0126] The user checks the route presented in the application on the device and removes the unnecessary room, "C�D Room," from the route.

[0127] Specific examples of when cleaning work begins

[0128] The server then sends the final route to the "cleaning robot," which then begins cleaning the designated area, efficiently cleaning without hitting any obstacles along the way.

[0129] Example prompts for generative AI models

[0130] "I've uploaded an office floor plan (PDF file). Please explain how the system generates optimal cleaning routes based on this floor plan and allows me to adjust the routes with drag and drop."

[0131] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0133] Step 1: Upload the drawing data

[0134] The user uses the terminal to upload the drawing data to the server. Specifically, the user opens the file selection window on the terminal, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The input here is the PDF file selected by the user, and that file is sent to the server. The terminal splits the file and sends it to the server, and displays the upload progress.

[0135] Input: PDF file

[0136] Output: Drawing data uploaded to the server

[0137] Specific behavior:

[0138] The user selects "Office Floor.pdf."

[0139] The device splits the file into 10MB chunks and sends them to the server.

[0140] Displays an upload progress bar on the device screen.

[0141] Step 2: Analyze the drawing data

[0142] The server analyzes the received drawing data. During this process, the server uses analysis tools such as Adobe Acrobat SDK to extract the contents of the PDF file. Based on the layer information and coordinate data of the drawing data, the server identifies the locations of rooms, corridors, and obstacles.

[0143] Input: Drawing data uploaded to the server

[0144] Output: Analyzed cleaning area and obstacle information

[0145] Specific behavior:

[0146] The server opens "OfficeFloor.pdf".

[0147] The server scans the layer information to detect room boundaries.

[0148] The width of the aisle is measured and the position of obstacles (desks, chairs, etc.) is recorded as coordinate data.

[0149] Step 3: Generate cleaning routes

[0150] The server generates an optimal cleaning route based on the analysis results. Specifically, it uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The calculation results are converted into JSON format and sent to the user's device.

[0151] Input: Analyzed cleaning area and obstacle information

[0152] Output: Cleaning route information in JSON format

[0153] Specific behavior:

[0154] The server calculates a route that passes through each room once.

[0155] Convert the calculation results into JSON format.

[0156] The server logs the computation time and the optimality of the route.

[0157] Step 4: Present and coordinate cleaning routes

[0158] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," provides a visual preview of the route and allows users to fine-tune the route using drag-and-drop operations.

[0159] Input: Cleaning route information in JSON format

[0160] Output: The final cleaning route as adjusted by the user

[0161] Specific behavior:

[0162] The device receives route information in JSON format and displays it visually in a GUI.

[0163] The user clicks on "CD Room" to exclude it from the route.

[0164] After making adjustments, the user clicks the "Confirm" button to save the final route.

[0165] Step 5: Start cleaning

[0166] The server sends the final cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts work based on the received route information and efficiently cleans within the specified area.

[0167] Input: Final cleaning route adjusted by the user

[0168] Output: The cleaning task performed by the cleaning equipment

[0169] Specific behavior:

[0170] The server sends the route data to the cleaning equipment's control API.

[0171] The cleaning equipment analyzes the route data and generates a cleaning schedule.

[0172] The cleaning equipment begins cleaning within the designated area.

[0173] (Application example 1)

[0174] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0175] In today's vast and complex facilities and factories, manual cleaning is not only laborious and time-consuming, but also inefficient, resulting in a high likelihood of a lack of consistency across the entire cleaning area. Furthermore, designing and adjusting cleaning routes requires specialized knowledge, making it difficult for average employees. Therefore, a system that can automatically generate and easily adjust efficient and unified cleaning routes is desired.

[0176] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0177] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to a cleaning machine and starting cleaning work, means for identifying the location of work areas and equipment within a large-scale facility when analyzing the uploaded drawing data, means for displaying a preview of the cleaning route on a smart device and allowing the user to fine-tune the route by dragging and dropping, and means for confirming the cleaning route and transmitting the confirmed route to the cleaning machine, thereby making it possible to improve the efficiency and automation of cleaning work and provide user-friendly adjustment functions.

[0178] "Drawing data" refers to digital data that includes layout information for buildings and facilities, and is expressed in PDF, CAD, or image format.

[0179] "Input format" refers to the file format of the drawing data that the system receives from the user.

[0180] A "cleaning route" is a route designed to efficiently carry out cleaning work, and is set to pass through a specific area in an optimal order.

[0181] "Analysis" is the process of processing the received drawing data to identify cleaning areas and the location of obstacles.

[0182] The "cleaning area" refers to the area where cleaning work should be carried out, including rooms, corridors, etc.

[0183] "Obstacles" are objects or installations within the cleaning area that obstruct cleaning work, such as furniture and machinery.

[0184] "Presenting to the user" refers to the act of displaying the cleaning route automatically generated by the system to the user.

[0185] "Adjustable" refers to a state in which the user can change the cleaning route at will.

[0186] "Cleaning machine" means equipment for automatically performing cleaning tasks, including robots and other automated cleaning devices.

[0187] A "smart device" is a portable information terminal that can connect to the Internet and operate various applications, including smartphones and tablets.

[0188] The "drag-and-drop operation" refers to a function that allows users to intuitively operate elements on the screen of a smart device by moving the displayed elements using this operation.

[0189] "Preview" refers to a display that shows the user in advance what the final version will look like, allowing the user to check it before making any adjustments.

[0190] A "cleaning device" is hardware for performing cleaning tasks, and is synonymous with a cleaning machine, but has a more comprehensive meaning.

[0191] This invention is a system for streamlining cleaning work in large facilities with complex layouts. The system receives drawing data as input, analyzes it, automatically generates a cleaning route, and finally transmits the route to a cleaning robot to start cleaning work.

[0192] Drawing data input

[0193] Users upload facility layout data using their terminals. The layout data can be in PDF, CAD, or image format. This layout data includes layout information for large facilities such as buildings and factories.

[0194] Drawing data analysis

[0195] The server analyzes the received drawing data. This process identifies the work area and obstacles. For example, in the case of PDF files, it identifies room boundaries and corridor widths based on layer information. In the case of image formats, it uses image analysis software such as OpenCV to identify areas and obstacles.

[0196] Cleaning route generation

[0197] The server automatically generates an optimal cleaning route based on the analysis results. This route is designed to minimize distance and time so that the cleaning robot can move efficiently. For example, the route is calculated to pass through each room only once, eliminating unnecessary movements. This significantly improves the efficiency of cleaning work.

[0198] Present and coordinate cleaning routes

[0199] Users can preview the generated cleaning route on their device. On the smart device screen, users can easily fine-tune the route using drag-and-drop operations. For example, they can set specific areas to be cleaned additionally or exclude areas that do not need cleaning. In this way, users can make final adjustments to the route to suit the actual cleaning situation.

[0200] Cleaning work begins

[0201] The final cleaning route is sent from the server to the cleaning robot. The cleaning robot follows the received route and cleans efficiently within the specified area. For example, the robot can clean the hallway and also clean each room accurately. This reduces the workload of cleaning staff and significantly improves work efficiency.

[0202] Examples of concrete examples and prompts

[0203] This system is particularly effective for cleaning work in large factories, for example. If the factory layout is complex, using this system can significantly improve the efficiency of cleaning work.

[0204] You can input prompts like the following to the generative AI model:

[0205] "Please tell me how to efficiently clean a large factory with a complex layout. Please explain in detail, focusing in particular on how to automatically generate a route that will enable the cleaning robot to clean the shortest distance."

[0206] The system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0207] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0208] Step 1:

[0209] A user uses a terminal to upload drawing data of the facility.

[0210] Input: Drawing data (PDF, CAD, image format)

[0211] Specific operation: The user selects the drawing data file using the file selection function of the terminal and clicks the upload button. The uploaded file is sent to the server.

[0212] Step 2:

[0213] The server analyzes the received drawing data and identifies the cleaning area and obstacles.

[0214] Input: Uploaded drawing data

[0215] Output: Analysis results (area and obstacle location information)

[0216] Specific operation: The server uses image analysis software such as OpenCV to detect edges and perform layer analysis on the drawing data to identify the boundaries of the cleaning area and the location of obstacles, thereby obtaining location information for rooms, corridors, and obstacles.

[0217] Step 3:

[0218] The server automatically generates the optimal cleaning route based on the analysis results.

[0219] Input: Analysis results (area and obstacle location information)

[0220] Output: Optimal cleaning route

[0221] Specific operation: The server generates a cleaning route using an algorithm that calculates the shortest and most efficient route through each cleaning area. If necessary, it optimizes the route using the Dijkstra algorithm or the A algorithm.

[0222] Step 4:

[0223] The server transmits the generated cleaning route to the terminal and presents it to the user.

[0224] Input: Optimal cleaning route

[0225] Output: Preview of cleaning route

[0226] Specific operation: The server converts the generated cleaning route into JSON format and sends it to the device. The device then displays a preview of the cleaning route on the smart device screen based on the received JSON data.

[0227] Step 5:

[0228] The user can check the cleaning route on the device and make fine adjustments as needed using drag-and-drop operations.

[0229] Input: Preview of cleaning route

[0230] Output: The final cleaning route adjusted by the user.

[0231] How it works: The user uses touch gestures on their smart device to fine-tune the displayed cleaning route with drag and drop, and the adjusted route is updated in real time.

[0232] Step 6:

[0233] The user finalizes the cleaning route and sends it to the server.

[0234] Input: Final cleaning route adjusted by the user

[0235] Output: Confirmed cleaning route

[0236] Specific operation: After the user finishes adjusting the route, he / she clicks the Confirm button, which sends the confirmed route from the device to the server.

[0237] Step 7:

[0238] The server transmits the finalized cleaning route to the cleaning robot.

[0239] Input: Confirmed cleaning route

[0240] Output: Route instructions sent to the cleaning robot

[0241] Specific operation: The server sends the determined route information to the cleaning robot through its communication interface, and the robot starts cleaning according to the received route information.

[0242] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0243] This invention is a system for improving the efficiency of cleaning work in buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides an interface and cleaning route suggestions that are adapted to the user.

[0244] System configuration

[0245] 1. Inputting drawing data

[0246] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[0247] 2. Analysis of drawing data

[0248] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[0249] 3. Cleaning route generation

[0250] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[0251] 4. User Emotion Recognition by Emotion Engine

[0252] The emotion engine recognizes the user's emotions by analyzing their voice input and facial expressions to determine their emotional state.

[0253] 5. Present and coordinate cleaning routes

[0254] The generated cleaning route is presented to the user, and the interface is adjusted based on the user's emotions as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will be changed to a simpler and more intuitive one.

[0255] 6. User-adjusted cleaning route

[0256] The user can check the cleaning route displayed on their device and make fine adjustments using drag-and-drop operations. The emotion engine also monitors the user's emotional state and adjusts the suggestions and presentation method as necessary.

[0257] 7. Send cleaning route and start cleaning

[0258] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route, thereby reducing the workload of cleaning staff and improving work efficiency.

[0259] Program processing

[0260] The program processing of this system will be explained in natural language below.

[0261] Uploading drawing data

[0262] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[0263] Drawing data analysis

[0264] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[0265] Cleaning route generation

[0266] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[0267] Recognizing user emotions with an emotion engine

[0268] The server analyzes the user's voice input and facial expressions to recognize emotions. For example, if the user is feeling anxious, it will recognize this and provide additional support to the user.

[0269] Present and coordinate cleaning routes

[0270] The user checks the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the interface adjusts according to emotions recognized by the emotion engine. For example, if the user is feeling stressed, only important information is highlighted.

[0271] User-adjusted cleaning route

[0272] Users can adjust the cleaning route by dragging and dropping, and the emotion engine monitors the user's emotional state to help them make the right decisions. For example, if the user is unsure, it will provide optimal suggestions.

[0273] Cleaning work begins

[0274] The server then sends the final cleaning route to the cleaning robot, which then follows the route and cleans efficiently within the designated area, reducing the burden on the user and improving the efficiency of cleaning work.

[0275] The processing flow will be explained below.

[0276] Step 1:

[0277] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[0278] Step 2:

[0279] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and selects the appropriate analysis algorithm.

[0280] Step 3:

[0281] The server analyzes the drawing data and identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.). For example, it uses the layer information in the PDF file to identify room boundaries and the width of corridors, and stores the area information in a database.

[0282] Step 4:

[0283] Based on the analysis results, the server automatically generates the optimal cleaning route. This is the path that the cleaning robot will follow most efficiently, and is designed taking into account the shortest distance and time. Specifically, it calculates a route that passes through each area only once, eliminating any unnecessary movement.

[0284] Step 5:

[0285] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[0286] Step 6:

[0287] The server uses an emotion engine to analyze the user's voice input and facial expressions to recognize emotions. Specifically, emotion data is collected by having the user speak into the camera or capture their facial expressions.

[0288] Step 7:

[0289] The server adjusts the display interface for the cleaning route based on the user's emotions. For example, if the user is feeling stressed, the interface is simplified and only important information is highlighted.

[0290] Step 8:

[0291] The user fine-tunes the presented cleaning route by dragging and dropping. Specifically, the user drags and moves points on the route with a mouse or touch screen. If the user feels stressed while making adjustments, the emotion engine suggests an easier way to do the adjustments.

[0292] Step 9:

[0293] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[0294] Step 10:

[0295] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[0296] Step 11:

[0297] The cleaning robot will then start cleaning according to the received cleaning route. The robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[0298] Step 12:

[0299] The server monitors the progress of the cleaning job and reports back to the user as needed, for example, providing information about which areas have been cleaned and where problems have arisen.

[0300] Example 2

[0301] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0302] While conventional cleaning robot systems have the ability to automatically generate cleaning routes based on drawing data, they lack the ability to provide an interface that takes user emotions into consideration and provide sufficient operability. As a result, when users are stressed or anxious, operating the system becomes cumbersome, making it difficult to perform efficient cleaning tasks. Furthermore, when users adjust the cleaning route, there is a lack of support during the process, making it difficult to determine the optimal cleaning route.

[0303] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0304] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning work, means for providing an interface adapted to the user using an emotion engine that recognizes the user's emotions, and means for suggesting adjustments to the cleaning route based on the user's emotions recognized by the emotion engine. This enables efficient and user-friendly cleaning work by providing an appropriate interface and support based on the user's emotional state.

[0305] "Drawing data" refers to digital data that contains layout information for buildings and facilities, and includes PDF, CAD, and image formats.

[0306] A "cleaning route" is a path along which a cleaning robot moves efficiently to perform cleaning work.

[0307] A "cleaning area" is the target area where the cleaning robot will actually clean, and includes rooms and corridors.

[0308] An "obstacle" is an element that exists within the cleaning area and obstructs the movement of the cleaning robot, such as furniture or pillars.

[0309] An "emotion engine" is software or hardware that analyzes a user's voice input and facial expressions to recognize their emotional state.

[0310] An "interface" is an operation screen or input means for a user to interact with a system, and includes both visual and operational elements.

[0311] "Suggestions" are revisions and guidance provided by the emotion engine to help users set the optimal cleaning route.

[0312] A "cleaning robot" is a mechanical device that automatically performs cleaning tasks according to a cleaning route set by a user.

[0313] "Analysis" refers to a series of data processing steps to identify cleaning areas and obstacles from drawing data.

[0314] The present invention is a system that realizes efficient cleaning work by automatically generating cleaning routes based on drawing data and providing an interface that responds to the user's emotional state. Each component and its specific operation will be described below.

[0315] Drawing data input

[0316] The user uses a device to upload blueprint data of the area to be cleaned to the server. This blueprint data is in PDF, CAD, or image format and includes layout information for the building or facility. Specifically, the user opens a web browser on the device, accesses the system interface, and clicks the "Select File" button to specify the file. The user then presses the "Send" button to send the blueprint data to the server.

[0317] Drawing data analysis

[0318] The server analyzes the received drawing data to identify cleaning areas and obstacles. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, the server identifies room boundaries, aisle widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[0319] Cleaning route generation

[0320] The server automatically generates the optimal cleaning route based on the analysis results. This generation process uses path-finding algorithms such as Dijkstra and A (A-star) to design a route that will clean efficiently and in the shortest distance. For example, the route is calculated to pass through each room only once and avoid unnecessary movement. The cleaning order of each area is also optimized.

[0321] Recognizing user emotions with an emotion engine

[0322] The server uses a generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when a user uses a microphone to say "I need help," the emotion engine analyzes the voice data and identifies emotions such as stress or anxiety. For facial expressions, when the user turns their face toward the camera, the AI ​​model analyzes their facial expressions to determine their emotional state.

[0323] Present and coordinate cleaning routes

[0324] The user checks the cleaning route generated by the server on their device. A preview of the cleaning route is displayed on the screen, and the interface adjusts according to the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information. The user can preview the cleaning route and make corrections as needed using drag-and-drop operations.

[0325] Cleaning work begins

[0326] The server then sends the final cleaning route to the cleaning robot, which then follows the received route to efficiently clean. The robot uses sensors to determine its own location and operates along the specified route, so any errors or obstacles that occur are automatically corrected.

[0327] Examples of specific examples and prompts

[0328] The user uploads a PDF file of the floor plan of the office building from their device to the server. The server receives the PDF file, identifies the locations of rooms, corridors, and obstacles, and generates a cleaning route. The user can check the generated cleaning route on the screen and adjust it as needed by dragging and dropping. Once the final route is determined, the server sends it to the cleaning robot, which then begins cleaning.

[0329] Prompt Sentence Examples

[0330] 1. "Please tell me the procedure for uploading the cleaning target drawing (PDF format)."

[0331] 2. "Please outline an algorithm for automatically generating cleaning routes."

[0332] 3. "Can you give me an example of how an interface can be adjusted to detect a user's stress level?"

[0333] This provides an interface and support based on the user's emotional state, enabling efficient and user-friendly cleaning work.

[0334] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0335] Step 1: Upload the drawing data

[0336] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. Specifically, the user opens a web browser, accesses the system interface, clicks the "Select File" button, selects a PDF, CAD, or image file from their local storage, and presses the "Submit" button.

[0337] Input: Drawing data in PDF, CAD, or image format

[0338] Output: Drawing data uploaded to the server

[0339] Step 2: Analyze the drawing data

[0340] The server receives the uploaded drawing data and begins analyzing it. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, it identifies room boundaries, corridor widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[0341] Input: Uploaded drawing data

[0342] Data processing: Analysis of layer information, application of shape recognition algorithms

[0343] Output: A virtual map that identifies room boundaries, corridor widths, and obstacle locations

[0344] Step 3: Generate cleaning routes

[0345] The server automatically generates an optimal cleaning route based on the analysis results. This process uses path-finding algorithms such as Dijkstra and A (Aster). Specifically, the route is calculated to pass through each room only once, eliminating unnecessary movements, and optimizing the cleaning order of each area.

[0346] Input: A virtual map based on the analysis results

[0347] Data Computation: Applying Pathfinding Algorithms

[0348] Output: Optimized cleaning route

[0349] Step 4: Recognizing user emotions with the emotion engine

[0350] The server uses the generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when analyzing voice input, if the user says "I need help," the voice data is recognized. When analyzing facial expressions, if the user shows their face to the camera, the emotion engine analyzes it and identifies their emotional state.

[0351] Input: User voice input, facial expression data

[0352] Data analysis: Analysis of voice data, analysis of facial expressions

[0353] Output: User's emotional state (e.g., stress, anxiety)

[0354] Step 5: Present and coordinate cleaning routes

[0355] The user checks the cleaning route generated by the server on their device. Specifically, the user previews the cleaning route on the screen, and the emotion engine recognizes the user's emotional state and adjusts the interface accordingly. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information.

[0356] Input: Cleaning route provided by the server, user's emotional state

[0357] Data display: interface adjustment, route preview display

[0358] Output: Cleaning route confirmed by the user

[0359] Step 6: User adjusts cleaning route

[0360] Users can adjust the cleaning route by dragging and dropping. Specifically, the user grabs the route on the screen and drags and drops it to another area. During this process, the emotion engine monitors the user's voice and facial expressions, and if the user is unsure, it displays optimal suggestions in the form of tooltips.

[0361] Input: User's cleaning route adjustment operations, emotion engine monitoring data

[0362] Data processing: Route changes, display of suggested content

[0363] Output: Adjusted cleaning route

[0364] Step 7: Start cleaning

[0365] The server then sends the final cleaning route to the cleaning robot. Specifically, the server transmits the final route to the cleaning robot via radio waves, and the robot begins to operate based on that route. The robot uses sensors to confirm its own location and proceeds with cleaning along the specified path.

[0366] Input: Finalized cleaning route

[0367] Data transmission: Sending cleaning route

[0368] Output: Cleaning robot starts operating

[0369] The above are the processing steps of the program for this system. By explaining the specific operations, inputs, and outputs in detail for each step, the function of the entire system can be clarified.

[0370] (Application example 2)

[0371] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0372] In modern logistics centers, product placement changes and the creation and adjustment of transport routes are frequent, placing an increased burden on operators. Furthermore, operators who are fatigued or stressed are unable to perform optimal tasks, which is a problem. Therefore, a new system is needed to improve the efficiency of logistics processes and reduce the burden on operators.

[0373] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0374] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for analyzing the user's voice input and facial expression to recognize emotions, means for adjusting the interface based on the recognized emotions, and means for transmitting the final cleaning route to an automated guided vehicle and starting the transportation work. This enables transportation work within a logistics center to be performed efficiently, reducing the burden on operators and improving the quality of work.

[0375] Creating definition statements

[0376] "Drawing data" refers to layout information for buildings and facilities that indicates the location of cleaning areas and obstacles, and is provided in formats such as PDF, CAD, and image formats.

[0377] A "cleaning route" is a path for an object to travel efficiently within a specified area, and is automatically generated taking into account the shortest distance and time.

[0378] "Means for recognizing emotions" refers to technology that analyzes a user's voice input and facial expressions to determine their emotional state.

[0379] An "interface" refers to the operating screen and operating method that allows a user to interact with a system, and which adapts to the user's emotional state.

[0380] A "user" is a person who operates the system, uploads drawing data, and coordinates cleaning routes.

[0381] An "automated transport vehicle" is an autonomous vehicle used to efficiently transport goods within a logistics center, and includes AGVs (automated guided vehicles).

[0382] A "server" is a central computer that stores and processes various data, and provides processing functions such as analyzing drawing data and recognizing emotions.

[0383] MODE FOR CARRYING OUT THE INVENTION

[0384] This invention is a system for efficiently operating automated guided vehicles in a logistics center, which automatically generates optimal transport routes based on drawing data and product placement information, and uses an emotion engine to recognize and respond to the emotional state of the operator. The details are explained below.

[0385] System configuration

[0386] 1. Inputting drawing data

[0387] Users use their terminals to upload drawing data showing the layout and product placement within the distribution center to the server, including PDF, CAD, image formats, etc. For example, a user uploads the latest floor plan of a distribution center in PDF format.

[0388] 2. Analysis of drawing data

[0389] The server analyzes the uploaded drawing data and identifies the product placement location, movement path, and location of obstacles. This analysis process uses a drawing analysis engine such as OpenCV.

[0390] 3. Generate transport routes

[0391] The server automatically generates the optimal transport route based on the identified product locations and obstacles. The designed route is created taking into account the shortest distance and time to move products efficiently. For example, it calculates the route for AGVs to efficiently transport products within a logistics center.

[0392] 4. Emotion engine recognizes operator emotions

[0393] The server analyzes the operator's voice input and facial expressions to recognize emotions, for example, using emotion recognition software from Microsoft® Azure® Cognitive Services to determine the operator's level of fatigue or stress.

[0394] 5. Delivery route presentation and adjustment interface

[0395] The user can check the delivery route generated by the server on their device. At this time, the emotion engine adjusts the interface based on the operator's emotional state. For example, if the user is tired, the interface will be adjusted to be simple and intuitive.

[0396] 6. Operators adjust transport routes

[0397] Users can fine-tune delivery routes using drag-and-drop operations. The emotion engine monitors the user's emotional state and provides appropriate suggestions and instructions. For example, when a user fine-tunes a part of a delivery route, the system provides optimal suggestions.

[0398] 7. Sending the transport route and starting the transport operation

[0399] The server then sends the final determined transport route to the automated guided vehicle (AGV). The AGV then follows the received route and transports the products efficiently within the specified area. This improves the efficiency of transport work and reduces the burden on the operator.

[0400] Hardware and software used

[0401] Hardware: Smartphones, tablets, smart glasses, AGVs (automated guided vehicles)

[0402] Software: Emotion recognition software (Microsoft Azure Cognitive Services), AGV control software, drawing analysis engine (OpenCV)

[0403] Examples and prompts

[0404] As a concrete example, at a certain logistics center, setting up transport routes was taking a lot of time and effort. With this system, managers can easily upload drawing data and automatically generate optimal transport routes. Examples of prompt sentences include the following:

[0405] Prompt Sentence Examples

[0406] "Upload the latest layout drawings to generate the optimal transport route."

[0407] By using this system, it is expected that transportation operations at logistics centers will be carried out more efficiently, reducing the burden on operators and improving the quality of work.

[0408] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0409] Program processing flow

[0410] Specific explanation of processing steps

[0411] Step 1:

[0412] The user uploads drawing data showing the layout of the distribution center and the placement of products from their terminal to the server. The input data can be a PDF, CAD, or image file. The server receives this drawing data and prepares it for the next step.

[0413] Step 2:

[0414] The server analyzes the uploaded drawing data. For example, it uses a drawing analysis engine such as OpenCV to identify product placement locations, movement paths, and the location of obstacles. The input data is the drawing data, and the output data is cleaning areas and location information of obstacles. This becomes the base data for generating subsequent transport routes.

[0415] Step 3:

[0416] The server automatically generates the optimal transport route based on the analysis results. Here, the route is calculated taking into account the shortest distance and time to move products efficiently. The input data is the location information of the identified cleaning area and obstacles, and the output data is the initial transport route.

[0417] Step 4:

[0418] The server uses an emotion engine to analyze the operator's voice input and facial expressions to recognize emotions. For example, it uses Microsoft Azure Cognitive Services to determine the operator's emotional state. The input data is the operator's voice and facial expression data, and the output data is the recognized emotional state.

[0419] Step 5:

[0420] The server presents the generated delivery route to the operator on the terminal and adjusts the interface based on the emotional state recognized by the emotion engine. The input data are the initial delivery route and the emotional state, and the output data is the delivery route displayed in the adjusted interface.

[0421] Step 6:

[0422] The user fine-tunes the delivery route using drag-and-drop operations on the terminal. The input data are the instructions the operator will use, and the output data is the fine-tuned delivery route. During this process, the server monitors the operator's emotional state and makes appropriate suggestions and instructions.

[0423] Step 7:

[0424] The server sends the final determined transport route to the automated guided vehicle (AGV). The input data is the finely adjusted final transport route, and the output data is the control command sent to the AGV. The AGV transports the goods efficiently within the specified range based on this command. This starts the transport work and allows it to be carried out efficiently.

[0425] Throughout these processing steps, a system is realized that allows transportation operations within the logistics center to be carried out efficiently and reduces the burden on operators.

[0426] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0428] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0429] [Second embodiment]

[0430] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0431] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0433] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0437] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0438] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0439] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0440] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0441] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0442] This invention is a system for streamlining cleaning work within buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. This system is particularly effective in facilities with large floors or complex layouts.

[0443] System configuration

[0444] 1. Inputting drawing data

[0445] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[0446] 2. Analysis of drawing data

[0447] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[0448] 3. Cleaning route generation

[0449] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[0450] 4. Present and coordinate cleaning routes

[0451] The generated cleaning route is presented to the user, who can check it on their device. If necessary, they can fine-tune the cleaning route using drag-and-drop operations. In this way, the user can make final adjustments to the route to suit the actual cleaning situation.

[0452] 5. Send the cleaning route and start cleaning

[0453] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route. This reduces the workload of cleaning staff and improves work efficiency.

[0454] Program processing

[0455] The program processing of this system will be explained in natural language below.

[0456] Uploading drawing data

[0457] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[0458] Drawing data analysis

[0459] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[0460] Cleaning route generation

[0461] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[0462] Present and coordinate cleaning routes

[0463] The user can view the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the user can adjust it using drag and drop. This allows the user to exclude areas that do not need cleaning or set specific areas for additional cleaning.

[0464] Cleaning work begins

[0465] The server sends the final cleaning route to the cleaning robot, which then starts cleaning based on this route. The robot follows the received route and efficiently cleans within the specified area. For example, the robot can clean the hallway and each room accurately.

[0466] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0467] The processing flow will be explained below.

[0468] Step 1:

[0469] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[0470] Step 2:

[0471] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and converts it into the appropriate format for use in the next analysis step.

[0472] Step 3:

[0473] The server begins analyzing the drawing data. Specifically, in the case of PDF or CAD data, it reads the drawing's layer information and identifies area information (rooms and corridors) and obstacles (furniture and pillars). In the case of image data, it uses image analysis algorithms to extract similar area information.

[0474] Step 4:

[0475] Based on the analysis results, the server automatically generates an optimal cleaning route that takes into account the cleaning area and obstacles. Specifically, it calculates a route that passes through each area only once and minimizes unnecessary movement.

[0476] Step 5:

[0477] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[0478] Step 6:

[0479] Users can fine-tune the proposed cleaning route by dragging and dropping. Users can exclude unnecessary areas or adjust to clean specific areas. Once adjustments are complete, users click the confirm button.

[0480] Step 7:

[0481] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[0482] Step 8:

[0483] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[0484] Step 9:

[0485] The cleaning robot will then start cleaning according to the received cleaning route. The cleaning robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[0486] Example 1

[0487] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0488] Conventional cleaning work requires a lot of time and effort, and it is difficult to efficiently set cleaning routes, especially in facilities with large floors or complex layouts. Furthermore, manual cleaning route setting by cleaning staff lacks flexibility and often results in inconsistent work quality. To solve these problems, a system was needed that could efficiently analyze drawing data and automatically generate and adjust cleaning routes.

[0489] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0490] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for sending the final cleaning route to a cleaning device and starting cleaning work, means for sending a file to the server and displaying the upload progress, means for analyzing the drawing data based on layer information and coordinate data, means for converting the cleaning route into JSON format and sending it to the user's terminal, means for displaying a route preview and allowing adjustment by drag and drop, and means for sending the generated optimal route to the API endpoint of the cleaning device. This allows users to easily operate the system, enabling efficient cleaning route setting and automation of actual cleaning work.

[0491] "Drawing data" refers to digital data such as PDFs, blueprints, and image formats that contain layout information for buildings and facilities.

[0492] "Cleaning route" refers to the path that a cleaning device follows within the area to be cleaned, including a path optimized for efficient cleaning.

[0493] "Cleaning area" refers to the specific area, such as a room, hallway, or corridor, where cleaning work is performed.

[0494] "Obstacles" refers to objects such as furniture or pillars that impede the movement of cleaning equipment.

[0495] "Cleaning equipment" refers to robots and devices that perform cleaning tasks automatically.

[0496] "User" refers to the person who operates the system to set and adjust cleaning routes.

[0497] "Terminal" refers to an information device such as a computer or tablet that is operated by a user.

[0498] "Server" refers to a central computer system that performs processes such as analyzing drawing data and automatically generating cleaning routes.

[0499] "Upload" refers to the operation of sending data from a terminal to a server.

[0500] "Analysis" refers to the process of extracting necessary information from drawing data and identifying cleaning areas and obstacles.

[0501] "Drag and drop" refers to an operation in which a user moves an object on the screen using a mouse or touch operation.

[0502] "JSON format" refers to a lightweight data exchange format for structuring and storing data.

[0503] "API Endpoint" refers to a specific interface for communicating with a cleaning device.

[0504] "Preview" refers to the state in which the generated cleaning route can be visually displayed and confirmed.

[0505] The present invention is a system for improving the efficiency of cleaning work within buildings and facilities, receiving drawing data as input and automatically generating cleaning routes. This system is particularly effective in facilities with large floor areas or complex layouts. Specific embodiments of this system are described below.

[0506] The system consists of three main components: a server, a terminal, and cleaning equipment. Drawing data is provided in PDF, blueprint, image format, etc. The server uses analysis tools such as Adobe Acrobat SDK to extract the necessary information from the drawing data.

[0507] Uploading drawing data

[0508] The user uses the device to upload the drawing data of the area to be cleaned to the server. Specifically, the user opens the file selection window on the device, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The device sends the selected file to the server and displays the upload progress on the screen.

[0509] Drawing data analysis

[0510] The server analyzes the received drawing data and identifies the locations of rooms, passageways, and obstacles. The analysis uses Adobe Acrobat SDK and other tools, making full use of the PDF file's layer information and image processing technology to extract the necessary data. For example, room boundaries, passageway widths, and the locations of furniture and pillars can be identified.

[0511] Cleaning route generation

[0512] Based on the analysis results, the server automatically generates an optimal cleaning route. The server uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The result is converted into JSON format and sent to the user's device.

[0513] Present and coordinate cleaning routes

[0514] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," visually displays a preview of the route and allows the user to fine-tune the route using drag-and-drop operations. Users can exclude areas that do not require cleaning and set additional areas to be cleaned.

[0515] Cleaning work begins

[0516] The server sends the finalized cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts cleaning work based on the received route information. The cleaning device moves efficiently and cleans within the specified area.

[0517] Examples of specific examples and prompts

[0518] A specific example of this system is shown below.

[0519] Example of uploading drawing data

[0520] The user selects the PDF file of their office floor on the terminal and clicks the "Upload" button.

[0521] Specific examples of drawing data analysis

[0522] The server receives "Office Floor.pdf" and uses software called "PDF Analysis Tool" to analyze the PDF layers and identify the locations of rooms, corridors, and obstacles to be cleaned.

[0523] Example of cleaning route generation

[0524] The server uses an algorithm to calculate the shortest route from "Room A" to "Room B" and generates a route.

[0525] Examples of cleaning route presentation and adjustment

[0526] The user checks the route presented in the application on the device and removes the unnecessary room, "C�D Room," from the route.

[0527] Specific examples of when cleaning work begins

[0528] The server then sends the final route to the "cleaning robot," which then begins cleaning the designated area, efficiently cleaning without hitting any obstacles along the way.

[0529] Example prompts for generative AI models

[0530] "I've uploaded an office floor plan (PDF file). Please explain how the system generates optimal cleaning routes based on this floor plan and allows me to adjust the routes with drag and drop."

[0531] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0532] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0533] Step 1: Upload the drawing data

[0534] The user uses the terminal to upload the drawing data to the server. Specifically, the user opens the file selection window on the terminal, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The input here is the PDF file selected by the user, and that file is sent to the server. The terminal splits the file and sends it to the server, and displays the upload progress.

[0535] Input: PDF file

[0536] Output: Drawing data uploaded to the server

[0537] Specific behavior:

[0538] The user selects "Office Floor.pdf."

[0539] The device splits the file into 10MB chunks and sends them to the server.

[0540] Displays an upload progress bar on the device screen.

[0541] Step 2: Analyze the drawing data

[0542] The server analyzes the received drawing data. During this process, the server uses analysis tools such as Adobe Acrobat SDK to extract the contents of the PDF file. Based on the layer information and coordinate data of the drawing data, the server identifies the locations of rooms, corridors, and obstacles.

[0543] Input: Drawing data uploaded to the server

[0544] Output: Analyzed cleaning area and obstacle information

[0545] Specific behavior:

[0546] The server opens "OfficeFloor.pdf".

[0547] The server scans the layer information to detect room boundaries.

[0548] The width of the aisle is measured and the position of obstacles (desks, chairs, etc.) is recorded as coordinate data.

[0549] Step 3: Generate cleaning routes

[0550] The server generates an optimal cleaning route based on the analysis results. Specifically, it uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The calculation results are converted into JSON format and sent to the user's device.

[0551] Input: Analyzed cleaning area and obstacle information

[0552] Output: Cleaning route information in JSON format

[0553] Specific behavior:

[0554] The server calculates a route that passes through each room once.

[0555] Convert the calculation results into JSON format.

[0556] The server logs the computation time and the optimality of the route.

[0557] Step 4: Present and coordinate cleaning routes

[0558] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," provides a visual preview of the route and allows users to fine-tune the route using drag-and-drop operations.

[0559] Input: Cleaning route information in JSON format

[0560] Output: The final cleaning route as adjusted by the user

[0561] Specific behavior:

[0562] The device receives route information in JSON format and displays it visually in a GUI.

[0563] The user clicks on "CD Room" to exclude it from the route.

[0564] After making adjustments, the user clicks the "Confirm" button to save the final route.

[0565] Step 5: Start cleaning

[0566] The server sends the final cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts work based on the received route information and efficiently cleans within the specified area.

[0567] Input: Final cleaning route adjusted by the user

[0568] Output: The cleaning task performed by the cleaning equipment

[0569] Specific behavior:

[0570] The server sends the route data to the cleaning equipment's control API.

[0571] The cleaning equipment analyzes the route data and generates a cleaning schedule.

[0572] The cleaning equipment begins cleaning within the designated area.

[0573] (Application example 1)

[0574] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0575] In today's vast and complex facilities and factories, manual cleaning is not only laborious and time-consuming, but also inefficient, resulting in a high likelihood of a lack of consistency across the entire cleaning area. Furthermore, designing and adjusting cleaning routes requires specialized knowledge, making it difficult for average employees. Therefore, a system that can automatically generate and easily adjust efficient and unified cleaning routes is desired.

[0576] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0577] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to a cleaning machine and starting cleaning work, means for identifying the location of work areas and equipment within a large-scale facility when analyzing the uploaded drawing data, means for displaying a preview of the cleaning route on a smart device and allowing the user to fine-tune the route by dragging and dropping, and means for confirming the cleaning route and transmitting the confirmed route to the cleaning machine, thereby making it possible to improve the efficiency and automation of cleaning work and provide user-friendly adjustment functions.

[0578] "Drawing data" refers to digital data that includes layout information for buildings and facilities, and is expressed in PDF, CAD, or image format.

[0579] "Input format" refers to the file format of the drawing data that the system receives from the user.

[0580] A "cleaning route" is a route designed to efficiently carry out cleaning work, and is set to pass through a specific area in an optimal order.

[0581] "Analysis" is the process of processing the received drawing data to identify cleaning areas and the location of obstacles.

[0582] The "cleaning area" refers to the area where cleaning work should be carried out, including rooms, corridors, etc.

[0583] "Obstacles" are objects or installations within the cleaning area that obstruct cleaning work, such as furniture and machinery.

[0584] "Presenting to the user" refers to the act of displaying the cleaning route automatically generated by the system to the user.

[0585] "Adjustable" refers to a state in which the user can change the cleaning route at will.

[0586] "Cleaning machine" means equipment for automatically performing cleaning tasks, including robots and other automated cleaning devices.

[0587] A "smart device" is a portable information terminal that can connect to the Internet and operate various applications, including smartphones and tablets.

[0588] The "drag-and-drop operation" refers to a function that allows users to intuitively operate elements on the screen of a smart device by moving the displayed elements using this operation.

[0589] "Preview" refers to a display that shows the user in advance what the final version will look like, allowing the user to check it before making any adjustments.

[0590] A "cleaning device" is hardware for performing cleaning tasks, and is synonymous with a cleaning machine, but has a more comprehensive meaning.

[0591] This invention is a system for streamlining cleaning work in large facilities with complex layouts. The system receives drawing data as input, analyzes it, automatically generates a cleaning route, and finally transmits the route to a cleaning robot to start cleaning work.

[0592] Drawing data input

[0593] Users upload facility layout data using their terminals. The layout data can be in PDF, CAD, or image format. This layout data includes layout information for large facilities such as buildings and factories.

[0594] Drawing data analysis

[0595] The server analyzes the received drawing data. This process identifies the work area and obstacles. For example, in the case of PDF files, it identifies room boundaries and corridor widths based on layer information. In the case of image formats, it uses image analysis software such as OpenCV to identify areas and obstacles.

[0596] Cleaning route generation

[0597] The server automatically generates an optimal cleaning route based on the analysis results. This route is designed to minimize distance and time so that the cleaning robot can move efficiently. For example, the route is calculated to pass through each room only once, eliminating unnecessary movements. This significantly improves the efficiency of cleaning work.

[0598] Present and coordinate cleaning routes

[0599] Users can preview the generated cleaning route on their device. On the smart device screen, users can easily fine-tune the route using drag-and-drop operations. For example, they can set specific areas to be cleaned additionally or exclude areas that do not need cleaning. In this way, users can make final adjustments to the route to suit the actual cleaning situation.

[0600] Cleaning work begins

[0601] The final cleaning route is sent from the server to the cleaning robot. The cleaning robot follows the received route and cleans efficiently within the specified area. For example, the robot can clean the hallway and also clean each room accurately. This reduces the workload of cleaning staff and significantly improves work efficiency.

[0602] Examples of concrete examples and prompts

[0603] This system is particularly effective for cleaning work in large factories, for example. If the factory layout is complex, using this system can significantly improve the efficiency of cleaning work.

[0604] You can input prompts like the following to the generative AI model:

[0605] "Please tell me how to efficiently clean a large factory with a complex layout. Please explain in detail, focusing in particular on how to automatically generate a route that will enable the cleaning robot to clean the shortest distance."

[0606] The system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0607] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0608] Step 1:

[0609] A user uses a terminal to upload drawing data of the facility.

[0610] Input: Drawing data (PDF, CAD, image format)

[0611] Specific operation: The user selects the drawing data file using the file selection function of the terminal and clicks the upload button. The uploaded file is sent to the server.

[0612] Step 2:

[0613] The server analyzes the received drawing data and identifies the cleaning area and obstacles.

[0614] Input: Uploaded drawing data

[0615] Output: Analysis results (area and obstacle location information)

[0616] Specific operation: The server uses image analysis software such as OpenCV to detect edges and perform layer analysis on the drawing data to identify the boundaries of the cleaning area and the location of obstacles, thereby obtaining location information for rooms, corridors, and obstacles.

[0617] Step 3:

[0618] The server automatically generates the optimal cleaning route based on the analysis results.

[0619] Input: Analysis results (area and obstacle location information)

[0620] Output: Optimal cleaning route

[0621] Specific operation: The server generates a cleaning route using an algorithm that calculates the shortest and most efficient route through each cleaning area. If necessary, it optimizes the route using the Dijkstra algorithm or the A algorithm.

[0622] Step 4:

[0623] The server transmits the generated cleaning route to the terminal and presents it to the user.

[0624] Input: Optimal cleaning route

[0625] Output: Preview of cleaning route

[0626] Specific operation: The server converts the generated cleaning route into JSON format and sends it to the device. The device then displays a preview of the cleaning route on the smart device screen based on the received JSON data.

[0627] Step 5:

[0628] The user can check the cleaning route on the device and make fine adjustments as needed using drag-and-drop operations.

[0629] Input: Preview of cleaning route

[0630] Output: The final cleaning route adjusted by the user.

[0631] How it works: The user uses touch gestures on their smart device to fine-tune the displayed cleaning route with drag and drop, and the adjusted route is updated in real time.

[0632] Step 6:

[0633] The user finalizes the cleaning route and sends it to the server.

[0634] Input: Final cleaning route adjusted by the user

[0635] Output: Confirmed cleaning route

[0636] Specific operation: After the user finishes adjusting the route, he / she clicks the Confirm button, which sends the confirmed route from the device to the server.

[0637] Step 7:

[0638] The server transmits the finalized cleaning route to the cleaning robot.

[0639] Input: Confirmed cleaning route

[0640] Output: Route instructions sent to the cleaning robot

[0641] Specific operation: The server sends the determined route information to the cleaning robot through its communication interface, and the robot starts cleaning according to the received route information.

[0642] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0643] This invention is a system for improving the efficiency of cleaning work in buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides an interface and cleaning route suggestions that are adapted to the user.

[0644] System configuration

[0645] 1. Inputting drawing data

[0646] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[0647] 2. Analysis of drawing data

[0648] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[0649] 3. Cleaning route generation

[0650] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[0651] 4. User Emotion Recognition by Emotion Engine

[0652] The emotion engine recognizes the user's emotions by analyzing their voice input and facial expressions to determine their emotional state.

[0653] 5. Present and coordinate cleaning routes

[0654] The generated cleaning route is presented to the user, and the interface is adjusted based on the user's emotions as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will be changed to a simpler and more intuitive one.

[0655] 6. User-adjusted cleaning route

[0656] The user can check the cleaning route displayed on their device and make fine adjustments using drag-and-drop operations. The emotion engine also monitors the user's emotional state and adjusts the suggestions and presentation method as necessary.

[0657] 7. Send cleaning route and start cleaning

[0658] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route, thereby reducing the workload of cleaning staff and improving work efficiency.

[0659] Program processing

[0660] The program processing of this system will be explained in natural language below.

[0661] Uploading drawing data

[0662] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[0663] Drawing data analysis

[0664] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[0665] Cleaning route generation

[0666] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[0667] Recognizing user emotions with an emotion engine

[0668] The server analyzes the user's voice input and facial expressions to recognize emotions. For example, if the user is feeling anxious, it will recognize this and provide additional support to the user.

[0669] Present and coordinate cleaning routes

[0670] The user checks the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the interface adjusts according to emotions recognized by the emotion engine. For example, if the user is feeling stressed, only important information is highlighted.

[0671] User-adjusted cleaning route

[0672] Users can adjust the cleaning route by dragging and dropping, and the emotion engine monitors the user's emotional state to help them make the right decisions. For example, if the user is unsure, it will provide optimal suggestions.

[0673] Cleaning work begins

[0674] The server then sends the final cleaning route to the cleaning robot, which then follows the route and cleans efficiently within the designated area, reducing the burden on the user and improving the efficiency of cleaning work.

[0675] The processing flow will be explained below.

[0676] Step 1:

[0677] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[0678] Step 2:

[0679] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and selects the appropriate analysis algorithm.

[0680] Step 3:

[0681] The server analyzes the drawing data and identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.). For example, it uses the layer information in the PDF file to identify room boundaries and the width of corridors, and stores the area information in a database.

[0682] Step 4:

[0683] Based on the analysis results, the server automatically generates the optimal cleaning route. This is the path that the cleaning robot will follow most efficiently, and is designed taking into account the shortest distance and time. Specifically, it calculates a route that passes through each area only once, eliminating any unnecessary movement.

[0684] Step 5:

[0685] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[0686] Step 6:

[0687] The server uses an emotion engine to analyze the user's voice input and facial expressions to recognize emotions. Specifically, emotion data is collected by having the user speak into the camera or capture their facial expressions.

[0688] Step 7:

[0689] The server adjusts the display interface for the cleaning route based on the user's emotions. For example, if the user is feeling stressed, the interface is simplified and only important information is highlighted.

[0690] Step 8:

[0691] The user fine-tunes the presented cleaning route by dragging and dropping. Specifically, the user drags and moves points on the route with a mouse or touch screen. If the user feels stressed while making adjustments, the emotion engine suggests an easier way to do the adjustments.

[0692] Step 9:

[0693] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[0694] Step 10:

[0695] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[0696] Step 11:

[0697] The cleaning robot will then start cleaning according to the received cleaning route. The robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[0698] Step 12:

[0699] The server monitors the progress of the cleaning job and reports back to the user as needed, for example, providing information about which areas have been cleaned and where problems have arisen.

[0700] Example 2

[0701] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0702] While conventional cleaning robot systems have the ability to automatically generate cleaning routes based on drawing data, they lack the ability to provide an interface that takes user emotions into consideration and provide sufficient operability. As a result, when users are stressed or anxious, operating the system becomes cumbersome, making it difficult to perform efficient cleaning tasks. Furthermore, when users adjust the cleaning route, there is a lack of support during the process, making it difficult to determine the optimal cleaning route.

[0703] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0704] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning work, means for providing an interface adapted to the user using an emotion engine that recognizes the user's emotions, and means for suggesting adjustments to the cleaning route based on the user's emotions recognized by the emotion engine. This enables efficient and user-friendly cleaning work by providing an appropriate interface and support based on the user's emotional state.

[0705] "Drawing data" refers to digital data that contains layout information for buildings and facilities, and includes PDF, CAD, and image formats.

[0706] A "cleaning route" is a path along which a cleaning robot moves efficiently to perform cleaning work.

[0707] A "cleaning area" is the target area where the cleaning robot will actually clean, and includes rooms and corridors.

[0708] An "obstacle" is an element that exists within the cleaning area and obstructs the movement of the cleaning robot, such as furniture or pillars.

[0709] An "emotion engine" is software or hardware that analyzes a user's voice input and facial expressions to recognize their emotional state.

[0710] An "interface" is an operation screen or input means for a user to interact with a system, and includes both visual and operational elements.

[0711] "Suggestions" are revisions and guidance provided by the emotion engine to help users set the optimal cleaning route.

[0712] A "cleaning robot" is a mechanical device that automatically performs cleaning tasks according to a cleaning route set by a user.

[0713] "Analysis" refers to a series of data processing steps to identify cleaning areas and obstacles from drawing data.

[0714] The present invention is a system that realizes efficient cleaning work by automatically generating cleaning routes based on drawing data and providing an interface that responds to the user's emotional state. Each component and its specific operation will be described below.

[0715] Drawing data input

[0716] The user uses a device to upload blueprint data of the area to be cleaned to the server. This blueprint data is in PDF, CAD, or image format and includes layout information for the building or facility. Specifically, the user opens a web browser on the device, accesses the system interface, and clicks the "Select File" button to specify the file. The user then presses the "Send" button to send the blueprint data to the server.

[0717] Drawing data analysis

[0718] The server analyzes the received drawing data to identify cleaning areas and obstacles. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, the server identifies room boundaries, aisle widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[0719] Cleaning route generation

[0720] The server automatically generates the optimal cleaning route based on the analysis results. This generation process uses path-finding algorithms such as Dijkstra and A (A-star) to design a route that will clean efficiently and in the shortest distance. For example, the route is calculated to pass through each room only once and avoid unnecessary movement. The cleaning order of each area is also optimized.

[0721] Recognizing user emotions with an emotion engine

[0722] The server uses a generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when a user uses a microphone to say "I need help," the emotion engine analyzes the voice data and identifies emotions such as stress or anxiety. For facial expressions, when the user turns their face toward the camera, the AI ​​model analyzes their facial expressions to determine their emotional state.

[0723] Present and coordinate cleaning routes

[0724] The user checks the cleaning route generated by the server on their device. A preview of the cleaning route is displayed on the screen, and the interface adjusts according to the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information. The user can preview the cleaning route and make corrections as needed using drag-and-drop operations.

[0725] Cleaning work begins

[0726] The server then sends the final cleaning route to the cleaning robot, which then follows the received route to efficiently clean. The robot uses sensors to determine its own location and operates along the specified route, so any errors or obstacles that occur are automatically corrected.

[0727] Examples of specific examples and prompts

[0728] The user uploads a PDF file of the floor plan of the office building from their device to the server. The server receives the PDF file, identifies the locations of rooms, corridors, and obstacles, and generates a cleaning route. The user can check the generated cleaning route on the screen and adjust it as needed by dragging and dropping. Once the final route is determined, the server sends it to the cleaning robot, which then begins cleaning.

[0729] Prompt Sentence Examples

[0730] 1. "Please tell me the procedure for uploading the cleaning target drawing (PDF format)."

[0731] 2. "Please outline an algorithm for automatically generating cleaning routes."

[0732] 3. "Can you give me an example of how an interface can be adjusted to detect a user's stress level?"

[0733] This provides an interface and support based on the user's emotional state, enabling efficient and user-friendly cleaning work.

[0734] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0735] Step 1: Upload the drawing data

[0736] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. Specifically, the user opens a web browser, accesses the system interface, clicks the "Select File" button, selects a PDF, CAD, or image file from their local storage, and presses the "Submit" button.

[0737] Input: Drawing data in PDF, CAD, or image format

[0738] Output: Drawing data uploaded to the server

[0739] Step 2: Analyze the drawing data

[0740] The server receives the uploaded drawing data and begins analyzing it. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, it identifies room boundaries, corridor widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[0741] Input: Uploaded drawing data

[0742] Data processing: Analysis of layer information, application of shape recognition algorithms

[0743] Output: A virtual map that identifies room boundaries, corridor widths, and obstacle locations

[0744] Step 3: Generate cleaning routes

[0745] The server automatically generates an optimal cleaning route based on the analysis results. This process uses path-finding algorithms such as Dijkstra and A (Aster). Specifically, the route is calculated to pass through each room only once, eliminating unnecessary movements, and optimizing the cleaning order of each area.

[0746] Input: A virtual map based on the analysis results

[0747] Data Computation: Applying Pathfinding Algorithms

[0748] Output: Optimized cleaning route

[0749] Step 4: Recognizing user emotions with the emotion engine

[0750] The server uses the generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when analyzing voice input, if the user says "I need help," the voice data is recognized. When analyzing facial expressions, if the user shows their face to the camera, the emotion engine analyzes it and identifies their emotional state.

[0751] Input: User voice input, facial expression data

[0752] Data analysis: Analysis of voice data, analysis of facial expressions

[0753] Output: User's emotional state (e.g., stress, anxiety)

[0754] Step 5: Present and coordinate cleaning routes

[0755] The user checks the cleaning route generated by the server on their device. Specifically, the user previews the cleaning route on the screen, and the emotion engine recognizes the user's emotional state and adjusts the interface accordingly. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information.

[0756] Input: Cleaning route provided by the server, user's emotional state

[0757] Data display: interface adjustment, route preview display

[0758] Output: Cleaning route confirmed by the user

[0759] Step 6: User adjusts cleaning route

[0760] Users can adjust the cleaning route by dragging and dropping. Specifically, the user grabs the route on the screen and drags and drops it to another area. During this process, the emotion engine monitors the user's voice and facial expressions, and if the user is unsure, it displays optimal suggestions in the form of tooltips.

[0761] Input: User's cleaning route adjustment operations, emotion engine monitoring data

[0762] Data processing: Route changes, display of suggested content

[0763] Output: Adjusted cleaning route

[0764] Step 7: Start cleaning

[0765] The server then sends the final cleaning route to the cleaning robot. Specifically, the server transmits the final route to the cleaning robot via radio waves, and the robot begins to operate based on that route. The robot uses sensors to confirm its own location and proceeds with cleaning along the specified path.

[0766] Input: Finalized cleaning route

[0767] Data transmission: Sending cleaning route

[0768] Output: Cleaning robot starts operating

[0769] The above are the processing steps of the program for this system. By explaining the specific operations, inputs, and outputs in detail for each step, the function of the entire system can be clarified.

[0770] (Application example 2)

[0771] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0772] In modern logistics centers, product placement changes and the creation and adjustment of transport routes are frequent, placing an increased burden on operators. Furthermore, operators who are fatigued or stressed are unable to perform optimal tasks, which is a problem. Therefore, a new system is needed to improve the efficiency of logistics processes and reduce the burden on operators.

[0773] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0774] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for analyzing the user's voice input and facial expression to recognize emotions, means for adjusting the interface based on the recognized emotions, and means for transmitting the final cleaning route to an automated guided vehicle and starting the transportation work. This enables transportation work within a logistics center to be performed efficiently, reducing the burden on operators and improving the quality of work.

[0775] Creating definition statements

[0776] "Drawing data" refers to layout information for buildings and facilities that indicates the location of cleaning areas and obstacles, and is provided in formats such as PDF, CAD, and image formats.

[0777] A "cleaning route" is a path for an object to travel efficiently within a specified area, and is automatically generated taking into account the shortest distance and time.

[0778] "Means for recognizing emotions" refers to technology that analyzes a user's voice input and facial expressions to determine their emotional state.

[0779] An "interface" refers to the operating screen and operating method that allows a user to interact with a system, and which adapts to the user's emotional state.

[0780] A "user" is a person who operates the system, uploads drawing data, and coordinates cleaning routes.

[0781] An "automated transport vehicle" is an autonomous vehicle used to efficiently transport goods within a logistics center, and includes AGVs (automated guided vehicles).

[0782] A "server" is a central computer that stores and processes various data, and provides processing functions such as analyzing drawing data and recognizing emotions.

[0783] MODE FOR CARRYING OUT THE INVENTION

[0784] This invention is a system for efficiently operating automated guided vehicles in a logistics center, which automatically generates optimal transport routes based on drawing data and product placement information, and uses an emotion engine to recognize and respond to the emotional state of the operator. The details are explained below.

[0785] System configuration

[0786] 1. Inputting drawing data

[0787] Users use their terminals to upload drawing data showing the layout and product placement within the distribution center to the server, including PDF, CAD, image formats, etc. For example, a user uploads the latest floor plan of a distribution center in PDF format.

[0788] 2. Analysis of drawing data

[0789] The server analyzes the uploaded drawing data and identifies the product placement location, movement path, and location of obstacles. This analysis process uses a drawing analysis engine such as OpenCV.

[0790] 3. Generate transport routes

[0791] The server automatically generates the optimal transport route based on the identified product locations and obstacles. The designed route is created taking into account the shortest distance and time to move products efficiently. For example, it calculates the route for AGVs to efficiently transport products within a logistics center.

[0792] 4. Emotion engine recognizes operator emotions

[0793] The server analyzes the operator's voice input and facial expressions to recognize emotions, for example, using emotion recognition software such as Microsoft Azure Cognitive Services to determine the operator's level of fatigue or stress.

[0794] 5. Delivery route presentation and adjustment interface

[0795] The user can check the delivery route generated by the server on their device. At this time, the emotion engine adjusts the interface based on the operator's emotional state. For example, if the user is tired, the interface will be adjusted to be simple and intuitive.

[0796] 6. Operators adjust transport routes

[0797] Users can fine-tune delivery routes using drag-and-drop operations. The emotion engine monitors the user's emotional state and provides appropriate suggestions and instructions. For example, when a user fine-tunes a part of a delivery route, the system provides optimal suggestions.

[0798] 7. Sending the transport route and starting the transport operation

[0799] The server then sends the final determined transport route to the automated guided vehicle (AGV). The AGV then follows the received route and transports the products efficiently within the specified area. This improves the efficiency of transport work and reduces the burden on the operator.

[0800] Hardware and software used

[0801] Hardware: Smartphones, tablets, smart glasses, AGVs (automated guided vehicles)

[0802] Software: Emotion recognition software (Microsoft Azure Cognitive Services), AGV control software, drawing analysis engine (OpenCV)

[0803] Examples and prompts

[0804] As a concrete example, at a certain logistics center, setting up transport routes was taking a lot of time and effort. With this system, managers can easily upload drawing data and automatically generate optimal transport routes. Examples of prompt sentences include the following:

[0805] Prompt Sentence Examples

[0806] "Upload the latest layout drawings to generate the optimal transport route."

[0807] By using this system, it is expected that transportation operations at logistics centers will be carried out more efficiently, reducing the burden on operators and improving the quality of work.

[0808] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0809] Program processing flow

[0810] Specific explanation of processing steps

[0811] Step 1:

[0812] The user uploads drawing data showing the layout of the distribution center and the placement of products from their terminal to the server. The input data can be a PDF, CAD, or image file. The server receives this drawing data and prepares it for the next step.

[0813] Step 2:

[0814] The server analyzes the uploaded drawing data. For example, it uses a drawing analysis engine such as OpenCV to identify product placement locations, movement paths, and the location of obstacles. The input data is the drawing data, and the output data is cleaning areas and location information of obstacles. This becomes the base data for generating subsequent transport routes.

[0815] Step 3:

[0816] The server automatically generates the optimal transport route based on the analysis results. Here, the route is calculated taking into account the shortest distance and time to move products efficiently. The input data is the location information of the identified cleaning area and obstacles, and the output data is the initial transport route.

[0817] Step 4:

[0818] The server uses an emotion engine to analyze the operator's voice input and facial expressions to recognize emotions. For example, it uses Microsoft Azure Cognitive Services to determine the operator's emotional state. The input data is the operator's voice and facial expression data, and the output data is the recognized emotional state.

[0819] Step 5:

[0820] The server presents the generated delivery route to the operator on the terminal and adjusts the interface based on the emotional state recognized by the emotion engine. The input data are the initial delivery route and the emotional state, and the output data is the delivery route displayed in the adjusted interface.

[0821] Step 6:

[0822] The user fine-tunes the delivery route using drag-and-drop operations on the terminal. The input data are the instructions the operator will use, and the output data is the fine-tuned delivery route. During this process, the server monitors the operator's emotional state and makes appropriate suggestions and instructions.

[0823] Step 7:

[0824] The server sends the final determined transport route to the automated guided vehicle (AGV). The input data is the finely adjusted final transport route, and the output data is the control command sent to the AGV. The AGV transports the goods efficiently within the specified range based on this command. This starts the transport work and allows it to be carried out efficiently.

[0825] Throughout these processing steps, a system is realized that allows transportation operations within the logistics center to be carried out efficiently and reduces the burden on operators.

[0826] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0827] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0828] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0829] [Third embodiment]

[0830] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0831] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0832] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0833] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0834] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0835] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0836] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0837] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0838] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0839] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0840] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0841] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0842] This invention is a system for streamlining cleaning work within buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. This system is particularly effective in facilities with large floors or complex layouts.

[0843] System configuration

[0844] 1. Inputting drawing data

[0845] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[0846] 2. Analysis of drawing data

[0847] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[0848] 3. Cleaning route generation

[0849] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[0850] 4. Present and coordinate cleaning routes

[0851] The generated cleaning route is presented to the user, who can check it on their device. If necessary, they can fine-tune the cleaning route using drag-and-drop operations. In this way, the user can make final adjustments to the route to suit the actual cleaning situation.

[0852] 5. Send the cleaning route and start cleaning

[0853] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route. This reduces the workload of cleaning staff and improves work efficiency.

[0854] Program processing

[0855] The program processing of this system will be explained in natural language below.

[0856] Uploading drawing data

[0857] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[0858] Drawing data analysis

[0859] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[0860] Cleaning route generation

[0861] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[0862] Present and coordinate cleaning routes

[0863] The user can view the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the user can adjust it using drag and drop. This allows the user to exclude areas that do not need cleaning or set specific areas for additional cleaning.

[0864] Cleaning work begins

[0865] The server sends the final cleaning route to the cleaning robot, which then starts cleaning based on this route. The robot follows the received route and efficiently cleans within the specified area. For example, the robot can clean the hallway and each room accurately.

[0866] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0867] The processing flow will be explained below.

[0868] Step 1:

[0869] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[0870] Step 2:

[0871] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and converts it into the appropriate format for use in the next analysis step.

[0872] Step 3:

[0873] The server begins analyzing the drawing data. Specifically, in the case of PDF or CAD data, it reads the drawing's layer information and identifies area information (rooms and corridors) and obstacles (furniture and pillars). In the case of image data, it uses image analysis algorithms to extract similar area information.

[0874] Step 4:

[0875] Based on the analysis results, the server automatically generates an optimal cleaning route that takes into account the cleaning area and obstacles. Specifically, it calculates a route that passes through each area only once and minimizes unnecessary movement.

[0876] Step 5:

[0877] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[0878] Step 6:

[0879] Users can fine-tune the proposed cleaning route by dragging and dropping. Users can exclude unnecessary areas or adjust to clean specific areas. Once adjustments are complete, users click the confirm button.

[0880] Step 7:

[0881] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[0882] Step 8:

[0883] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[0884] Step 9:

[0885] The cleaning robot will then start cleaning according to the received cleaning route. The cleaning robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[0886] Example 1

[0887] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0888] Conventional cleaning work requires a lot of time and effort, and it is difficult to efficiently set cleaning routes, especially in facilities with large floors or complex layouts. Furthermore, manual cleaning route setting by cleaning staff lacks flexibility and often results in inconsistent work quality. To solve these problems, a system was needed that could efficiently analyze drawing data and automatically generate and adjust cleaning routes.

[0889] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0890] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for sending the final cleaning route to a cleaning device and starting cleaning work, means for sending a file to the server and displaying the upload progress, means for analyzing the drawing data based on layer information and coordinate data, means for converting the cleaning route into JSON format and sending it to the user's terminal, means for displaying a route preview and allowing adjustment by drag and drop, and means for sending the generated optimal route to the API endpoint of the cleaning device. This allows users to easily operate the system, enabling efficient cleaning route setting and automation of actual cleaning work.

[0891] "Drawing data" refers to digital data such as PDFs, blueprints, and image formats that contain layout information for buildings and facilities.

[0892] "Cleaning route" refers to the path that a cleaning device follows within the area to be cleaned, including a path optimized for efficient cleaning.

[0893] "Cleaning area" refers to the specific area, such as a room, hallway, or corridor, where cleaning work is performed.

[0894] "Obstacles" refers to objects such as furniture or pillars that impede the movement of cleaning equipment.

[0895] "Cleaning equipment" refers to robots and devices that perform cleaning tasks automatically.

[0896] "User" refers to the person who operates the system to set and adjust cleaning routes.

[0897] "Terminal" refers to an information device such as a computer or tablet that is operated by a user.

[0898] "Server" refers to a central computer system that performs processes such as analyzing drawing data and automatically generating cleaning routes.

[0899] "Upload" refers to the operation of sending data from a terminal to a server.

[0900] "Analysis" refers to the process of extracting necessary information from drawing data and identifying cleaning areas and obstacles.

[0901] "Drag and drop" refers to an operation in which a user moves an object on the screen using a mouse or touch operation.

[0902] "JSON format" refers to a lightweight data exchange format for structuring and storing data.

[0903] "API Endpoint" refers to a specific interface for communicating with a cleaning device.

[0904] "Preview" refers to the state in which the generated cleaning route can be visually displayed and confirmed.

[0905] The present invention is a system for improving the efficiency of cleaning work within buildings and facilities, receiving drawing data as input and automatically generating cleaning routes. This system is particularly effective in facilities with large floor areas or complex layouts. Specific embodiments of this system are described below.

[0906] The system consists of three main components: a server, a terminal, and cleaning equipment. Drawing data is provided in PDF, blueprint, image format, etc. The server uses analysis tools such as Adobe Acrobat SDK to extract the necessary information from the drawing data.

[0907] Uploading drawing data

[0908] The user uses the device to upload the drawing data of the area to be cleaned to the server. Specifically, the user opens the file selection window on the device, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The device sends the selected file to the server and displays the upload progress on the screen.

[0909] Drawing data analysis

[0910] The server analyzes the received drawing data and identifies the locations of rooms, passageways, and obstacles. The analysis uses Adobe Acrobat SDK and other tools, making full use of the PDF file's layer information and image processing technology to extract the necessary data. For example, room boundaries, passageway widths, and the locations of furniture and pillars can be identified.

[0911] Cleaning route generation

[0912] Based on the analysis results, the server automatically generates an optimal cleaning route. The server uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The result is converted into JSON format and sent to the user's device.

[0913] Present and coordinate cleaning routes

[0914] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," visually displays a preview of the route and allows the user to fine-tune the route using drag-and-drop operations. Users can exclude areas that do not require cleaning and set additional areas to be cleaned.

[0915] Cleaning work begins

[0916] The server sends the finalized cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts cleaning work based on the received route information. The cleaning device moves efficiently and cleans within the specified area.

[0917] Examples of specific examples and prompts

[0918] A specific example of this system is shown below.

[0919] Example of uploading drawing data

[0920] The user selects the PDF file of their office floor on the terminal and clicks the "Upload" button.

[0921] Specific examples of drawing data analysis

[0922] The server receives "Office Floor.pdf" and uses software called "PDF Analysis Tool" to analyze the PDF layers and identify the locations of rooms, corridors, and obstacles to be cleaned.

[0923] Example of cleaning route generation

[0924] The server uses an algorithm to calculate the shortest route from "Room A" to "Room B" and generates a route.

[0925] Examples of cleaning route presentation and adjustment

[0926] The user checks the route presented in the application on the device and removes the unnecessary room, "C�D Room," from the route.

[0927] Specific examples of when cleaning work begins

[0928] The server then sends the final route to the "cleaning robot," which then begins cleaning the designated area, efficiently cleaning without hitting any obstacles along the way.

[0929] Example prompts for generative AI models

[0930] "I've uploaded an office floor plan (PDF file). Please explain how the system generates optimal cleaning routes based on this floor plan and allows me to adjust the routes with drag and drop."

[0931] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[0932] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0933] Step 1: Upload the drawing data

[0934] The user uses the terminal to upload the drawing data to the server. Specifically, the user opens the file selection window on the terminal, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The input here is the PDF file selected by the user, and that file is sent to the server. The terminal splits the file and sends it to the server, and displays the upload progress.

[0935] Input: PDF file

[0936] Output: Drawing data uploaded to the server

[0937] Specific behavior:

[0938] The user selects "Office Floor.pdf."

[0939] The device splits the file into 10MB chunks and sends them to the server.

[0940] Displays an upload progress bar on the device screen.

[0941] Step 2: Analyze the drawing data

[0942] The server analyzes the received drawing data. During this process, the server uses analysis tools such as Adobe Acrobat SDK to extract the contents of the PDF file. Based on the layer information and coordinate data of the drawing data, the server identifies the locations of rooms, corridors, and obstacles.

[0943] Input: Drawing data uploaded to the server

[0944] Output: Analyzed cleaning area and obstacle information

[0945] Specific behavior:

[0946] The server opens "OfficeFloor.pdf".

[0947] The server scans the layer information to detect room boundaries.

[0948] The width of the aisle is measured and the position of obstacles (desks, chairs, etc.) is recorded as coordinate data.

[0949] Step 3: Generate cleaning routes

[0950] The server generates an optimal cleaning route based on the analysis results. Specifically, it uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The calculation results are converted into JSON format and sent to the user's device.

[0951] Input: Analyzed cleaning area and obstacle information

[0952] Output: Cleaning route information in JSON format

[0953] Specific behavior:

[0954] The server calculates a route that passes through each room once.

[0955] Convert the calculation results into JSON format.

[0956] The server logs the computation time and the optimality of the route.

[0957] Step 4: Present and coordinate cleaning routes

[0958] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," provides a visual preview of the route and allows users to fine-tune the route using drag-and-drop operations.

[0959] Input: Cleaning route information in JSON format

[0960] Output: The final cleaning route as adjusted by the user

[0961] Specific behavior:

[0962] The device receives route information in JSON format and displays it visually in a GUI.

[0963] The user clicks on "CD Room" to exclude it from the route.

[0964] After making adjustments, the user clicks the "Confirm" button to save the final route.

[0965] Step 5: Start cleaning

[0966] The server sends the final cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts work based on the received route information and efficiently cleans within the specified area.

[0967] Input: Final cleaning route adjusted by the user

[0968] Output: The cleaning task performed by the cleaning equipment

[0969] Specific behavior:

[0970] The server sends the route data to the cleaning equipment's control API.

[0971] The cleaning equipment analyzes the route data and generates a cleaning schedule.

[0972] The cleaning equipment begins cleaning within the designated area.

[0973] (Application example 1)

[0974] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0975] In today's vast and complex facilities and factories, manual cleaning is not only laborious and time-consuming, but also inefficient, resulting in a high likelihood of a lack of consistency across the entire cleaning area. Furthermore, designing and adjusting cleaning routes requires specialized knowledge, making it difficult for average employees. Therefore, a system that can automatically generate and easily adjust efficient and unified cleaning routes is desired.

[0976] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0977] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to a cleaning machine and starting cleaning work, means for identifying the location of work areas and equipment within a large-scale facility when analyzing the uploaded drawing data, means for displaying a preview of the cleaning route on a smart device and allowing the user to fine-tune the route by dragging and dropping, and means for confirming the cleaning route and transmitting the confirmed route to the cleaning machine, thereby making it possible to improve the efficiency and automation of cleaning work and provide user-friendly adjustment functions.

[0978] "Drawing data" refers to digital data that includes layout information for buildings and facilities, and is expressed in PDF, CAD, or image format.

[0979] "Input format" refers to the file format of the drawing data that the system receives from the user.

[0980] A "cleaning route" is a route designed to efficiently carry out cleaning work, and is set to pass through a specific area in an optimal order.

[0981] "Analysis" is the process of processing the received drawing data to identify cleaning areas and the location of obstacles.

[0982] The "cleaning area" refers to the area where cleaning work should be carried out, including rooms, corridors, etc.

[0983] "Obstacles" are objects or installations within the cleaning area that obstruct cleaning work, such as furniture and machinery.

[0984] "Presenting to the user" refers to the act of displaying the cleaning route automatically generated by the system to the user.

[0985] "Adjustable" refers to a state in which the user can change the cleaning route at will.

[0986] "Cleaning machine" means equipment for automatically performing cleaning tasks, including robots and other automated cleaning devices.

[0987] A "smart device" is a portable information terminal that can connect to the Internet and operate various applications, including smartphones and tablets.

[0988] The "drag-and-drop operation" refers to a function that allows users to intuitively operate elements on the screen of a smart device by moving the displayed elements using this operation.

[0989] "Preview" refers to a display that shows the user in advance what the final version will look like, allowing the user to check it before making any adjustments.

[0990] A "cleaning device" is hardware for performing cleaning tasks, and is synonymous with a cleaning machine, but has a more comprehensive meaning.

[0991] This invention is a system for streamlining cleaning work in large facilities with complex layouts. The system receives drawing data as input, analyzes it, automatically generates a cleaning route, and finally transmits the route to a cleaning robot to start cleaning work.

[0992] Drawing data input

[0993] Users upload facility layout data using their terminals. The layout data can be in PDF, CAD, or image format. This layout data includes layout information for large facilities such as buildings and factories.

[0994] Drawing data analysis

[0995] The server analyzes the received drawing data. This process identifies the work area and obstacles. For example, in the case of PDF files, it identifies room boundaries and corridor widths based on layer information. In the case of image formats, it uses image analysis software such as OpenCV to identify areas and obstacles.

[0996] Cleaning route generation

[0997] The server automatically generates an optimal cleaning route based on the analysis results. This route is designed to minimize distance and time so that the cleaning robot can move efficiently. For example, the route is calculated to pass through each room only once, eliminating unnecessary movements. This significantly improves the efficiency of cleaning work.

[0998] Present and coordinate cleaning routes

[0999] Users can preview the generated cleaning route on their device. On the smart device screen, users can easily fine-tune the route using drag-and-drop operations. For example, they can set specific areas to be cleaned additionally or exclude areas that do not need cleaning. In this way, users can make final adjustments to the route to suit the actual cleaning situation.

[1000] Cleaning work begins

[1001] The final cleaning route is sent from the server to the cleaning robot. The cleaning robot follows the received route and cleans efficiently within the specified area. For example, the robot can clean the hallway and also clean each room accurately. This reduces the workload of cleaning staff and significantly improves work efficiency.

[1002] Examples of concrete examples and prompts

[1003] This system is particularly effective for cleaning work in large factories, for example. If the factory layout is complex, using this system can significantly improve the efficiency of cleaning work.

[1004] You can input prompts like the following to the generative AI model:

[1005] "Please tell me how to efficiently clean a large factory with a complex layout. Please explain in detail, focusing in particular on how to automatically generate a route that will enable the cleaning robot to clean the shortest distance."

[1006] The system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[1007] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1008] Step 1:

[1009] A user uses a terminal to upload drawing data of the facility.

[1010] Input: Drawing data (PDF, CAD, image format)

[1011] Specific operation: The user selects the drawing data file using the file selection function of the terminal and clicks the upload button. The uploaded file is sent to the server.

[1012] Step 2:

[1013] The server analyzes the received drawing data and identifies the cleaning area and obstacles.

[1014] Input: Uploaded drawing data

[1015] Output: Analysis results (area and obstacle location information)

[1016] Specific operation: The server uses image analysis software such as OpenCV to detect edges and perform layer analysis on the drawing data to identify the boundaries of the cleaning area and the location of obstacles, thereby obtaining location information for rooms, corridors, and obstacles.

[1017] Step 3:

[1018] The server automatically generates the optimal cleaning route based on the analysis results.

[1019] Input: Analysis results (area and obstacle location information)

[1020] Output: Optimal cleaning route

[1021] Specific operation: The server generates a cleaning route using an algorithm that calculates the shortest and most efficient route through each cleaning area. If necessary, it optimizes the route using the Dijkstra algorithm or the A algorithm.

[1022] Step 4:

[1023] The server transmits the generated cleaning route to the terminal and presents it to the user.

[1024] Input: Optimal cleaning route

[1025] Output: Preview of cleaning route

[1026] Specific operation: The server converts the generated cleaning route into JSON format and sends it to the device. The device then displays a preview of the cleaning route on the smart device screen based on the received JSON data.

[1027] Step 5:

[1028] The user can check the cleaning route on the device and make fine adjustments as needed using drag-and-drop operations.

[1029] Input: Preview of cleaning route

[1030] Output: The final cleaning route adjusted by the user.

[1031] How it works: The user uses touch gestures on their smart device to fine-tune the displayed cleaning route with drag and drop, and the adjusted route is updated in real time.

[1032] Step 6:

[1033] The user finalizes the cleaning route and sends it to the server.

[1034] Input: Final cleaning route adjusted by the user

[1035] Output: Confirmed cleaning route

[1036] Specific operation: After the user finishes adjusting the route, he / she clicks the Confirm button, which sends the confirmed route from the device to the server.

[1037] Step 7:

[1038] The server transmits the finalized cleaning route to the cleaning robot.

[1039] Input: Confirmed cleaning route

[1040] Output: Route instructions sent to the cleaning robot

[1041] Specific operation: The server sends the determined route information to the cleaning robot through its communication interface, and the robot starts cleaning according to the received route information.

[1042] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1043] This invention is a system for improving the efficiency of cleaning work in buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides an interface and cleaning route suggestions that are adapted to the user.

[1044] System configuration

[1045] 1. Inputting drawing data

[1046] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[1047] 2. Analysis of drawing data

[1048] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[1049] 3. Cleaning route generation

[1050] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[1051] 4. User Emotion Recognition by Emotion Engine

[1052] The emotion engine recognizes the user's emotions by analyzing their voice input and facial expressions to determine their emotional state.

[1053] 5. Present and coordinate cleaning routes

[1054] The generated cleaning route is presented to the user, and the interface is adjusted based on the user's emotions as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will be changed to a simpler and more intuitive one.

[1055] 6. User-adjusted cleaning route

[1056] The user can check the cleaning route displayed on their device and make fine adjustments using drag-and-drop operations. The emotion engine also monitors the user's emotional state and adjusts the suggestions and presentation method as necessary.

[1057] 7. Send cleaning route and start cleaning

[1058] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route, thereby reducing the workload of cleaning staff and improving work efficiency.

[1059] Program processing

[1060] The program processing of this system will be explained in natural language below.

[1061] Uploading drawing data

[1062] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[1063] Drawing data analysis

[1064] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[1065] Cleaning route generation

[1066] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[1067] Recognizing user emotions with an emotion engine

[1068] The server analyzes the user's voice input and facial expressions to recognize emotions. For example, if the user is feeling anxious, it will recognize this and provide additional support to the user.

[1069] Present and coordinate cleaning routes

[1070] The user checks the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the interface adjusts according to emotions recognized by the emotion engine. For example, if the user is feeling stressed, only important information is highlighted.

[1071] User-adjusted cleaning route

[1072] Users can adjust the cleaning route by dragging and dropping, and the emotion engine monitors the user's emotional state to help them make the right decisions. For example, if the user is unsure, it will provide optimal suggestions.

[1073] Cleaning work begins

[1074] The server then sends the final cleaning route to the cleaning robot, which then follows the route and cleans efficiently within the designated area, reducing the burden on the user and improving the efficiency of cleaning work.

[1075] The processing flow will be explained below.

[1076] Step 1:

[1077] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[1078] Step 2:

[1079] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and selects the appropriate analysis algorithm.

[1080] Step 3:

[1081] The server analyzes the drawing data and identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.). For example, it uses the layer information in the PDF file to identify room boundaries and the width of corridors, and stores the area information in a database.

[1082] Step 4:

[1083] Based on the analysis results, the server automatically generates the optimal cleaning route. This is the path that the cleaning robot will follow most efficiently, and is designed taking into account the shortest distance and time. Specifically, it calculates a route that passes through each area only once, eliminating any unnecessary movement.

[1084] Step 5:

[1085] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[1086] Step 6:

[1087] The server uses an emotion engine to analyze the user's voice input and facial expressions to recognize emotions. Specifically, emotion data is collected by having the user speak into the camera or capture their facial expressions.

[1088] Step 7:

[1089] The server adjusts the display interface for the cleaning route based on the user's emotions. For example, if the user is feeling stressed, the interface is simplified and only important information is highlighted.

[1090] Step 8:

[1091] The user fine-tunes the presented cleaning route by dragging and dropping. Specifically, the user drags and moves points on the route with a mouse or touch screen. If the user feels stressed while making adjustments, the emotion engine suggests an easier way to do the adjustments.

[1092] Step 9:

[1093] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[1094] Step 10:

[1095] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[1096] Step 11:

[1097] The cleaning robot will then start cleaning according to the received cleaning route. The robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[1098] Step 12:

[1099] The server monitors the progress of the cleaning job and reports back to the user as needed, for example, providing information about which areas have been cleaned and where problems have arisen.

[1100] Example 2

[1101] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1102] While conventional cleaning robot systems have the ability to automatically generate cleaning routes based on drawing data, they lack the ability to provide an interface that takes user emotions into consideration and provide sufficient operability. As a result, when users are stressed or anxious, operating the system becomes cumbersome, making it difficult to perform efficient cleaning tasks. Furthermore, when users adjust the cleaning route, there is a lack of support during the process, making it difficult to determine the optimal cleaning route.

[1103] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1104] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning work, means for providing an interface adapted to the user using an emotion engine that recognizes the user's emotions, and means for suggesting adjustments to the cleaning route based on the user's emotions recognized by the emotion engine. This enables efficient and user-friendly cleaning work by providing an appropriate interface and support based on the user's emotional state.

[1105] "Drawing data" refers to digital data that contains layout information for buildings and facilities, and includes PDF, CAD, and image formats.

[1106] A "cleaning route" is a path along which a cleaning robot moves efficiently to perform cleaning work.

[1107] A "cleaning area" is the target area where the cleaning robot will actually clean, and includes rooms and corridors.

[1108] An "obstacle" is an element that exists within the cleaning area and obstructs the movement of the cleaning robot, such as furniture or pillars.

[1109] An "emotion engine" is software or hardware that analyzes a user's voice input and facial expressions to recognize their emotional state.

[1110] An "interface" is an operation screen or input means for a user to interact with a system, and includes both visual and operational elements.

[1111] "Suggestions" are revisions and guidance provided by the emotion engine to help users set the optimal cleaning route.

[1112] A "cleaning robot" is a mechanical device that automatically performs cleaning tasks according to a cleaning route set by a user.

[1113] "Analysis" refers to a series of data processing steps to identify cleaning areas and obstacles from drawing data.

[1114] The present invention is a system that realizes efficient cleaning work by automatically generating cleaning routes based on drawing data and providing an interface that responds to the user's emotional state. Each component and its specific operation will be described below.

[1115] Drawing data input

[1116] The user uses a device to upload blueprint data of the area to be cleaned to the server. This blueprint data is in PDF, CAD, or image format and includes layout information for the building or facility. Specifically, the user opens a web browser on the device, accesses the system interface, and clicks the "Select File" button to specify the file. The user then presses the "Send" button to send the blueprint data to the server.

[1117] Drawing data analysis

[1118] The server analyzes the received drawing data to identify cleaning areas and obstacles. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, the server identifies room boundaries, aisle widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[1119] Cleaning route generation

[1120] The server automatically generates the optimal cleaning route based on the analysis results. This generation process uses path-finding algorithms such as Dijkstra and A (A-star) to design a route that will clean efficiently and in the shortest distance. For example, the route is calculated to pass through each room only once and avoid unnecessary movement. The cleaning order of each area is also optimized.

[1121] Recognizing user emotions with an emotion engine

[1122] The server uses a generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when a user uses a microphone to say "I need help," the emotion engine analyzes the voice data and identifies emotions such as stress or anxiety. For facial expressions, when the user turns their face toward the camera, the AI ​​model analyzes their facial expressions to determine their emotional state.

[1123] Present and coordinate cleaning routes

[1124] The user checks the cleaning route generated by the server on their device. A preview of the cleaning route is displayed on the screen, and the interface adjusts according to the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information. The user can preview the cleaning route and make corrections as needed using drag-and-drop operations.

[1125] Cleaning work begins

[1126] The server then sends the final cleaning route to the cleaning robot, which then follows the received route to efficiently clean. The robot uses sensors to determine its own location and operates along the specified route, so any errors or obstacles that occur are automatically corrected.

[1127] Examples of specific examples and prompts

[1128] The user uploads a PDF file of the floor plan of the office building from their device to the server. The server receives the PDF file, identifies the locations of rooms, corridors, and obstacles, and generates a cleaning route. The user can check the generated cleaning route on the screen and adjust it as needed by dragging and dropping. Once the final route is determined, the server sends it to the cleaning robot, which then begins cleaning.

[1129] Prompt Sentence Examples

[1130] 1. "Please tell me the procedure for uploading the cleaning target drawing (PDF format)."

[1131] 2. "Please outline an algorithm for automatically generating cleaning routes."

[1132] 3. "Can you give me an example of how an interface can be adjusted to detect a user's stress level?"

[1133] This provides an interface and support based on the user's emotional state, enabling efficient and user-friendly cleaning work.

[1134] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1135] Step 1: Upload the drawing data

[1136] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. Specifically, the user opens a web browser, accesses the system interface, clicks the "Select File" button, selects a PDF, CAD, or image file from their local storage, and presses the "Submit" button.

[1137] Input: Drawing data in PDF, CAD, or image format

[1138] Output: Drawing data uploaded to the server

[1139] Step 2: Analyze the drawing data

[1140] The server receives the uploaded drawing data and begins analyzing it. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, it identifies room boundaries, corridor widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[1141] Input: Uploaded drawing data

[1142] Data processing: Analysis of layer information, application of shape recognition algorithms

[1143] Output: A virtual map that identifies room boundaries, corridor widths, and obstacle locations

[1144] Step 3: Generate cleaning routes

[1145] The server automatically generates an optimal cleaning route based on the analysis results. This process uses path-finding algorithms such as Dijkstra and A (Aster). Specifically, the route is calculated to pass through each room only once, eliminating unnecessary movements, and optimizing the cleaning order of each area.

[1146] Input: A virtual map based on the analysis results

[1147] Data Computation: Applying Pathfinding Algorithms

[1148] Output: Optimized cleaning route

[1149] Step 4: Recognizing user emotions with the emotion engine

[1150] The server uses the generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when analyzing voice input, if the user says "I need help," the voice data is recognized. When analyzing facial expressions, if the user shows their face to the camera, the emotion engine analyzes it and identifies their emotional state.

[1151] Input: User voice input, facial expression data

[1152] Data analysis: Analysis of voice data, analysis of facial expressions

[1153] Output: User's emotional state (e.g., stress, anxiety)

[1154] Step 5: Present and coordinate cleaning routes

[1155] The user checks the cleaning route generated by the server on their device. Specifically, the user previews the cleaning route on the screen, and the emotion engine recognizes the user's emotional state and adjusts the interface accordingly. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information.

[1156] Input: Cleaning route provided by the server, user's emotional state

[1157] Data display: interface adjustment, route preview display

[1158] Output: Cleaning route confirmed by the user

[1159] Step 6: User adjusts cleaning route

[1160] Users can adjust the cleaning route by dragging and dropping. Specifically, the user grabs the route on the screen and drags and drops it to another area. During this process, the emotion engine monitors the user's voice and facial expressions, and if the user is unsure, it displays optimal suggestions in the form of tooltips.

[1161] Input: User's cleaning route adjustment operations, emotion engine monitoring data

[1162] Data processing: Route changes, display of suggested content

[1163] Output: Adjusted cleaning route

[1164] Step 7: Start cleaning

[1165] The server then sends the final cleaning route to the cleaning robot. Specifically, the server transmits the final route to the cleaning robot via radio waves, and the robot begins to operate based on that route. The robot uses sensors to confirm its own location and proceeds with cleaning along the specified path.

[1166] Input: Finalized cleaning route

[1167] Data transmission: Sending cleaning route

[1168] Output: Cleaning robot starts operating

[1169] The above are the processing steps of the program for this system. By explaining the specific operations, inputs, and outputs in detail for each step, the function of the entire system can be clarified.

[1170] (Application example 2)

[1171] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1172] In modern logistics centers, product placement changes and the creation and adjustment of transport routes are frequent, placing an increased burden on operators. Furthermore, operators who are fatigued or stressed are unable to perform optimal tasks, which is a problem. Therefore, a new system is needed to improve the efficiency of logistics processes and reduce the burden on operators.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1174] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for analyzing the user's voice input and facial expression to recognize emotions, means for adjusting the interface based on the recognized emotions, and means for transmitting the final cleaning route to an automated guided vehicle and starting the transportation work. This enables transportation work within a logistics center to be performed efficiently, reducing the burden on operators and improving the quality of work.

[1175] Creating definition statements

[1176] "Drawing data" refers to layout information for buildings and facilities that indicates the location of cleaning areas and obstacles, and is provided in formats such as PDF, CAD, and image formats.

[1177] A "cleaning route" is a path for an object to travel efficiently within a specified area, and is automatically generated taking into account the shortest distance and time.

[1178] "Means for recognizing emotions" refers to technology that analyzes a user's voice input and facial expressions to determine their emotional state.

[1179] An "interface" refers to the operating screen and operating method that allows a user to interact with a system, and which adapts to the user's emotional state.

[1180] A "user" is a person who operates the system, uploads drawing data, and coordinates cleaning routes.

[1181] An "automated transport vehicle" is an autonomous vehicle used to efficiently transport goods within a logistics center, and includes AGVs (automated guided vehicles).

[1182] A "server" is a central computer that stores and processes various data, and provides processing functions such as analyzing drawing data and recognizing emotions.

[1183] MODE FOR CARRYING OUT THE INVENTION

[1184] This invention is a system for efficiently operating automated guided vehicles in a logistics center, which automatically generates optimal transport routes based on drawing data and product placement information, and uses an emotion engine to recognize and respond to the emotional state of the operator. The details are explained below.

[1185] System configuration

[1186] 1. Inputting drawing data

[1187] Users use their terminals to upload drawing data showing the layout and product placement within the distribution center to the server, including PDF, CAD, image formats, etc. For example, a user uploads the latest floor plan of a distribution center in PDF format.

[1188] 2. Analysis of drawing data

[1189] The server analyzes the uploaded drawing data and identifies the product placement location, movement path, and location of obstacles. This analysis process uses a drawing analysis engine such as OpenCV.

[1190] 3. Generate transport routes

[1191] The server automatically generates the optimal transport route based on the identified product locations and obstacles. The designed route is created taking into account the shortest distance and time to move products efficiently. For example, it calculates the route for AGVs to efficiently transport products within a logistics center.

[1192] 4. Emotion engine recognizes operator emotions

[1193] The server analyzes the operator's voice input and facial expressions to recognize emotions, for example, using emotion recognition software such as Microsoft Azure Cognitive Services to determine the operator's level of fatigue or stress.

[1194] 5. Delivery route presentation and adjustment interface

[1195] The user can check the delivery route generated by the server on their device. At this time, the emotion engine adjusts the interface based on the operator's emotional state. For example, if the user is tired, the interface will be adjusted to be simple and intuitive.

[1196] 6. Operators adjust transport routes

[1197] Users can fine-tune delivery routes using drag-and-drop operations. The emotion engine monitors the user's emotional state and provides appropriate suggestions and instructions. For example, when a user fine-tunes a part of a delivery route, the system provides optimal suggestions.

[1198] 7. Sending the transport route and starting the transport operation

[1199] The server then sends the final determined transport route to the automated guided vehicle (AGV). The AGV then follows the received route and transports the products efficiently within the specified area. This improves the efficiency of transport work and reduces the burden on the operator.

[1200] Hardware and software used

[1201] Hardware: Smartphones, tablets, smart glasses, AGVs (automated guided vehicles)

[1202] Software: Emotion recognition software (Microsoft Azure Cognitive Services), AGV control software, drawing analysis engine (OpenCV)

[1203] Examples and prompts

[1204] As a concrete example, at a certain logistics center, setting up transport routes was taking a lot of time and effort. With this system, managers can easily upload drawing data and automatically generate optimal transport routes. Examples of prompt sentences include the following:

[1205] Prompt Sentence Examples

[1206] "Upload the latest layout drawings to generate the optimal transport route."

[1207] By using this system, it is expected that transportation operations at logistics centers will be carried out more efficiently, reducing the burden on operators and improving the quality of work.

[1208] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1209] Program processing flow

[1210] Specific explanation of processing steps

[1211] Step 1:

[1212] The user uploads drawing data showing the layout of the distribution center and the placement of products from their terminal to the server. The input data can be a PDF, CAD, or image file. The server receives this drawing data and prepares it for the next step.

[1213] Step 2:

[1214] The server analyzes the uploaded drawing data. For example, it uses a drawing analysis engine such as OpenCV to identify product placement locations, movement paths, and the location of obstacles. The input data is the drawing data, and the output data is cleaning areas and location information of obstacles. This becomes the base data for generating subsequent transport routes.

[1215] Step 3:

[1216] The server automatically generates the optimal transport route based on the analysis results. Here, the route is calculated taking into account the shortest distance and time to move products efficiently. The input data is the location information of the identified cleaning area and obstacles, and the output data is the initial transport route.

[1217] Step 4:

[1218] The server uses an emotion engine to analyze the operator's voice input and facial expressions to recognize emotions. For example, it uses Microsoft Azure Cognitive Services to determine the operator's emotional state. The input data is the operator's voice and facial expression data, and the output data is the recognized emotional state.

[1219] Step 5:

[1220] The server presents the generated delivery route to the operator on the terminal and adjusts the interface based on the emotional state recognized by the emotion engine. The input data are the initial delivery route and the emotional state, and the output data is the delivery route displayed in the adjusted interface.

[1221] Step 6:

[1222] The user fine-tunes the delivery route using drag-and-drop operations on the terminal. The input data are the instructions the operator will use, and the output data is the fine-tuned delivery route. During this process, the server monitors the operator's emotional state and makes appropriate suggestions and instructions.

[1223] Step 7:

[1224] The server sends the final determined transport route to the automated guided vehicle (AGV). The input data is the finely adjusted final transport route, and the output data is the control command sent to the AGV. The AGV transports the goods efficiently within the specified range based on this command. This starts the transport work and allows it to be carried out efficiently.

[1225] Throughout these processing steps, a system is realized that allows transportation operations within the logistics center to be carried out efficiently and reduces the burden on operators.

[1226] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1227] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1228] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1229] [Fourth embodiment]

[1230] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1231] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1232] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1233] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1234] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1235] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1236] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1237] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1238] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1239] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1240] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1241] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1242] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1243] This invention is a system for streamlining cleaning work within buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. This system is particularly effective in facilities with large floors or complex layouts.

[1244] System configuration

[1245] 1. Inputting drawing data

[1246] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[1247] 2. Analysis of drawing data

[1248] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[1249] 3. Cleaning route generation

[1250] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[1251] 4. Present and coordinate cleaning routes

[1252] The generated cleaning route is presented to the user, who can check it on their device. If necessary, they can fine-tune the cleaning route using drag-and-drop operations. In this way, the user can make final adjustments to the route to suit the actual cleaning situation.

[1253] 5. Send the cleaning route and start cleaning

[1254] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route. This reduces the workload of cleaning staff and improves work efficiency.

[1255] Program processing

[1256] The program processing of this system will be explained in natural language below.

[1257] Uploading drawing data

[1258] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[1259] Drawing data analysis

[1260] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[1261] Cleaning route generation

[1262] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[1263] Present and coordinate cleaning routes

[1264] The user can view the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the user can adjust it using drag and drop. This allows the user to exclude areas that do not need cleaning or set specific areas for additional cleaning.

[1265] Cleaning work begins

[1266] The server sends the final cleaning route to the cleaning robot, which then starts cleaning based on this route. The robot follows the received route and efficiently cleans within the specified area. For example, the robot can clean the hallway and each room accurately.

[1267] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[1268] The processing flow will be explained below.

[1269] Step 1:

[1270] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[1271] Step 2:

[1272] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and converts it into the appropriate format for use in the next analysis step.

[1273] Step 3:

[1274] The server begins analyzing the drawing data. Specifically, in the case of PDF or CAD data, it reads the drawing's layer information and identifies area information (rooms and corridors) and obstacles (furniture and pillars). In the case of image data, it uses image analysis algorithms to extract similar area information.

[1275] Step 4:

[1276] Based on the analysis results, the server automatically generates an optimal cleaning route that takes into account the cleaning area and obstacles. Specifically, it calculates a route that passes through each area only once and minimizes unnecessary movement.

[1277] Step 5:

[1278] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[1279] Step 6:

[1280] Users can fine-tune the proposed cleaning route by dragging and dropping. Users can exclude unnecessary areas or adjust to clean specific areas. Once adjustments are complete, users click the confirm button.

[1281] Step 7:

[1282] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[1283] Step 8:

[1284] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[1285] Step 9:

[1286] The cleaning robot will then start cleaning according to the received cleaning route. The cleaning robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[1287] Example 1

[1288] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1289] Conventional cleaning work requires a lot of time and effort, and it is difficult to efficiently set cleaning routes, especially in facilities with large floors or complex layouts. Furthermore, manual cleaning route setting by cleaning staff lacks flexibility and often results in inconsistent work quality. To solve these problems, a system was needed that could efficiently analyze drawing data and automatically generate and adjust cleaning routes.

[1290] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1291] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for sending the final cleaning route to a cleaning device and starting cleaning work, means for sending a file to the server and displaying the upload progress, means for analyzing the drawing data based on layer information and coordinate data, means for converting the cleaning route into JSON format and sending it to the user's terminal, means for displaying a route preview and allowing adjustment by drag and drop, and means for sending the generated optimal route to the API endpoint of the cleaning device. This allows users to easily operate the system, enabling efficient cleaning route setting and automation of actual cleaning work.

[1292] "Drawing data" refers to digital data such as PDFs, blueprints, and image formats that contain layout information for buildings and facilities.

[1293] "Cleaning route" refers to the path that a cleaning device follows within the area to be cleaned, including a path optimized for efficient cleaning.

[1294] "Cleaning area" refers to the specific area, such as a room, hallway, or corridor, where cleaning work is performed.

[1295] "Obstacles" refers to objects such as furniture or pillars that impede the movement of cleaning equipment.

[1296] "Cleaning equipment" refers to robots and devices that perform cleaning tasks automatically.

[1297] "User" refers to the person who operates the system to set and adjust cleaning routes.

[1298] "Terminal" refers to an information device such as a computer or tablet that is operated by a user.

[1299] "Server" refers to a central computer system that performs processes such as analyzing drawing data and automatically generating cleaning routes.

[1300] "Upload" refers to the operation of sending data from a terminal to a server.

[1301] "Analysis" refers to the process of extracting necessary information from drawing data and identifying cleaning areas and obstacles.

[1302] "Drag and drop" refers to an operation in which a user moves an object on the screen using a mouse or touch operation.

[1303] "JSON format" refers to a lightweight data exchange format for structuring and storing data.

[1304] "API Endpoint" refers to a specific interface for communicating with a cleaning device.

[1305] "Preview" refers to the state in which the generated cleaning route can be visually displayed and confirmed.

[1306] The present invention is a system for improving the efficiency of cleaning work within buildings and facilities, receiving drawing data as input and automatically generating cleaning routes. This system is particularly effective in facilities with large floor areas or complex layouts. Specific embodiments of this system are described below.

[1307] The system consists of three main components: a server, a terminal, and cleaning equipment. Drawing data is provided in PDF, blueprint, image format, etc. The server uses analysis tools such as Adobe Acrobat SDK to extract the necessary information from the drawing data.

[1308] Uploading drawing data

[1309] The user uses the device to upload the drawing data of the area to be cleaned to the server. Specifically, the user opens the file selection window on the device, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The device sends the selected file to the server and displays the upload progress on the screen.

[1310] Drawing data analysis

[1311] The server analyzes the received drawing data and identifies the locations of rooms, passageways, and obstacles. The analysis uses Adobe Acrobat SDK and other tools, making full use of the PDF file's layer information and image processing technology to extract the necessary data. For example, room boundaries, passageway widths, and the locations of furniture and pillars can be identified.

[1312] Cleaning route generation

[1313] Based on the analysis results, the server automatically generates an optimal cleaning route. The server uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The result is converted into JSON format and sent to the user's device.

[1314] Present and coordinate cleaning routes

[1315] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," visually displays a preview of the route and allows the user to fine-tune the route using drag-and-drop operations. Users can exclude areas that do not require cleaning and set additional areas to be cleaned.

[1316] Cleaning work begins

[1317] The server sends the finalized cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts cleaning work based on the received route information. The cleaning device moves efficiently and cleans within the specified area.

[1318] Examples of specific examples and prompts

[1319] A specific example of this system is shown below.

[1320] Example of uploading drawing data

[1321] The user selects the PDF file of their office floor on the terminal and clicks the "Upload" button.

[1322] Specific examples of drawing data analysis

[1323] The server receives "Office Floor.pdf" and uses software called "PDF Analysis Tool" to analyze the PDF layers and identify the locations of rooms, corridors, and obstacles to be cleaned.

[1324] Example of cleaning route generation

[1325] The server uses an algorithm to calculate the shortest route from "Room A" to "Room B" and generates a route.

[1326] Examples of cleaning route presentation and adjustment

[1327] The user checks the route presented in the application on the device and removes the unnecessary room, "C�D Room," from the route.

[1328] Specific examples of when cleaning work begins

[1329] The server then sends the final route to the "cleaning robot," which then begins cleaning the designated area, efficiently cleaning without hitting any obstacles along the way.

[1330] Example prompts for generative AI models

[1331] "I've uploaded an office floor plan (PDF file). Please explain how the system generates optimal cleaning routes based on this floor plan and allows me to adjust the routes with drag and drop."

[1332] In this way, the system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[1333] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1334] Step 1: Upload the drawing data

[1335] The user uses the terminal to upload the drawing data to the server. Specifically, the user opens the file selection window on the terminal, selects the PDF file of the area to be cleaned, and clicks the "Upload" button. The input here is the PDF file selected by the user, and that file is sent to the server. The terminal splits the file and sends it to the server, and displays the upload progress.

[1336] Input: PDF file

[1337] Output: Drawing data uploaded to the server

[1338] Specific behavior:

[1339] The user selects "Office Floor.pdf."

[1340] The device splits the file into 10MB chunks and sends them to the server.

[1341] Displays an upload progress bar on the device screen.

[1342] Step 2: Analyze the drawing data

[1343] The server analyzes the received drawing data. During this process, the server uses analysis tools such as Adobe Acrobat SDK to extract the contents of the PDF file. Based on the layer information and coordinate data of the drawing data, the server identifies the locations of rooms, corridors, and obstacles.

[1344] Input: Drawing data uploaded to the server

[1345] Output: Analyzed cleaning area and obstacle information

[1346] Specific behavior:

[1347] The server opens "OfficeFloor.pdf".

[1348] The server scans the layer information to detect room boundaries.

[1349] The width of the aisle is measured and the position of obstacles (desks, chairs, etc.) is recorded as coordinate data.

[1350] Step 3: Generate cleaning routes

[1351] The server generates an optimal cleaning route based on the analysis results. Specifically, it uses a breadth-first search algorithm to calculate the shortest route for the cleaning robot to move efficiently. The calculation results are converted into JSON format and sent to the user's device.

[1352] Input: Analyzed cleaning area and obstacle information

[1353] Output: Cleaning route information in JSON format

[1354] Specific behavior:

[1355] The server calculates a route that passes through each room once.

[1356] Convert the calculation results into JSON format.

[1357] The server logs the computation time and the optimality of the route.

[1358] Step 4: Present and coordinate cleaning routes

[1359] The user can view the cleaning route generated by the server on their device. The device application, "CleanoRouteViewer," provides a visual preview of the route and allows users to fine-tune the route using drag-and-drop operations.

[1360] Input: Cleaning route information in JSON format

[1361] Output: The final cleaning route as adjusted by the user

[1362] Specific behavior:

[1363] The device receives route information in JSON format and displays it visually in a GUI.

[1364] The user clicks on "CD Room" to exclude it from the route.

[1365] After making adjustments, the user clicks the "Confirm" button to save the final route.

[1366] Step 5: Start cleaning

[1367] The server sends the final cleaning route to the cleaning device. Specifically, the server sends the final route data to the cleaning device's API endpoint. The cleaning device starts work based on the received route information and efficiently cleans within the specified area.

[1368] Input: Final cleaning route adjusted by the user

[1369] Output: The cleaning task performed by the cleaning equipment

[1370] Specific behavior:

[1371] The server sends the route data to the cleaning equipment's control API.

[1372] The cleaning equipment analyzes the route data and generates a cleaning schedule.

[1373] The cleaning equipment begins cleaning within the designated area.

[1374] (Application example 1)

[1375] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1376] In today's vast and complex facilities and factories, manual cleaning is not only laborious and time-consuming, but also inefficient, resulting in a high likelihood of a lack of consistency across the entire cleaning area. Furthermore, designing and adjusting cleaning routes requires specialized knowledge, making it difficult for average employees. Therefore, a system that can automatically generate and easily adjust efficient and unified cleaning routes is desired.

[1377] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1378] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to a cleaning machine and starting cleaning work, means for identifying the location of work areas and equipment within a large-scale facility when analyzing the uploaded drawing data, means for displaying a preview of the cleaning route on a smart device and allowing the user to fine-tune the route by dragging and dropping, and means for confirming the cleaning route and transmitting the confirmed route to the cleaning machine, thereby making it possible to improve the efficiency and automation of cleaning work and provide user-friendly adjustment functions.

[1379] "Drawing data" refers to digital data that includes layout information for buildings and facilities, and is expressed in PDF, CAD, or image format.

[1380] "Input format" refers to the file format of the drawing data that the system receives from the user.

[1381] A "cleaning route" is a route designed to efficiently carry out cleaning work, and is set to pass through a specific area in an optimal order.

[1382] "Analysis" is the process of processing the received drawing data to identify cleaning areas and the location of obstacles.

[1383] The "cleaning area" refers to the area where cleaning work should be carried out, including rooms, corridors, etc.

[1384] "Obstacles" are objects or installations within the cleaning area that obstruct cleaning work, such as furniture and machinery.

[1385] "Presenting to the user" refers to the act of displaying the cleaning route automatically generated by the system to the user.

[1386] "Adjustable" refers to a state in which the user can change the cleaning route at will.

[1387] "Cleaning machine" means equipment for automatically performing cleaning tasks, including robots and other automated cleaning devices.

[1388] A "smart device" is a portable information terminal that can connect to the Internet and operate various applications, including smartphones and tablets.

[1389] The "drag-and-drop operation" refers to a function that allows users to intuitively operate elements on the screen of a smart device by moving the displayed elements using this operation.

[1390] "Preview" refers to a display that shows the user in advance what the final version will look like, allowing the user to check it before making any adjustments.

[1391] A "cleaning device" is hardware for performing cleaning tasks, and is synonymous with a cleaning machine, but has a more comprehensive meaning.

[1392] This invention is a system for streamlining cleaning work in large facilities with complex layouts. The system receives drawing data as input, analyzes it, automatically generates a cleaning route, and finally transmits the route to a cleaning robot to start cleaning work.

[1393] Drawing data input

[1394] Users upload facility layout data using their terminals. The layout data can be in PDF, CAD, or image format. This layout data includes layout information for large facilities such as buildings and factories.

[1395] Drawing data analysis

[1396] The server analyzes the received drawing data. This process identifies the work area and obstacles. For example, in the case of PDF files, it identifies room boundaries and corridor widths based on layer information. In the case of image formats, it uses image analysis software such as OpenCV to identify areas and obstacles.

[1397] Cleaning route generation

[1398] The server automatically generates an optimal cleaning route based on the analysis results. This route is designed to minimize distance and time so that the cleaning robot can move efficiently. For example, the route is calculated to pass through each room only once, eliminating unnecessary movements. This significantly improves the efficiency of cleaning work.

[1399] Present and coordinate cleaning routes

[1400] Users can preview the generated cleaning route on their device. On the smart device screen, users can easily fine-tune the route using drag-and-drop operations. For example, they can set specific areas to be cleaned additionally or exclude areas that do not need cleaning. In this way, users can make final adjustments to the route to suit the actual cleaning situation.

[1401] Cleaning work begins

[1402] The final cleaning route is sent from the server to the cleaning robot. The cleaning robot follows the received route and cleans efficiently within the specified area. For example, the robot can clean the hallway and also clean each room accurately. This reduces the workload of cleaning staff and significantly improves work efficiency.

[1403] Examples of concrete examples and prompts

[1404] This system is particularly effective for cleaning work in large factories, for example. If the factory layout is complex, using this system can significantly improve the efficiency of cleaning work.

[1405] You can input prompts like the following to the generative AI model:

[1406] "Please tell me how to efficiently clean a large factory with a complex layout. Please explain in detail, focusing in particular on how to automatically generate a route that will enable the cleaning robot to clean the shortest distance."

[1407] The system of the present invention can be easily operated even by users without specialized knowledge, and realizes efficient setting of cleaning routes and automation of cleaning work.

[1408] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1409] Step 1:

[1410] A user uses a terminal to upload drawing data of the facility.

[1411] Input: Drawing data (PDF, CAD, image format)

[1412] Specific operation: The user selects the drawing data file using the file selection function of the terminal and clicks the upload button. The uploaded file is sent to the server.

[1413] Step 2:

[1414] The server analyzes the received drawing data and identifies the cleaning area and obstacles.

[1415] Input: Uploaded drawing data

[1416] Output: Analysis results (area and obstacle location information)

[1417] Specific operation: The server uses image analysis software such as OpenCV to detect edges and perform layer analysis on the drawing data to identify the boundaries of the cleaning area and the location of obstacles, thereby obtaining location information for rooms, corridors, and obstacles.

[1418] Step 3:

[1419] The server automatically generates the optimal cleaning route based on the analysis results.

[1420] Input: Analysis results (area and obstacle location information)

[1421] Output: Optimal cleaning route

[1422] Specific operation: The server generates a cleaning route using an algorithm that calculates the shortest and most efficient route through each cleaning area. If necessary, it optimizes the route using the Dijkstra algorithm or the A algorithm.

[1423] Step 4:

[1424] The server transmits the generated cleaning route to the terminal and presents it to the user.

[1425] Input: Optimal cleaning route

[1426] Output: Preview of cleaning route

[1427] Specific operation: The server converts the generated cleaning route into JSON format and sends it to the device. The device then displays a preview of the cleaning route on the smart device screen based on the received JSON data.

[1428] Step 5:

[1429] The user can check the cleaning route on the device and make fine adjustments as needed using drag-and-drop operations.

[1430] Input: Preview of cleaning route

[1431] Output: The final cleaning route adjusted by the user.

[1432] How it works: The user uses touch gestures on their smart device to fine-tune the displayed cleaning route with drag and drop, and the adjusted route is updated in real time.

[1433] Step 6:

[1434] The user finalizes the cleaning route and sends it to the server.

[1435] Input: Final cleaning route adjusted by the user

[1436] Output: Confirmed cleaning route

[1437] Specific operation: After the user finishes adjusting the route, he / she clicks the Confirm button, which sends the confirmed route from the device to the server.

[1438] Step 7:

[1439] The server transmits the finalized cleaning route to the cleaning robot.

[1440] Input: Confirmed cleaning route

[1441] Output: Route instructions sent to the cleaning robot

[1442] Specific operation: The server sends the determined route information to the cleaning robot through its communication interface, and the robot starts cleaning according to the received route information.

[1443] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1444] This invention is a system for improving the efficiency of cleaning work in buildings and facilities. It receives drawing data as input and automatically generates cleaning routes. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides an interface and cleaning route suggestions that are adapted to the user.

[1445] System configuration

[1446] 1. Inputting drawing data

[1447] Users use their devices to upload drawing data to the server. This drawing data can be in PDF, CAD, or image format, and includes layout information for buildings and facilities.

[1448] 2. Analysis of drawing data

[1449] The server analyzes the received drawing data and generates a cleaning route. The analysis process identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.).

[1450] 3. Cleaning route generation

[1451] The server automatically generates an optimal cleaning route based on the identified cleaning area and obstacles, and the route is designed to minimize distance and time for efficient cleaning.

[1452] 4. User Emotion Recognition by Emotion Engine

[1453] The emotion engine recognizes the user's emotions by analyzing their voice input and facial expressions to determine their emotional state.

[1454] 5. Present and coordinate cleaning routes

[1455] The generated cleaning route is presented to the user, and the interface is adjusted based on the user's emotions as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will be changed to a simpler and more intuitive one.

[1456] 6. User-adjusted cleaning route

[1457] The user can check the cleaning route displayed on their device and make fine adjustments using drag-and-drop operations. The emotion engine also monitors the user's emotional state and adjusts the suggestions and presentation method as necessary.

[1458] 7. Send cleaning route and start cleaning

[1459] Once the user has finalized the cleaning route, the server sends this route to the cleaning robot, which then carries out the cleaning work according to the received route, thereby reducing the workload of cleaning staff and improving work efficiency.

[1460] Program processing

[1461] The program processing of this system will be explained in natural language below.

[1462] Uploading drawing data

[1463] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. For example, the user selects a PDF file with a floor plan of a building and clicks the send button.

[1464] Drawing data analysis

[1465] The server receives the uploaded drawing data and begins analyzing it. During this process, it identifies the locations of rooms, corridors, and obstacles, and defines the cleaning area. For example, it uses layer information in the PDF file to identify room boundaries and the width of corridors.

[1466] Cleaning route generation

[1467] Based on the analysis results, the server automatically generates an optimal cleaning route. This is a path that allows the cleaning robot to move efficiently, and is designed taking into account the shortest distance and time. For example, the route is calculated to pass through each room only once, eliminating unnecessary movement.

[1468] Recognizing user emotions with an emotion engine

[1469] The server analyzes the user's voice input and facial expressions to recognize emotions. For example, if the user is feeling anxious, it will recognize this and provide additional support to the user.

[1470] Present and coordinate cleaning routes

[1471] The user checks the cleaning route generated by the server on their device. A preview of the route is displayed on the screen, and the interface adjusts according to emotions recognized by the emotion engine. For example, if the user is feeling stressed, only important information is highlighted.

[1472] User-adjusted cleaning route

[1473] Users can adjust the cleaning route by dragging and dropping, and the emotion engine monitors the user's emotional state to help them make the right decisions. For example, if the user is unsure, it will provide optimal suggestions.

[1474] Cleaning work begins

[1475] The server then sends the final cleaning route to the cleaning robot, which then follows the route and cleans efficiently within the designated area, reducing the burden on the user and improving the efficiency of cleaning work.

[1476] The processing flow will be explained below.

[1477] Step 1:

[1478] The user prepares the drawing data and uploads it to the server from their device. Specifically, the user opens the file selection screen on their device, selects a PDF, CAD, or image file, and clicks the send button.

[1479] Step 2:

[1480] The server receives and stores the uploaded drawing data, determines the format of the drawing data, and selects the appropriate analysis algorithm.

[1481] Step 3:

[1482] The server analyzes the drawing data and identifies cleaning areas (rooms, corridors, etc.) and obstacles (furniture, pillars, etc.). For example, it uses the layer information in the PDF file to identify room boundaries and the width of corridors, and stores the area information in a database.

[1483] Step 4:

[1484] Based on the analysis results, the server automatically generates the optimal cleaning route. This is the path that the cleaning robot will follow most efficiently, and is designed taking into account the shortest distance and time. Specifically, it calculates a route that passes through each area only once, eliminating any unnecessary movement.

[1485] Step 5:

[1486] The server presents the generated cleaning route to the user's device, where the user can preview the route in the device's browser or a dedicated application.

[1487] Step 6:

[1488] The server uses an emotion engine to analyze the user's voice input and facial expressions to recognize emotions. Specifically, emotion data is collected by having the user speak into the camera or capture their facial expressions.

[1489] Step 7:

[1490] The server adjusts the display interface for the cleaning route based on the user's emotions. For example, if the user is feeling stressed, the interface is simplified and only important information is highlighted.

[1491] Step 8:

[1492] The user fine-tunes the presented cleaning route by dragging and dropping. Specifically, the user drags and moves points on the route with a mouse or touch screen. If the user feels stressed while making adjustments, the emotion engine suggests an easier way to do the adjustments.

[1493] Step 9:

[1494] The server receives and stores the final cleaning route confirmed by the user, and prepares to send this final route to the cleaning robot.

[1495] Step 10:

[1496] The server transmits the final cleaning route to the cleaning robot. Specifically, it converts the route information into a data format that the robot can understand and transfers it to the robot via wireless communication or other means.

[1497] Step 11:

[1498] The cleaning robot will then start cleaning according to the received cleaning route. The robot will accurately follow the specified route and efficiently clean the entire cleaning area. The robot will use sensors to avoid obstacles as it cleans.

[1499] Step 12:

[1500] The server monitors the progress of the cleaning job and reports back to the user as needed, for example, providing information about which areas have been cleaned and where problems have arisen.

[1501] Example 2

[1502] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1503] While conventional cleaning robot systems have the ability to automatically generate cleaning routes based on drawing data, they lack the ability to provide an interface that takes user emotions into consideration and provide sufficient operability. As a result, when users are stressed or anxious, operating the system becomes cumbersome, making it difficult to perform efficient cleaning tasks. Furthermore, when users adjust the cleaning route, there is a lack of support during the process, making it difficult to determine the optimal cleaning route.

[1504] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1505] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning work, means for providing an interface adapted to the user using an emotion engine that recognizes the user's emotions, and means for suggesting adjustments to the cleaning route based on the user's emotions recognized by the emotion engine. This enables efficient and user-friendly cleaning work by providing an appropriate interface and support based on the user's emotional state.

[1506] "Drawing data" refers to digital data that contains layout information for buildings and facilities, and includes PDF, CAD, and image formats.

[1507] A "cleaning route" is a path along which a cleaning robot moves efficiently to perform cleaning work.

[1508] A "cleaning area" is the target area where the cleaning robot will actually clean, and includes rooms and corridors.

[1509] An "obstacle" is an element that exists within the cleaning area and obstructs the movement of the cleaning robot, such as furniture or pillars.

[1510] An "emotion engine" is software or hardware that analyzes a user's voice input and facial expressions to recognize their emotional state.

[1511] An "interface" is an operation screen or input means for a user to interact with a system, and includes both visual and operational elements.

[1512] "Suggestions" are revisions and guidance provided by the emotion engine to help users set the optimal cleaning route.

[1513] A "cleaning robot" is a mechanical device that automatically performs cleaning tasks according to a cleaning route set by a user.

[1514] "Analysis" refers to a series of data processing steps to identify cleaning areas and obstacles from drawing data.

[1515] The present invention is a system that realizes efficient cleaning work by automatically generating cleaning routes based on drawing data and providing an interface that responds to the user's emotional state. Each component and its specific operation will be described below.

[1516] Drawing data input

[1517] The user uses a device to upload blueprint data of the area to be cleaned to the server. This blueprint data is in PDF, CAD, or image format and includes layout information for the building or facility. Specifically, the user opens a web browser on the device, accesses the system interface, and clicks the "Select File" button to specify the file. The user then presses the "Send" button to send the blueprint data to the server.

[1518] Drawing data analysis

[1519] The server analyzes the received drawing data to identify cleaning areas and obstacles. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, the server identifies room boundaries, aisle widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[1520] Cleaning route generation

[1521] The server automatically generates the optimal cleaning route based on the analysis results. This generation process uses path-finding algorithms such as Dijkstra and A (A-star) to design a route that will clean efficiently and in the shortest distance. For example, the route is calculated to pass through each room only once and avoid unnecessary movement. The cleaning order of each area is also optimized.

[1522] Recognizing user emotions with an emotion engine

[1523] The server uses a generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when a user uses a microphone to say "I need help," the emotion engine analyzes the voice data and identifies emotions such as stress or anxiety. For facial expressions, when the user turns their face toward the camera, the AI ​​model analyzes their facial expressions to determine their emotional state.

[1524] Present and coordinate cleaning routes

[1525] The user checks the cleaning route generated by the server on their device. A preview of the cleaning route is displayed on the screen, and the interface adjusts according to the user's emotional state as recognized by the emotion engine. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information. The user can preview the cleaning route and make corrections as needed using drag-and-drop operations.

[1526] Cleaning work begins

[1527] The server then sends the final cleaning route to the cleaning robot, which then follows the received route to efficiently clean. The robot uses sensors to determine its own location and operates along the specified route, so any errors or obstacles that occur are automatically corrected.

[1528] Examples of specific examples and prompts

[1529] The user uploads a PDF file of the floor plan of the office building from their device to the server. The server receives the PDF file, identifies the locations of rooms, corridors, and obstacles, and generates a cleaning route. The user can check the generated cleaning route on the screen and adjust it as needed by dragging and dropping. Once the final route is determined, the server sends it to the cleaning robot, which then begins cleaning.

[1530] Prompt Sentence Examples

[1531] 1. "Please tell me the procedure for uploading the cleaning target drawing (PDF format)."

[1532] 2. "Please outline an algorithm for automatically generating cleaning routes."

[1533] 3. "Can you give me an example of how an interface can be adjusted to detect a user's stress level?"

[1534] This provides an interface and support based on the user's emotional state, enabling efficient and user-friendly cleaning work.

[1535] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1536] Step 1: Upload the drawing data

[1537] The user uses a terminal to upload the blueprint data of the area to be cleaned to the server. Specifically, the user opens a web browser, accesses the system interface, clicks the "Select File" button, selects a PDF, CAD, or image file from their local storage, and presses the "Submit" button.

[1538] Input: Drawing data in PDF, CAD, or image format

[1539] Output: Drawing data uploaded to the server

[1540] Step 2: Analyze the drawing data

[1541] The server receives the uploaded drawing data and begins analyzing it. For PDF files, the analysis uses layer information, while for CAD and image data, it uses shape recognition algorithms. Specifically, it identifies room boundaries, corridor widths, and obstacle locations from each layer and shape, and displays them on a virtual map.

[1542] Input: Uploaded drawing data

[1543] Data processing: Analysis of layer information, application of shape recognition algorithms

[1544] Output: A virtual map that identifies room boundaries, corridor widths, and obstacle locations

[1545] Step 3: Generate cleaning routes

[1546] The server automatically generates an optimal cleaning route based on the analysis results. This process uses path-finding algorithms such as Dijkstra and A (Aster). Specifically, the route is calculated to pass through each room only once, eliminating unnecessary movements, and optimizing the cleaning order of each area.

[1547] Input: A virtual map based on the analysis results

[1548] Data Computation: Applying Pathfinding Algorithms

[1549] Output: Optimized cleaning route

[1550] Step 4: Recognizing user emotions with the emotion engine

[1551] The server uses the generative AI model to analyze the user's voice input and facial expressions to recognize their emotional state. Specifically, when analyzing voice input, if the user says "I need help," the voice data is recognized. When analyzing facial expressions, if the user shows their face to the camera, the emotion engine analyzes it and identifies their emotional state.

[1552] Input: User voice input, facial expression data

[1553] Data analysis: Analysis of voice data, analysis of facial expressions

[1554] Output: User's emotional state (e.g., stress, anxiety)

[1555] Step 5: Present and coordinate cleaning routes

[1556] The user checks the cleaning route generated by the server on their device. Specifically, the user previews the cleaning route on the screen, and the emotion engine recognizes the user's emotional state and adjusts the interface accordingly. For example, if the user is feeling stressed, the interface will have a simple design that highlights only the important information.

[1557] Input: Cleaning route provided by the server, user's emotional state

[1558] Data display: interface adjustment, route preview display

[1559] Output: Cleaning route confirmed by the user

[1560] Step 6: User adjusts cleaning route

[1561] Users can adjust the cleaning route by dragging and dropping. Specifically, the user grabs the route on the screen and drags and drops it to another area. During this process, the emotion engine monitors the user's voice and facial expressions, and if the user is unsure, it displays optimal suggestions in the form of tooltips.

[1562] Input: User's cleaning route adjustment operations, emotion engine monitoring data

[1563] Data processing: Route changes, display of suggested content

[1564] Output: Adjusted cleaning route

[1565] Step 7: Start cleaning

[1566] The server then sends the final cleaning route to the cleaning robot. Specifically, the server transmits the final route to the cleaning robot via radio waves, and the robot begins to operate based on that route. The robot uses sensors to confirm its own location and proceeds with cleaning along the specified path.

[1567] Input: Finalized cleaning route

[1568] Data transmission: Sending cleaning route

[1569] Output: Cleaning robot starts operating

[1570] The above are the processing steps of the program for this system. By explaining the specific operations, inputs, and outputs in detail for each step, the function of the entire system can be clarified.

[1571] (Application example 2)

[1572] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1573] In modern logistics centers, product placement changes and the creation and adjustment of transport routes are frequent, placing an increased burden on operators. Furthermore, operators who are fatigued or stressed are unable to perform optimal tasks, which is a problem. Therefore, a new system is needed to improve the efficiency of logistics processes and reduce the burden on operators.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1575] In this invention, the server includes means for receiving drawing data as input and automatically generating a cleaning route, means for analyzing the drawing data and identifying cleaning areas and obstacles, means for presenting the automatically generated cleaning route to a user and allowing the user to adjust it, means for analyzing the user's voice input and facial expression to recognize emotions, means for adjusting the interface based on the recognized emotions, and means for transmitting the final cleaning route to an automated guided vehicle and starting the transportation work. This enables transportation work within a logistics center to be performed efficiently, reducing the burden on operators and improving the quality of work.

[1576] Creating definition statements

[1577] "Drawing data" refers to layout information for buildings and facilities that indicates the location of cleaning areas and obstacles, and is provided in formats such as PDF, CAD, and image formats.

[1578] A "cleaning route" is a path for an object to travel efficiently within a specified area, and is automatically generated taking into account the shortest distance and time.

[1579] "Means for recognizing emotions" refers to technology that analyzes a user's voice input and facial expressions to determine their emotional state.

[1580] An "interface" refers to the operating screen and operating method that allows a user to interact with a system, and which adapts to the user's emotional state.

[1581] A "user" is a person who operates the system, uploads drawing data, and coordinates cleaning routes.

[1582] An "automated transport vehicle" is an autonomous vehicle used to efficiently transport goods within a logistics center, and includes AGVs (automated guided vehicles).

[1583] A "server" is a central computer that stores and processes various data, and provides processing functions such as analyzing drawing data and recognizing emotions.

[1584] MODE FOR CARRYING OUT THE INVENTION

[1585] This invention is a system for efficiently operating automated guided vehicles in a logistics center, which automatically generates optimal transport routes based on drawing data and product placement information, and uses an emotion engine to recognize and respond to the emotional state of the operator. The details are explained below.

[1586] System configuration

[1587] 1. Inputting drawing data

[1588] Users use their terminals to upload drawing data showing the layout and product placement within the distribution center to the server, including PDF, CAD, image formats, etc. For example, a user uploads the latest floor plan of a distribution center in PDF format.

[1589] 2. Analysis of drawing data

[1590] The server analyzes the uploaded drawing data and identifies the product placement location, movement path, and location of obstacles. This analysis process uses a drawing analysis engine such as OpenCV.

[1591] 3. Generate transport routes

[1592] The server automatically generates the optimal transport route based on the identified product locations and obstacles. The designed route is created taking into account the shortest distance and time to move products efficiently. For example, it calculates the route for AGVs to efficiently transport products within a logistics center.

[1593] 4. Emotion engine recognizes operator emotions

[1594] The server analyzes the operator's voice input and facial expressions to recognize emotions, for example, using emotion recognition software such as Microsoft Azure Cognitive Services to determine the operator's level of fatigue or stress.

[1595] 5. Delivery route presentation and adjustment interface

[1596] The user can check the delivery route generated by the server on their device. At this time, the emotion engine adjusts the interface based on the operator's emotional state. For example, if the user is tired, the interface will be adjusted to be simple and intuitive.

[1597] 6. Operators adjust transport routes

[1598] Users can fine-tune delivery routes using drag-and-drop operations. The emotion engine monitors the user's emotional state and provides appropriate suggestions and instructions. For example, when a user fine-tunes a part of a delivery route, the system provides optimal suggestions.

[1599] 7. Sending the transport route and starting the transport operation

[1600] The server then sends the final determined transport route to the automated guided vehicle (AGV). The AGV then follows the received route and transports the products efficiently within the specified area. This improves the efficiency of transport work and reduces the burden on the operator.

[1601] Hardware and software used

[1602] Hardware: Smartphones, tablets, smart glasses, AGVs (automated guided vehicles)

[1603] Software: Emotion recognition software (Microsoft Azure Cognitive Services), AGV control software, drawing analysis engine (OpenCV)

[1604] Examples and prompts

[1605] As a concrete example, at a certain logistics center, setting up transport routes was taking a lot of time and effort. With this system, managers can easily upload drawing data and automatically generate optimal transport routes. Examples of prompt sentences include the following:

[1606] Prompt Sentence Examples

[1607] "Upload the latest layout drawings to generate the optimal transport route."

[1608] By using this system, it is expected that transportation operations at logistics centers will be carried out more efficiently, reducing the burden on operators and improving the quality of work.

[1609] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1610] Program processing flow

[1611] Specific explanation of processing steps

[1612] Step 1:

[1613] The user uploads drawing data showing the layout of the distribution center and the placement of products from their terminal to the server. The input data can be a PDF, CAD, or image file. The server receives this drawing data and prepares it for the next step.

[1614] Step 2:

[1615] The server analyzes the uploaded drawing data. For example, it uses a drawing analysis engine such as OpenCV to identify product placement locations, movement paths, and the location of obstacles. The input data is the drawing data, and the output data is cleaning areas and location information of obstacles. This becomes the base data for generating subsequent transport routes.

[1616] Step 3:

[1617] The server automatically generates the optimal transport route based on the analysis results. Here, the route is calculated taking into account the shortest distance and time to move products efficiently. The input data is the location information of the identified cleaning area and obstacles, and the output data is the initial transport route.

[1618] Step 4:

[1619] The server uses an emotion engine to analyze the operator's voice input and facial expressions to recognize emotions. For example, it uses Microsoft Azure Cognitive Services to determine the operator's emotional state. The input data is the operator's voice and facial expression data, and the output data is the recognized emotional state.

[1620] Step 5:

[1621] The server presents the generated delivery route to the operator on the terminal and adjusts the interface based on the emotional state recognized by the emotion engine. The input data are the initial delivery route and the emotional state, and the output data is the delivery route displayed in the adjusted interface.

[1622] Step 6:

[1623] The user fine-tunes the delivery route using drag-and-drop operations on the terminal. The input data are the instructions the operator will use, and the output data is the fine-tuned delivery route. During this process, the server monitors the operator's emotional state and makes appropriate suggestions and instructions.

[1624] Step 7:

[1625] The server sends the final determined transport route to the automated guided vehicle (AGV). The input data is the finely adjusted final transport route, and the output data is the control command sent to the AGV. The AGV transports the goods efficiently within the specified range based on this command. This starts the transport work and allows it to be carried out efficiently.

[1626] Throughout these processing steps, a system is realized that allows transportation operations within the logistics center to be carried out efficiently and reduces the burden on operators.

[1627] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1628] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1629] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1630] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1631] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1632] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1633] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1634] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1635] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1636] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1637] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1638] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1639] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1640] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1641] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1642] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1643] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1644] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1645] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1646] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1647] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1648] The following is further disclosed regarding the above embodiment.

[1649] (Claim 1)

[1650] A means for receiving drawing data as an input format and automatically generating a cleaning route;

[1651] A means for analyzing the drawing data and identifying cleaning areas and obstacles;

[1652] a means for presenting the automatically generated cleaning route to a user and allowing the user to adjust the route;

[1653] means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning;

[1654] A system including:

[1655] (Claim 2)

[1656] 2. The system according to claim 1, wherein the drawing data is in one of PDF, CAD, and image formats.

[1657] (Claim 3)

[1658] 2. The system of claim 1, wherein user adjustments of cleaning routes are performed using drag-and-drop operations.

[1659] "Example 1"

[1660] (Claim 1)

[1661] A means for receiving drawing data as an input format and automatically generating a cleaning route;

[1662] A means for analyzing the drawing data and identifying cleaning areas and obstacles;

[1663] a means for presenting the automatically generated cleaning route to a user and allowing the user to adjust the route;

[1664] means for transmitting the final cleaning route to the cleaning device to initiate the cleaning operation;

[1665] A means of sending the file to the server and displaying the upload progress;

[1666] A method for analyzing drawing data based on layer information and coordinate data,

[1667] A means for converting the cleaning route into JSON format and sending it to the user's terminal;

[1668] A means for displaying a preview of the route and allowing it to be adjusted by drag and drop;

[1669] A means for transmitting the generated optimal route to an API endpoint of the cleaning device;

[1670] A system including:

[1671] (Claim 2)

[1672] 2. The system according to claim 1, wherein the drawing data is in one of PDF, blueprint, and image format.

[1673] (Claim 3)

[1674] 10. The system of claim 1, wherein user adjustment of the cleaning route is performed through a visualized interface.

[1675] "Application Example 1"

[1676] (Claim 1)

[1677] A means for receiving drawing data as an input format and automatically generating a cleaning route;

[1678] A means for analyzing the drawing data and identifying cleaning areas and obstacles;

[1679] a means for presenting the automatically generated cleaning route to a user and allowing the user to adjust the route;

[1680] means for transmitting the final cleaning route to the cleaning machine to initiate the cleaning operation;

[1681] means for identifying work areas and equipment locations within the wide area facility upon analyzing the uploaded drawing data;

[1682] A way for users to preview cleaning routes on their smart devices and fine-tune them with drag-and-drop controls;

[1683] A system including means for determining a cleaning route and transmitting the determined route to a cleaning device.

[1684] (Claim 2)

[1685] 2. The system according to claim 1, wherein the drawing data is in one of PDF, CAD, and image formats.

[1686] (Claim 3)

[1687] 2. The system of claim 1, wherein user adjustments of cleaning routes are performed using drag-and-drop operations.

[1688] "Example 2: Combining Emotion Engines"

[1689] (Claim 1)

[1690] A means for receiving drawing data as an input format and automatically generating a cleaning route;

[1691] A means for analyzing the drawing data and identifying cleaning areas and obstacles;

[1692] a means for presenting the automatically generated cleaning route to a user and allowing the user to adjust the route;

[1693] means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning;

[1694] means for providing an interface adapted to the user using an emotion engine that recognizes the user's emotions;

[1695] means for suggesting adjustment of a cleaning route based on the user's emotions recognized by the emotion engine;

[1696] A system including:

[1697] (Claim 2)

[1698] 2. The system according to claim 1, wherein the drawing data is in one of PDF, CAD, and image formats.

[1699] (Claim 3)

[1700] 2. The system of claim 1, wherein user adjustments of cleaning routes are performed using drag-and-drop operations.

[1701] "Application example 2 when combining emotion engines"

[1702] (Claim 1)

[1703] A means for receiving drawing data as an input format and automatically generating a cleaning route;

[1704] A means for analyzing the drawing data and identifying cleaning areas and obstacles;

[1705] a means for presenting the automatically generated cleaning route to a user and allowing the user to adjust the route;

[1706] A means for recognizing emotions by analyzing a user's voice input and facial expressions;

[1707] means for adjusting the interface based on the recognized emotion;

[1708] means for transmitting the final cleaning route to the automated guided vehicle and causing the automated guided vehicle to start the transport operation;

[1709] A system including:

[1710] (Claim 2)

[1711] 2. The system according to claim 1, wherein the drawing data is in one of PDF, CAD, and image formats.

[1712] (Claim 3)

[1713] 2. The system of claim 1, wherein user adjustments of cleaning routes are performed using drag-and-drop operations. [Explanation of symbols]

[1714] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving drawing data as an input format and automatically generating a cleaning route; A means for analyzing the drawing data and identifying cleaning areas and obstacles; a means for presenting the automatically generated cleaning route to a user and allowing the user to adjust the route; means for transmitting the final cleaning route to the cleaning robot and causing it to start cleaning; A system including:

2. 2. The system according to claim 1, wherein the drawing data is in any one of PDF, CAD, and image formats.

3. 2. The system of claim 1, wherein user adjustment of the cleaning route is accomplished by a drag-and-drop operation.

Images