Support system for room cleaning inspection tasks

Abstract

This system provides support for guest room cleaning inspections, enabling more specific and detailed assistance to inspectors in their inspection work, and also supporting improvements in the inspectors' operations. [Solution] A room cleaning inspection support system in which a server 100 and smart glasses 200 worn by an inspector are connected via a network, wherein the server comprises a collected data management unit, a room information management unit, an item information management unit, an inspection work information management unit, a room patrol route determination unit, a unit for determining the time spent in each area, a gaze voice information determination unit, and an inappropriate operation determination unit.

Description

Technical Field

[0001] The present invention relates to a technology for assisting the inspection work of inspectors and checkers who check the finishing condition after a cleaning staff cleans the guest room in a facility such as a hotel.

Background Art

[0002] In facilities with accommodation such as hotels, inns, hospitals, and nursing homes, cleaning of guest rooms (including tidying up) is an essential task. In many cases, the facility operator outsources the cleaning work to cleaning staff in order to reduce the burden of cleaning costs. However, recently, there has also been an increasing number of cases where the facility operator shifts to or considers internalizing the direct employment of cleaning staff.

[0003] Ideally, the cleaning staff should perform perfect cleaning during cleaning. However, from the perspective of cost reduction and other aspects, part-time staff with high fluidity is the mainstream, and the number of foreign talents is also increasing.

[0004] Under such circumstances, after the cleaning staff finishes cleaning (for example, 30 minutes / room), an inspector or checker (the name is irrelevant, and it broadly includes cleaning inspection responsible persons who play an equivalent role. Hereinafter, it is collectively referred to as "inspector") checks the finishing condition (for example, 7 minutes / room). This is a business judgment to achieve the optimal cost-effectiveness through a double-check system by the inspector while maintaining the quality of the cleaning staff within the allowable limit.

[0005] However, conventionally, the role of the inspector has been played by a person with a certain period of experience as a cleaning staff, and it has required a considerable amount of time for training. However, the current facility operators do not have time for personnel training, and there is also a problem that the quality and level of the inspector are declining. Furthermore, while facility operators wanted inspectors to complete inspections of numerous guest rooms in a short amount of time to further improve operational efficiency, simply demanding shorter working hours presented a dilemma and risk: it would only result in an increase in areas that were not inspected (not looked at), leading to a further decline in the quality of inspection work.

[0006] Therefore, there was a need for a system that could improve the quality of inspection work by supporting the inspector's inspection process itself, and also support the inspector's efforts to improve their work. Furthermore, it was hoped that such a system would help establish the status of inspectors as professionals.

[0007] As a related technology, there is a management system that can access a floor database containing the room layout of accommodations and a team database containing the work information of the accommodation's cleaning staff team. This system has a first interface that acquires images of the surfaces of existing room indicators placed in linen rooms, etc., in real time via the network, a cleaning management unit that plans a cleaning schedule based on the information obtained by analyzing the acquired images, and a second interface that transmits information related to the planned cleaning schedule to the terminals of the cleaning staff team via the network, thereby improving the efficiency of cleaning operations. After cleaning is completed, a pre-set checklist according to the hotel's standards for the completed room is displayed on the quality inspector's tablet, allowing them to perform a quality check according to the checklist. The inspection results are stored as history in the inspection database, and relevant staff can refer to them on the inspection history screen. In addition to the checklist, it is also possible to send and store pre-check and post-check photos as images in the inspection management database as inspection information. In the future, it will also be possible to use an artificial intelligence (AI) module with machine learning to improve the quality of cleaning. (See Patent Document 1) [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Patent No. 7051147 [Overview of the project] [Problems that the invention aims to solve]

[0009] However, with the aforementioned technologies, the checklist items are only displayed on the quality inspector's tablet, and pre- and post-check photos are only stored as images in the inspection management database. This makes it difficult to provide more specific and detailed support for the inspector's inspection work itself, and it is also impossible to support the inspector's work improvement through feedback such as indoor patrol routes and the time spent in each area of ​​the room. This invention, based on the advanced knowledge, extensive experience, and high technical capabilities unique to a specialized company deeply familiar with practical applications, has reached a high level of practicality and boasts a level of realism and completeness as a service that could not be achieved at the level of an individual's idea.

[0010] The object of the present invention is to provide a room cleaning inspection support system that can provide more specific and detailed support for the inspector's inspection work itself, and can also support the inspector's work improvement. [Means for solving the problem]

[0011] The present invention provides a support system for assisting with inspection work during room cleaning. A room cleaning inspection support system in which a server and smart glasses are connected via a network, The aforementioned server, A data collection management unit receives data including at least one of image data, audio data, time data, and location data from smart glasses worn by an inspector who performs inspection work for the cleaning of guest rooms in the facility. A room information management unit that, using a predetermined learning model generated by machine learning based on training data by a predetermined artificial intelligence module, analyzes the data received from the smart glasses, extracts facility name and room identification information for the rooms of the facility, searches the storage unit of the server based on the facility name and room identification information, and displays at least one of the area layout information for each area of ​​the room and the ideal completed state image data for each area of ​​the room, which are stored in association with the facility name and room identification information, on the display unit of the smart glasses. The item information management unit analyzes the data received from the smart glasses using the learning model, extracts item identification information for the items in the guest room, searches the storage unit based on the item identification information, and displays at least one of the item name, information on where it is located in each area, and ideal completion state image data stored in association with the item identification information on the display unit of the smart glasses. An inspection work information management unit that, using the learning model, analyzes the data received from the smart glasses, extracts at least one of the facility room identification information and the item identification information, searches the storage unit based on at least one of the facility room identification information and the item identification information, and displays at least one of the optimal route information, optimal stay time information for each area, inspection work item information, and inspection work procedure information stored in association with at least one of the facility room identification information and the item identification information on the display unit of the smart glasses, An indoor patrol route determination unit that acquires the optimal route information stored in the memory unit, analyzes the data received from the smart glasses using the learning model, creates actual route information, and displays comparison information between the optimal route information and the actual route information on the display unit of the server, An area stay time determination unit that acquires optimal stay time information for each area stored in the memory unit, analyzes the data received from the smart glasses using the learning model, creates actual stay time information for each area, and displays comparison information between the optimal stay time information and the actual stay time information on the display unit of the server, A gaze-voice information determination unit acquires at least one of the inspection task item information, the inspection task procedure information, and the inspection task calling information stored in the memory unit, analyzes the data received from the smart glasses using the learning model, and if there are at least one of the missing inspection tasks or insufficient inspection tasks, displays on the display unit of the server that there are at least one of the missing inspection tasks or insufficient inspection tasks, An inappropriate operation determination unit acquires the inappropriate operation information stored in the memory unit, analyzes the data received from the smart glasses using the learning model, and if an inappropriate operation is detected, displays the presence of the inappropriate operation on the display unit of the server. It is characterized by having the following features.

[0012] By configuring the server in this way, the functions of the room information management unit, item information management unit, etc., can provide concrete and detailed support for the inspector's inspection work itself, and the functions of the server, such as the room patrol route determination unit, gaze and voice information determination unit, etc., can also support the improvement of the inspector's work. [Effects of the Invention]

[0013] According to the present invention, it is possible to provide a room cleaning inspection support system that can provide more specific and detailed support for the inspector's inspection work itself, and can also support the inspector's work improvement. [Brief explanation of the drawing]

[0014] [Figure 1] This diagram shows the configuration of a guest room cleaning inspection support system according to an embodiment of the present invention. [Figure 2] This is a functional block diagram of the server and smart glasses that constitute the guest room cleaning inspection support system according to an embodiment of the present invention. [Figure 3]This is a diagram showing an example of the table structure of each database in the inspection work support system for guest room cleaning according to an embodiment of the present invention. [Figure 4] This is a flowchart showing the processing operation of the inspection work support system for guest room cleaning according to an embodiment of the present invention. [Figure 5] This is a diagram showing an example of the display screen of the smart glasses in the inspection work support system for guest room cleaning according to an embodiment of the present invention. [Figure 6] This is a diagram showing an example of the display screen of the smart glasses in the inspection work support system for guest room cleaning according to an embodiment of the present invention. [Figure 7] This is a diagram showing an example of the display screen of the server in the inspection work support system for guest room cleaning according to an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the inspection work support system 10 for guest room cleaning is composed of a server 100 as an information processing device installed on the facility operation side or the like, and smart glasses 200 worn on the head by an inspector for use. The server 100 and the smart glasses 200 are each wirelessly (wired may also be used) connected by a predetermined network such as the Internet. Here, the server 100 is shown as a single information processing device, but the functions it has may also be realized by a plurality of information processing devices.

[0016] Server 100 is an information processing device controlled by a CPU (Central Processing Unit), equipped with storage means such as ROM (Read Only Memory) and RAM (Random Access Memory), and equipped with known input / output means. Examples include workstations, high-performance personal computers, tablet terminals, and mobile terminals. It may also be a cloud server. It may be operated by the service provider. Various known technologies may be applied to the detailed configuration, and explanations other than those mentioned above are omitted here.

[0017] Smart Glasses 200 is a wearable information processing device shaped like glasses and worn on the head like glasses. It is controlled by a CPU (Central Processing Unit) and equipped with memory means such as ROM (Read Only Memory) and RAM (Random Access Memory). It has a display and communication functions that can add and overlay information on the actual view, and it has a built-in microphone and camera. It can also overlay virtual objects and digital information onto the real world using augmented reality technology. Various known technologies may be applied to the detailed configuration, and explanations other than those mentioned above are omitted here. Note that the term "smart glasses" is not limited to information processing devices with equivalent functions.

[0018] Figure 2 shows the functional blocks of the server 100 and smart glasses 200 that constitute the guest room cleaning inspection support system 10 according to this embodiment.

[0019] Referring to Figure 2, the server 100 includes an equipment user management unit 101, a guest room information management unit 102, an item information management unit 103, an inspection work information management unit 104, a collected data management unit 105, a room patrol route determination unit 106, a unit 107 for determining the time spent in each area, a gaze-voice information determination unit 108, an inappropriate operation determination unit 109, a display unit 110, a storage unit 111, and a model generation unit 112. Each of these units may be implemented using known input / output interfaces, CPUs, HDDs, etc.

[0020] The device user management unit 101 has the function of storing the user ID (name and password, etc.) in the smart glasses identification information table and user identification information table of the device user information DB 1111 when registering a device or user for use in this service, and the function of performing device authentication and user authentication by searching the predetermined smart glasses identification information table, user identification information table, etc. in the device user information DB 1111 based on the smart glasses ID and user ID received from the smart glasses 200.

[0021] The guest room information management unit 102 has the function of storing and managing guest room information in the guest room information DB 1112, etc. Based on image data and voice data received from the smart glasses 200, an artificial intelligence (AI) module, etc., uses image and voice recognition technology, etc., to recognize and extract facility name guest room identification information, and has the function of displaying area layout information (for example, so-called floor plan information for each area of ​​the room, such as the bathroom, toilet, bed area, desk area, floor to be cleaned by a vacuum cleaner, etc.) and image data of the ideal completed state of each area on the display unit 202 of the smart glasses 200. For image and voice recognition, a predetermined program may be used, or a learning model 1116 generated by known machine learning, etc., by an artificial intelligence (AI) module (including the model generation unit 112 of the server 100 described later, and broadly including artificial intelligence (AI) modules mounted on other information processing devices, etc.) may be used (the same applies to "image and voice recognition", etc. in the following description).

[0022] The item information management unit 103 has the function of storing and managing item information for each area of ​​the room (items include, for example, cups, beds, pillows, hangers, and other furnishings) in the item information DB 1113, etc. Based on image data and audio data received from the smart glasses 200, an artificial intelligence (AI) module, etc., uses image and voice recognition technology, etc., to recognize and extract each item (item identification information) in the inspector's line of sight, etc., and has the function of displaying the item name, information on where it is located in each area of ​​the room, image data of the ideal completed state, etc., from the item information DB 1113 on the display unit 202 of the smart glasses 200. Furthermore, if the inspection work procedure information, etc., described later, is linked to the item identification information in the inspection work information DB 1114, the item information management unit 103 may also have the function of displaying the inspection work procedure information, etc., on the display unit 202 of the smart glasses 200.

[0023] The Inspection Operations Information Management Unit 104 has the function of storing and managing inspection operations information in the Inspection Operations Information DB 1114, etc. Based on image data and audio data received from the smart glasses 200, an artificial intelligence (AI) module, etc., uses image and voice recognition technology, etc., to recognize and extract facility room identification information and items (item identification information), and has the function of displaying optimal route information (a movement route for efficiently patrolling the room during inspection work), optimal dwell time data for each area, inspection work item information (for example, text data), inspection work procedure information (for example, still image data, video data, audio data, etc.), inappropriate operation information (for example, still image data, video data, audio data, etc.) etc. from the Inspection Operations Information DB 1114 on the display unit 202 of the smart glasses 200. The optimal route information described above may be pre-configured or dynamically determined in real time by an artificial intelligence (AI) module or the like. Furthermore, unlike inspectors, cleaning staff may prioritize areas with clutter, dirt, water damage, or fragile items, making it difficult or undesirable to set a predetermined route. The optimal dwell time data for each area mentioned above includes the minimum time required for inspection, which varies depending on the room type, and the necessary and sufficient time for effective inspection work is pre-set. Regarding the inspection items listed above, the inspection items performed by inspectors differ from the daily tasks handled by cleaning staff and include items related to maintenance and upkeep. For example, these include items related to periodic maintenance (every 3 or 6 months, etc.), and items for deciding on necessary repairs or maintenance (for example, checking if dust has accumulated in the air conditioner vents and deciding to carry out cleaning ahead of schedule). While checking whether the lights on or off on the stands are the tasks of cleaning staff, checking whether dust has accumulated inside the stands may be an inspection item performed by the inspector.

[0024] The data collection management unit 105 has the function of storing and managing data (image data in the inspector's line of sight, voice data from the inspector's speech, time data from a timer mechanism, location information data from a GPS mechanism, etc.) received and collected from smart glasses 200 worn by inspectors performing inspection work for the cleaning of guest rooms in the facility in the data collection DB 1115, etc.

[0025] The indoor patrol route determination unit 106 has the function of obtaining optimal route information stored in the inspection work information DB 1114, and generating actual route information based on location information data (by GPS mechanism, etc.) received from the smart glasses 200 stored in the collected data DB 1115, and outputting comparison information of both (optimal route information and actual route information). The optimal route information and actual route information may be displayed together vertically, horizontally, or superimposed on the display unit 110 of the server 100, as described later, or the differences between the two may be analyzed and displayed. This allows for confirmation of whether the inspector is adhering to the efficiency of the travel route, and can uncover individual movement habits that the person is unaware of, such as going to check an area that should only be checked once (for example, around the bed) twice. The indoor patrol route determination unit 106 may also recognize and extract various areas within the room from image data and audio data received from the smart glasses 200 using the learning model 1116, and generate actual route information based on that. The indoor patrol route determination unit 106 may also display the actual route information, which is accumulated along with the optimal route information, on the server 100 and / or smart glasses 200 to provide real-time warnings.

[0026] The area dwell time determination unit 107 has the function of obtaining optimal dwell time data for each area of ​​the room stored in the inspection work information DB 1114, and creating actual dwell time data for each area based on time data (by a timer mechanism, etc.) and location information data (by a GPS mechanism, etc.) received from the smart glasses 200 stored in the collected data DB 1115, and outputting comparison information of the two (optimal dwell time data and actual dwell time data for each area). The optimal dwell time data and actual dwell time data for each area may be displayed together on the display unit 110 of the server 100 (described later), either vertically, horizontally, or superimposed, or the difference between the two (for example, a positive or negative value) may be analyzed and displayed. In other words, faster is not always better, and simply rushing the inspection work could lead to the risk of cutting corners. Furthermore, the area dwell time determination unit 107 may recognize and extract each area of ​​the room from image data and audio data received from the smart glasses 200 using the learning model 1116, and create location information data based on that. Furthermore, when inspectors complete inspection tasks in each area of ​​a room, if they verbally announce "Toilet OK," "Check OK," etc., or perform pointing confirmations along with the announcements, the Smart Glasses 200 may be used to determine the completion of the inspection tasks based on image data and audio data received from the Smart Glasses 200, and this data may be used to measure the time spent in the room. The area dwell time determination unit 107 may also display the elapsed time data along with the optimal dwell time data on the server 100 and / or smart glasses 200 to provide real-time alerts.

[0027] The gaze-voice information determination unit 108 acquires inspection work item information, inspection work procedure information, and inspection work calling information stored in the inspection work information DB 1114. It also has a function to analyze and output warning information regarding missing or insufficient inspection work information based on image data (mainly images of the work in the direction the inspector's gaze is directed from their hands), voice data (mainly voices of the inspector's calling during work), time data (by a timer mechanism, etc.), and location information data (by a GPS mechanism, etc.) received from the smart glasses 200 and stored in the collected data DB 1115. The inspection work item information (not mandatory) and the missing or insufficient inspection work information may be displayed together vertically, horizontally, or superimposed on the display unit 110 of the server 100, as described later. When designated supervisors provide guidance to inspectors to improve their inspection work, they can provide evidence-based and reliable guidance using photographic evidence. For example, when checking for remaining hair in a bathroom, if it is necessary to check the back of the door or the ceiling, images of what the inspector's line of sight moves are recorded, making it clear whether or not they are looking at (checking) these areas. This reduces the likelihood of decreased motivation and interpersonal friction caused by discrepancies in whether or not the inspector is looking at something during guidance. Calling out during inspection work refers to, for example, saying "Toilet OK" or "Check OK" when completing the inspection of each area in the room. Even if pointing and confirming is required along with calling out, the image of the work, mainly from the inspector's gaze, also captures the pointing and confirming action. Therefore, the gaze-voice information determination unit 108 can determine not only whether calling out is missing, but also whether pointing and confirming is performed. The gaze-based audio information determination unit 108 may also display information about missed inspection tasks on the server 100 and / or smart glasses 200 to provide real-time warnings. The gaze-voice information determination unit 108 may issue a warning if the image data received from the smart glasses 200 does not correspond to the image data included in the inspection procedure information, or if the image data is present but only for an extremely short period of time (for example, a certain amount of time is required to look at a cup to check if it is clean). Furthermore, even if image data is present, if the duration is longer than a predetermined tolerance range compared to the optimal work time data based on the inspection procedure information, the unit may display a warning indicating that the inspection is inefficient (the inspection is inefficient). In addition, the unit may display the optimal work time data based on the inspection procedure information and the actual work time data together, either vertically, horizontally, or superimposed, or analyze and display the difference between the two (for example, a positive or negative value).

[0028] The inappropriate operation detection unit 109 acquires inappropriate operation information stored in the inspection work information DB 1114, and also has the function of analyzing inappropriate operation information and other data stored in the collected data DB 1115 from the smart glasses 200, such as image data (mainly images of the inspector's gaze in the direction from their hands during work), audio data (mainly the inspector's voice using their work-related names), time data (by a timer mechanism, etc.), and location information data (by a GPS mechanism, etc.), and outputting warning notifications. When providing guidance to improve inspectors' inspection work, it becomes possible to provide evidence-based and reliable guidance using photographic evidence. For example, actions such as brushing with a cloth (rag / duster) or putting one's foot on the bathtub (and wiping high places) are examples of inappropriate actions. In other words, from the perspective of whether dust has been properly removed, using a cloth in a brushing motion is an inappropriate action as it simply scatters dust. Designated instructors can provide guidance to inspectors on the proper use of tools and provide guidance from a safety perspective. The inappropriate operation detection unit 109 may also display inappropriate operation information, etc., on the server 100 and / or smart glasses 200 to provide real-time warnings.

[0029] The display unit 110 has the function of displaying various screens and the like on a display mechanism such as a display.

[0030] The memory unit 111 has the function of storing equipment user information DB 1111, guest room information DB 1112, item information DB 1113, inspection work information DB 1114, collected data DB 1115, learning model 1116, etc., which will be described later.

[0031] The model generation unit 112 has the function of generating a learning model 1116 based on predetermined learning data using a predetermined artificial intelligence (AI) module and storing it in the storage unit 111. The learning model 1116 may be generated by known machine learning or the like. The above-mentioned equipment user management unit 101, guest room information management unit 102, item information management unit 103, inspection work information management unit 104, collected data management unit 105, indoor patrol route determination unit 106, area dwell time determination unit 107, gaze information determination unit 108, inappropriate operation determination unit 109, etc. may perform detection and analysis of matches and mismatches between image data, audio data, etc. received from smart glasses 200 and each area, each item, other data, etc. in the room, using the learning model 1116 generated by the model generation unit 112 and stored in the storage unit 211. Note that multiple learning models 1116 may be prepared depending on the application. Furthermore, the model generation unit 112 may also have a function to store learning models 1116 generated by information processing devices other than the server 100, as well as programs for predetermined data analysis, etc., in the storage unit 111.

[0032] The device user information DB1111 is a database that has predetermined tables in which various data are linked to predetermined identification information, etc. An example of its table structure is shown in Figure 3. For example, there is a smart glasses identification information table, a user identification information table, and so on.

[0033] The guest room information DB1112 is a database having predetermined tables in which various data are linked and stored to predetermined identification information, and an example of its table structure is shown in Figure 3. For example, in the guest room information table, linked to the facility name and guest room identification information, area layout information, area identification image data (typical basic image data of each area in the room; for example, for the toilet area, this refers to image data of the toilet bowl, etc., which may be used as training data when generating a learning model 1116 by the artificial intelligence (AI) module), and ideal completed state image data for each area are stored. This also includes a method in which, instead of directly storing image data, etc., in a predetermined table, data such as the destination of a link to image data stored on a predetermined storage medium is stored in the predetermined table (the same applies to "image data," etc., in the following explanation).

[0034] The item information DB1113 is a database having predetermined tables in which various data are linked and stored to predetermined identification information, etc. An example of its table structure is shown in Figure 3. For example, the item information table stores the item name, information on where it is located in each area of ​​the room, item identification image data (basic image data including characteristic parts of each item, etc., which may be used as training data when generating a learning model 1116 by an artificial intelligence (AI) module), and image data of the ideal completed state of each item, etc., linked to the item identification information.

[0035] The Inspection Operations Information DB1114 is a database having predetermined tables in which various data are linked to predetermined identification information, etc., and an example of its table structure is shown in Figure 3. For example, the inspection operations table stores optimal route information linked to facility name and room identification information, optimal dwell time data for each area of ​​the room, inspection operations item information (for example, text data), inspection operations procedure information (which may be linked to inspection operations item information; for example, still image data, moving image data, audio data, etc., which may be used as training data when generating the AI ​​learning model 1116), inspection operations calling information (for example, text data, audio data, etc.), inappropriate operation information (for example, still image data, moving image data, audio data, etc., which may be used as training data when generating the AI ​​learning model 1116, etc.). Inspection work item information (which may be a large number, e.g., 300 or more items) and inspection work procedure information (e.g., still image data, video image data, audio data, etc.) include, for example, calling out "Inspection Start" upon entering the room and "Inspection End" upon leaving, as well as pointing and calling out confirmations such as "Toilet OK" and "Check OK" when inspecting each area of ​​the room. Other items include checking the tension of the bed sheets, whether the corners of the sheets are folded into a triangle or square, the position of the pillows (centered or to the side), the shape of the folded edges of the pillowcases, the position of the hanger grips, and the presence or absence of amenities (inside the refrigerator, toiletries, mineral water, etc.). Furthermore, the position and finish of each item may be adjusted to make the room more pleasant for the guest, and final setting of each item may be performed. In addition, in luxury hotels, women-only floors, etc., additional inspection items may be required, or additional items may be added to the minimum inspection items to accommodate requests from the facility operator. Furthermore, inspection procedure information, etc., may be linked to item identification information.

[0036] The collected data DB1115 is a database having predetermined tables in which various data are linked to predetermined identification information, etc., and an example of its table structure is shown in Figure 3. For example, the collected data table stores information received and collected from the smart glasses 200, linked to the smart glasses identification information (image data, audio data, time data, location information data (either from a GPS mechanism, or the server 100 analyzing the location of the inspector in the room based on the image data received from the smart glasses 200 using a learning model 1116)), etc.

[0037] Referring to Figure 2, the smart glasses 200 comprises an imaging unit 201, a display unit 202, a microphone speaker unit 203, a timer unit 204, a location information data management unit 205, a data transmission / reception unit 206, and a storage unit 207. Each of these units may be implemented using known input / output interfaces, a CPU, or an HDD.

[0038] The imaging unit 201 has the function of capturing images using a camera mechanism or the like.

[0039] The display unit 202 has the function of displaying various data on a display located in the area corresponding to the lens of the eyeglasses.

[0040] The microphone speaker unit 203 has the function of receiving audio input and outputting audio using a microphone and speaker mechanism.

[0041] The timer unit 204 has the function of managing and outputting information such as time data and elapsed time using a timer mechanism.

[0042] The location information data management unit 205 has the function of managing and outputting location information data using a GPS mechanism or the like.

[0043] The data transmission / reception unit 206 has the function of exchanging various types of data with the server 100 and the like.

[0044] The storage unit 207 has the function of storing various types of data in a predetermined storage medium.

[0045] Next, the processing flow of the guest room cleaning inspection support system 10 according to this embodiment will be described in detail with reference to the flowchart in Figure 4. Examples of the screens shown in Figures 5, 6, and 7 will also be referenced as appropriate. The following data transmission and reception are carried out via the internet or a designated network, as shown in Figure 1. Note that the transmission and reception of various data between the server 100 and the smart glasses 200 is a well-known mechanism, and various known technologies may be applied; therefore, only the bare minimum explanation is provided here, and a detailed explanation is omitted.

[0046] Furthermore, the following explanation assumes a scenario where a hotel (facility) guest room is inspected by cleaning staff after cleaning, with the server 100 installed in the hotel manager's office and the smart glasses 200 worn by the inspector.

[0047] Referring to Figure 4, first, the device user management unit 101 of the server 100 searches the smart glasses identification information table, user identification information table, etc. in the device user information DB 1111 based on the smart glasses ID (identification information of the device itself) and user ID (name and password, etc., which may be obtained as voice input data by the microphone mechanism of the microphone speaker unit 203 of the smart glasses 200 and recognized based on that) received from the data transmission / reception unit 206 of the smart glasses 200, and performs device authentication and user authentication for this service (step S1). Since this is a typical user authentication process during login based on prior account registration, a detailed explanation is omitted. Note that this step S1 is not mandatory.

[0048] Next, the imaging unit 201 of the smart glasses 200 starts to continuously capture images (videos) over time using a camera mechanism, the microphone speaker unit 203 receives audio input using a microphone mechanism, the timer unit 204 outputs time information using a timer mechanism, the location information data management unit 205 outputs location information data using a GPS mechanism, and the collected data management unit 105 of the server 100 stores the information received and collected from the data transmission / reception unit 206 of the smart glasses 200 (data including at least one of image data, audio data, time data, location information data, etc.) in the collected data DB 1115, linked to the smart glasses identification information (step S2). Note that this step S2 may be performed concurrently with step S1 above.

[0049] Next, upon entering the hotel room, the inspector calls out "Inspection start," and the room information management unit 102 of the server 100 uses an artificial intelligence (AI) module, etc., based on image data and voice data received from the smart glasses 200, to recognize and extract the facility name and room number as facility name and room identification information using image and voice recognition technology, etc., based on a predetermined learning model 1116 (this may involve recognizing characters and symbols displayed on doors and walls, or acquiring the hotel name, room number, etc., as voice input data using the microphone mechanism of the microphone speaker unit 203 of the smart glasses 200 and recognizing based on that). The unit then displays at least a portion of the room layout information (floor plan information) and ideal completed state image data of each area of ​​the room from the room information DB 1112 on the display unit 202 of the smart glasses 200. Furthermore, the inspection work information management unit 104 of the server 100 may use an artificial intelligence (AI) module, etc., based on image data and audio data received from the smart glasses 200, to recognize and extract the facility name and room number as facility name and room identification information using image and voice recognition technology, etc., based on a predetermined learning model 1116, and display the indoor patrol route (the movement route may be indicated by arrows) from the inspection work information DB 1114 on the display unit 202 of the smart glasses 200 (step S3). As an example, as shown in Figure 5, digital information will be overlaid on the real world scene that is actually being seen. Additionally, the time spent in each area of ​​the room will be measured.

[0050] Simultaneously, when the inspector enters the hotel room, the item information management unit 103 of the server 100 uses image and voice data received from the smart glasses 200 to enable an artificial intelligence (AI) module to recognize and extract items in each area of ​​the room (for example, cups, beds, pillows, hangers, and other furnishings) using image and voice recognition technology based on a predetermined learning model 1116. Here, the display unit 202 of the smart glasses 200 displays items such as beds (the actual real-world scene being seen) with "dot" marks (digital information) superimposed on them (this is not mandatory; see also Figure 5).

[0051] Then, the inspector performs the inspection work on the first area of ​​the room (Step S4). When the inspector approaches a designated item, the inspection work information management unit 104 of the server 100 may use an artificial intelligence (AI) module, etc., based on image data and audio data received from the smart glasses 200, to use image and voice recognition technology, etc., via a predetermined learning model 1116 to display the optimal dwell time data for each area, inspection work item information (e.g., text data), inspection work procedure information (e.g., still image data, moving image data, audio data, etc.) on the display unit 202 of the smart glasses 200. As an example, as shown in Figure 6, digital information is overlaid on the real world scene that the inspector is actually seeing. Alternatively, the inspection work item information and inspection work procedure information may be displayed on the display unit 202 of the smart glasses 200 by reading an AR marker attached to the item. The inspection task information may include, for example, more than 300 items. In addition, the inspection task procedure information may include checking the tension of the bed sheets, whether the corners of the sheets are folded into a triangle or square, the position of the pillow (center, towards the edge), the shape of the folded edge of the pillowcase, the position of the hanger grip, and the presence or absence of amenities (inside the refrigerator, toiletries, mineral water, etc.), and the inspection task procedure may be displayed as a video on the display unit 202 of the smart glasses 200. The item information management unit 103 of the server 100 may also display at least one of the following information on the display unit 202 of the smart glasses 200: the item name, information on where it is located in each area of ​​the room, and information on the ideal completion status, all taken from the item information DB 1113. At the end of each area of ​​the room, the inspector may call out "Toilet OK," "Check OK," etc., or perform a pointing confirmation along with the call.

[0052] Next, the inspector sequentially carries out inspection work on the following areas of the room (Step S5). The Inspection Operations Information Management Unit 104 may, based on location data from the smart glasses 200, display arrows guiding the inspector as they move through each area of ​​the room. Initially, the entire indoor patrol route may be displayed, but gradually the arrows may be shortened to show only the routes for the areas of the room where inspection work is not yet completed. Alternatively, as the inspector moves through each area of ​​the room, the arrows guiding the inspector as they move may be shortened sequentially to show only the next area to be inspected.

[0053] Then, after the inspection work has been carried out in all areas of each room, the inspector may call out "Inspection complete" when the guest leaves the hotel room (Step S6). The gaze-tracking voice information determination unit 108 of the server 100 may perform a warning notification for missing inspection tasks at this stage (step S7), which is part of the process in step S10 described later, where it analyzes and outputs information about missing or insufficient inspection tasks and issues a warning notification. This prevents obvious mistakes such as not performing inspection tasks for particularly important items in the list of inspection tasks. If the inspection tasks themselves are performed but are insufficient in quality, time, etc., this step 7 may be disregarded and become the subject of guidance in step S10 described later, or the information about insufficient inspection tasks may be analyzed and output to issue a warning notification in step 7. If any inspection tasks are missed, the inspector will perform those tasks and then return to step S6 above.

[0054] Steps S1 through S7 described above primarily represent a flow chart related to supporting the inspector's inspection work itself. This system enables effective and efficient inspection work, improves the quality of inspections, and allows individual inspectors to handle inspections of different room types they are unfamiliar with. For example, previously, even if an inspector was dispatched to another hotel in the same group to provide support, they may not be able to handle different room types due to lack of experience or knowledge (they would not be able to carry out inspections smoothly). However, this system makes it possible to handle such cases.

[0055] The following describes the workflow for supporting inspectors in improving their work processes. Note that the following steps may be performed in any order and may be carried out concurrently. Feedback from designated instructors not only points out when individual inspection tasks are not being performed correctly, but also identifies areas for improvement, such as when tasks are taking too long. It also allows for the identification of individual habits and tendencies that the individual may not be aware of. Not only is post-event instruction possible, but real-time remote guidance is also available. Furthermore, it can be used for self-assessment.

[0056] First, the indoor patrol route determination unit 106 of the server 100 acquires optimal route information stored in the inspection work information DB 1114, and generates actual route information based on location information data (by GPS mechanism, etc.) received from the smart glasses 200 stored in the collected data DB 1115, and outputs comparison information of the two (optimal route information and actual route information) (step S8). The display unit 110 of the server 100 may display the optimal route information and actual route information together vertically, horizontally, or superimposed, or it may analyze and display the differences between the two (see Figure 7 as an example). This makes it possible to confirm whether the inspector is adhering to the efficiency of the travel route, and to uncover individual movement habits that the person is unaware of, such as going to check an area that should only be checked once (for example, around the bed) twice. The indoor patrol route determination unit 106 may also recognize and extract various areas within the room from image data and audio data received from the smart glasses 200 using the learning model 1116, and generate actual route information based on that. The indoor patrol route determination unit 106 may also display the actual route information, which is accumulated along with the optimal route information, on the server 100 and / or smart glasses 200 to provide real-time warnings.

[0057] Next, the server 100's area dwell time determination unit 107 acquires the optimal dwell time data for each area of ​​the room stored in the inspection work information DB 1114, and also creates actual dwell time data for each area based on the time data (by a timer mechanism, etc.) and location information data (by a GPS mechanism, etc.) received from the smart glasses 200 stored in the collected data DB 1115, and outputs comparison information of the two (optimal dwell time data and actual dwell time data for each area) (step S9). The server 100's display unit 110 may display the optimal dwell time data and the actual dwell time data for each area together, either vertically, horizontally, or superimposed, or it may analyze and display the difference between the two (for example, a positive or negative value) (see Figure 7 as an example). In other words, faster is not always better, and simply rushing the inspection work could lead to the risk of cutting corners. Furthermore, the area dwell time determination unit 107 may recognize and extract each area of ​​the room from image data and audio data received from the smart glasses 200 using the learning model 1116, and create location information data based on that. Furthermore, when inspectors complete inspection tasks in each area of ​​a room, if they verbally announce "Toilet OK," "Check OK," etc., or perform pointing confirmations along with the announcements, the Smart Glasses 200 may be used to determine the completion of the inspection tasks based on image data and audio data received from the Smart Glasses 200, and this data may be used to measure the time spent in the room. The area dwell time determination unit 107 may also display the elapsed time data along with the optimal dwell time data on the server 100 and / or smart glasses 200 to provide real-time alerts.

[0058] Next, the gaze-voice information determination unit 108 of the server 100 acquires at least one of the inspection work item information, inspection work procedure information, and inspection work call information stored in the inspection work information DB 1114. The learning model 1116 also analyzes the image data (mainly images of work in the line of sight of the inspector's hands), voice data (mainly voices of the inspector's calls during work), time data (by a timer mechanism, etc.), and location information data (by a GPS mechanism, etc.) received from the smart glasses 200 and stored in the collected data DB 1115 to determine whether there are at least two cases of missing or insufficient inspection work (for example, by determining image matches, etc.) and outputs a warning notification (for example, by displaying inspection work items that have been missed, or by displaying images of work that may be considered insufficient inspection work) (step S10). The display unit 110 of the server 100, described later, may display inspection task item information (not mandatory) and information on missing or insufficient inspection tasks together, either vertically, horizontally, or superimposed, or it may analyze and display the differences between the two. Even if an inspection task omission warning notification is issued in step S7 above, it is also possible to operate in such a way that, for example, in step S7 above, only particularly important items from the list of inspection task items are judged, and in step S10, the judgment is made including the detailed items of the list of inspection task items. When designated supervisors provide guidance to inspectors to improve their inspection work, they can provide evidence-based and reliable guidance using photographic evidence. For example, when checking for remaining hair in a bathroom, if it is necessary to check the back of the door or the ceiling, images of what the inspector's line of sight moves are recorded, making it clear whether or not they are looking at (checking) these areas. This reduces the likelihood of decreased motivation and interpersonal friction caused by discrepancies in whether or not the inspector is looking at something during guidance. Calling out during inspection work refers to, for example, saying "Toilet OK" or "Check OK" when completing the inspection of each area in the room. Even if pointing and confirming is also required along with calling out, the pointing and confirming action is also captured in the image of the work, mainly from the inspector's gaze, so the gaze-voice information determination unit 108 can determine whether or not the pointing and confirming action was performed. The gaze-based audio information determination unit 108 may also display information about missed inspection tasks on the server 100 and / or smart glasses 200 to provide real-time warnings. The gaze-voice information determination unit 108 may issue a warning if the image data received from the smart glasses 200 does not correspond to the image data included in the inspection procedure information, or if the image data is present but only for an extremely short period of time (for example, a certain amount of time is required to look at a cup to check if it is clean). Furthermore, even if image data is present, if the duration is longer than a predetermined tolerance range compared to the optimal work time data based on the inspection procedure information, it may display a warning indicating that the inspection is inefficient. In addition, the optimal work time data based on the inspection procedure information and the actual work time data may be displayed together vertically, horizontally, or superimposed, or the difference between the two (for example, a positive or negative value) may be analyzed and displayed.

[0059] Next, the inappropriate operation detection unit 109 of the server 100 acquires inappropriate operation information (for example, still image data, moving image data, audio data, etc., which may be linked to the name of the inappropriate operation) stored in the inspection work information DB 1114, and the learning model 1116 analyzes the inappropriate operation information, etc., against the image data (mainly images of the work in the line of sight of the inspector's hands), audio data (mainly voices of the inspector's work-related names), time data (by a timer mechanism, etc.), and location information data (by a GPS mechanism, etc.) received from the smart glasses 200 stored in the collected data DB 1115 (for example, determining image matches and similarity trends), outputs (for example, displaying the image of the relevant work), and issues a warning notification (step S11). When providing guidance to improve inspectors' inspection work, it becomes possible to provide evidence-based and reliable guidance using photographic evidence. For example, actions such as brushing with a cloth (rag / duster) or putting one's foot on the bathtub (and wiping high places) are examples of inappropriate actions. In other words, from the perspective of whether dust has been properly removed, using a cloth in a brushing motion is an inappropriate action as it simply scatters dust. Designated instructors can provide guidance to inspectors on the proper use of tools and provide guidance from a safety perspective. The inappropriate operation detection unit 109 may also display inappropriate operation information, etc., on the server 100 and / or smart glasses 200 to provide real-time warnings.

[0060] According to the above embodiment, it is possible to realize a room cleaning inspection support system that can provide more specific and detailed support for the inspection work itself performed by inspectors, which requires them to complete many tasks in a short time and demands knowledge, judgment, and even a service-oriented mindset, and can also support the improvement of inspectors' work.

[0061] Furthermore, although not to be exemplified individually, the present invention may be implemented with various modifications without departing from its spirit. For example, the functions of each device / system may be realized by loading a program for realizing the functions of each device / system into each device / system and executing it. Moreover, the program may be transmitted to other computer systems via a computer-readable recording medium such as a CD-ROM or magneto-optical disk, or via a transmission medium such as the internet or a telephone line using transmission waves. In addition, some systems may be realized through human action. Furthermore, each of the above processes may be carried out by combining multiple functions of each device, and the known data used in such processes may be obtained from a network in conjunction with known databases, etc., or the obtained data may be stored and used in the database of this system. In other words, details that can be understood and implemented by a person skilled in the art from common technical knowledge may be omitted as appropriate. Furthermore, as long as the above effects are achieved, the order of each step in the flowchart may be changed, performed simultaneously, or non-essential processes may be omitted. Alternatively, the artificial intelligence (AI) module of the server 100 may directly access the data received from the smart glasses 200 (without storing it in a collected data database, etc.) and execute all or part of the above-mentioned recognition and analysis processes using the learning model 1116, etc. [Explanation of symbols]

[0062] 10. Support system for guest room cleaning inspection tasks 100 servers 200 Smart Glasses

Claims

[Claim 1] A room cleaning inspection support system in which a server and smart glasses are connected via a network, The aforementioned server, A data collection management unit receives data including at least one of image data, audio data, time data, and location data from smart glasses worn by an inspector who performs inspection work for the cleaning of guest rooms in the facility. A room information management unit that, using a predetermined learning model generated by machine learning based on training data by a predetermined artificial intelligence module, analyzes the data received from the smart glasses, extracts facility name and room identification information for the rooms of the facility, searches the storage unit of the server based on the facility name and room identification information, and displays at least one of the area layout information for each area of ​​the room and the ideal completed state image data for each area of ​​the room, which are stored in association with the facility name and room identification information, on the display unit of the smart glasses. The item information management unit analyzes the data received from the smart glasses using the learning model, extracts item identification information for the items in the guest room, searches the storage unit based on the item identification information, and displays at least one of the item name, information on where it is located in each area, and ideal completion state image data stored in association with the item identification information on the display unit of the smart glasses. An inspection work information management unit that, using the learning model, analyzes the data received from the smart glasses, extracts at least one of the facility room identification information and the item identification information, searches the storage unit based on at least one of the facility room identification information and the item identification information, and displays at least one of the optimal route information, optimal stay time information for each area, inspection work item information, and inspection work procedure information stored in association with at least one of the facility room identification information and the item identification information on the display unit of the smart glasses, An indoor patrol route determination unit that acquires the optimal route information stored in the memory unit, analyzes the data received from the smart glasses using the learning model, creates actual route information, and displays comparison information between the optimal route information and the actual route information on the display unit of the server, An area stay time determination unit that acquires optimal stay time information for each area stored in the memory unit, analyzes the data received from the smart glasses using the learning model, creates actual stay time information for each area, and displays comparison information between the optimal stay time information and the actual stay time information on the display unit of the server, A gaze-voice information determination unit acquires at least one of the inspection task item information, the inspection task procedure information, and the inspection task calling information stored in the memory unit, analyzes the data received from the smart glasses using the learning model, and if there are at least one of the missing inspection tasks or insufficient inspection tasks, displays on the display unit of the server that there are at least one of the missing inspection tasks or insufficient inspection tasks, An inappropriate operation determination unit acquires the inappropriate operation information stored in the memory unit, analyzes the data received from the smart glasses using the learning model, and if an inappropriate operation is detected, displays the presence of the inappropriate operation on the display unit of the server. A room cleaning inspection support system characterized by having the following features.

Images