System and method for improving reception service using face recognition and computer vision

Abstract

The present application relates to a system and method for enhancing reception services using facial recognition and computer vision. A computer system for facilitating reception services in a resort, comprising: a camera configured to capture images of at least a portion of the resort; a computing device in electronic communication with the camera, the computing device comprising a processor and a non-transitory memory device. The non-transitory memory device comprises computer readable instructions that, when executed by the processor, cause the computing device to: receive at least one captured image in the camera; identify at least one individual in the image using a facial recognition module; and determine a service performance for the at least one individual using a computer vision module. The system further comprises an electronic device configured to display information about the service event.

Description

Technical Field

[0001] This invention relates to facial recognition and computer vision technology. Background Technology

[0002] Hospitality services in resorts often rely solely on human attention and operation. For example, restaurant services (such as refilling drinks, taking orders when guests are ready, and presenting bills after meals) typically require staff to be attentive to these needs, which can lead to delays when staff are busy or negligent. Furthermore, without automated, computer-based oversight, it's difficult to determine whether service providers are meeting the resort's service standards, such as the timeliness and / or frequency of performing certain tasks for guests. Additionally, service standards may be particularly low in areas where staff lack tipping or other performance-based compensation incentives. Moreover, some guests may have unique, personalized service standards that resorts may struggle to meet because these standards may not be recorded in a centralized database, and it can be difficult to identify when a guest arrives, especially if the booking did not use the guest's name.

[0003] The information disclosed in this background section is only intended to enhance the understanding of the background of this invention, and therefore may contain information that does not constitute prior art. Summary of the Invention

[0004] This invention relates to various embodiments of a computer system configured to facilitate hospitality services within a resort. The computer system includes: a camera configured to capture images of at least a portion of the resort; and a computing device electronically communicating with the camera and including a processor and a non-transitory memory device communicating with the processor. The non-transitory memory device includes computer-readable instructions that, when executed by the processor, cause the computing device to: receive at least one captured image from the camera; identify at least one individual in the at least one captured image from the camera using a face recognition module; determine the service performance status for that individual using a computer vision module; and compare the service performance status with service standards stored in the non-transitory memory device. The system also includes an electronic device electronically communicating with the computing device, the electronic device being configured to generate an alarm in response to a failure to meet service standards.

[0005] Electronic devices can be tablet computers, computer monitors, virtual reality headsets, augmented reality headsets, smartwatches, smartphones, or headsets.

[0006] Alarms can be audio or visual.

[0007] The electronic device may be a display device, and the display device may be configured to display a graphical user interface, the graphical user interface including a graphical representation of the layout of the part of the resort, location identifiers of at least one individual in the layout, and service standards associated with the at least one individual.

[0008] The display device can also be configured to display the identity of at least one individual.

[0009] Non-transitory memory devices may include a face recognition database containing face images, identifiers associated with the face images, and preferences associated with the identifiers.

[0010] Non-transitory storage devices may include computer vision databases containing events, service tasks associated with the events, and service standards associated with the service tasks.

[0011] At least one of the service standards can be based on one of the preferences.

[0012] The computer vision module may include convolutional neural networks.

[0013] When executed by a processor, computer-readable instructions can also enable a computing device to use a computer vision module to determine that an event involving at least one individual has occurred.

[0014] Events can include guests arriving at the restaurant, tables becoming empty, guests preparing to order, guests finishing their drinks, guests finishing their meals, or guests leaving the restaurant.

[0015] Service standards can be duration, frequency, or number of occurrences.

[0016] The aforementioned parts of the resort may include restaurants, reception areas, meeting areas, meeting rooms, room service areas, pool areas, or valet parking areas.

[0017] The non-transitory memory device can store the preferences of at least one individual, and the computer-readable instructions, when executed by the processor, can cause the display of the electronic device to display the name of at least one individual and the preferences of at least one individual in response to the facial recognition module recognizing at least one individual.

[0018] Preferences may include the dietary preferences of at least one individual.

[0019] The resort can be a comprehensive resort.

[0020] Services may include: seating guests at empty tables in the restaurant, greeting guests already seated at tables in the restaurant, staff visiting guests seated at tables in the restaurant, staff refilling empty beverage cups, staff clearing tables at the end of the meal, staff handing bills to guests, or staff turning over tables in the restaurant.

[0021] Service standards can include: time for empty tables to be seated, time for greeting guests, number of times service staff visit seated guests, time for service staff to refill empty beverage cups, time for receiving food and / or beverages after a guest orders, time for service staff to clear the table, time for service staff to deliver the bill, or time for service staff to turn over a table for the next guest.

[0022] When executed by a processor, the computer-readable instructions also enable the computing device to: detect at least one service problem in at least one image captured by at least one camera using a computer vision module; identify at least one individual in at least one image captured by at least one camera, the at least one individual being associated with at least one service problem using a face recognition module; and display information about at least one service problem and an identifier of at least one individual associated with at least one service problem on a display of the electronic device.

[0023] When executed by a processor, computer-readable instructions can also enable computing devices to: use a computer vision module to detect activities that indicate a resort guest needs assistance; and generate an alarm on an electronic device configured to notify resort staff that the guest needs assistance.

[0024] When executed by a processor, the computer-readable instructions may also enable the computing device to: use a facial recognition module to detect at least one VIP guest in an image captured by at least one of the cameras; and generate an alarm on an electronic device to notify resort staff of the presence of the at least one VIP guest.

[0025] When executed by a processor, the computer-readable instructions also enable the computing device to: use a computer vision module to detect service events in images captured by at least one of the cameras; use a face recognition module to identify individuals associated with service events in images captured by at least one of the cameras; and track and store service events on the computing device as individuals move within the resort.

[0026] This section provides a summary of some features and concepts of embodiments of the present invention, which are further elaborated in the detailed embodiments below. This section is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. One or more features or steps described may be combined with one or more other described features or steps to constitute an implementable device or method. Attached Figure Description

[0027] The features and advantages of embodiments of this disclosure will become more apparent from the following detailed description and in conjunction with the accompanying drawings. In the drawings, the same reference numerals are used throughout to refer to the same features and components. The drawings are not necessarily drawn to scale.

[0028] Figure 1 This is a perspective view of a computer-based system utilizing both facial recognition and computer vision, according to an embodiment of the present disclosure. Figure 2 yes Figure 1 A schematic block diagram of an embodiment of the system shown; Figure 3 A face recognition database according to one embodiment of the present disclosure is described; Figure 4 A computer vision database according to one embodiment of the present disclosure is described; Figure 5A The operation of a system according to an embodiment of the present disclosure when applied in a restaurant is described; Figures 5B-5D The present disclosure describes a dashboard displayed on a graphical user interface (GUI) of an electronic device according to embodiments thereof; and Figure 6 This is a flowchart according to one embodiment of the present disclosure, illustrating the task of a computer-based method for identifying an individual and tracking the service performance involving that individual. Detailed Implementation

[0029] This disclosure relates to various embodiments of a computer-based system that utilizes both facial recognition and computer vision to enhance hospitality services at resorts or other venues. The system is configured to: use a facial recognition module (facial recognition engine) to analyze video and / or images captured from one or more cameras at the venue to identify one or more individuals in the video and / or images. The system is also configured to: use a computer vision module (computer vision engine) to analyze video and / or images captured by one or more cameras to track the occurrence of one or more events involving the identified individual, and / or the performance of one or more services for the identified individual, such as restaurant service, valet parking service, housekeeping service, butler service, retail store service, pool service, front desk service, limousine service, taxi service, public area cleaning service, entertainment service, facility service, food preparation service, and / or meeting service. The system is further configured to: compare the services tracked by the computer vision module with one or more service criteria (i.e., service metrics) and report the comparison results to the service provider via an electronic device, such as a tablet computer, a head-mounted device, or a headset. In this way, the computer-based system of this disclosure enables enhanced provision of hospitality services.

[0030] In the following description, exemplary embodiments will be described in more detail with reference to the accompanying drawings, wherein the same reference numerals refer to the same elements throughout. However, the invention may be embodied in a variety of different forms and should not be construed as being limited to the exemplary embodiments shown herein. Rather, these embodiments are provided as examples to make this disclosure exhaustive and complete, and to fully convey various aspects and features of the invention to those skilled in the art. Therefore, processes, elements, and techniques not essential for a full understanding of the various aspects and features of the invention by those skilled in the art may not be described. Unless otherwise stated, the same reference numerals denote the same elements throughout the drawings and textual description, and therefore, their description may not be repeated.

[0031] In the accompanying drawings, relative dimensions of elements and regions may be exaggerated and / or simplified for clarity. Although relative terms are used herein to describe the spatial relationship of one element relative to another, it should be understood that these terms are intended to cover different orientations of the various elements and components of the invention beyond those shown in the drawings. Spatial relative terms such as “below,” “under,” “lower,” “below,” “above,” and “upper” may be used herein for ease of description to illustrate the relationship of one element or feature as shown to one or more other elements or features. It should be understood that these spatial relative terms are intended to cover different orientations of the device in use or operation, not just those shown in the drawings. For example, if the device in the drawings is flipped, an element described as “below,” “under,” or “below” other elements or features will be oriented “above” other elements or features. Thus, the exemplary terms “below” and “below” can cover both above and below orientations. The device may be oriented in other ways (e.g., rotated 90 degrees or in other orientations), and the spatial relative descriptive terms used herein should be interpreted accordingly.

[0032] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. Unless the context clearly indicates otherwise, the singular forms “a” and “an” are intended to include the plural forms as well. It should also be understood that the terms “comprising” or “including” as used herein specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or inclusion of one or more other features, integers, steps, operations, elements, components, and / or groups. The term “and / or” as used herein includes any and all combinations of one or more associated listed items. When expressions such as “at least one” precede a list element, they modify the entire list element, not a single element within the list.

[0033] Any numerical range described herein is intended to include all subranges within the range that have the same numerical precision. For example, the range “1.0 to 10.0” is intended to include all subranges between the minimum value 1.0 and the maximum value 10.0 (inclusive), that is, a minimum value greater than or equal to 1.0 and a maximum value less than or equal to 10.0, such as 2.4 to 7.6. Any maximum numerical limit described herein is intended to include all lower numerical limits it covers, and any minimum numerical limit described herein is intended to include all higher numerical limits it covers.

[0034] The terms “basic,” “about,” and similar terms used herein are approximate terms, not terms of degree, and are intended to explain the inherent biases of measured or calculated values ​​recognized by those skilled in the art. Furthermore, the word “may” used in describing embodiments of the invention refers to “one or more embodiments of the invention.” The term “use” as used herein is to be considered synonymous with the term “utilize.” Meanwhile, the term “exemplary” is intended to indicate an example or illustration.

[0035] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms as defined in general dictionaries shall be interpreted as having the same meaning as in the relevant technical and / or specification context, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0036] Now for reference Figure 1 A system 100 according to one embodiment of the present disclosure includes a plurality of cameras 101 (e.g., a plurality of video cameras), a computing device (e.g., a host or server) 102 configured to receive and process images captured from the cameras 101, and at least one electronic device 103 in electronic communication with the host or server 102. The plurality of cameras 101 may be connected to the host or server 102 via a wired connection or via a wireless network. The host or server 102 includes a processor 104, a non-transitory memory device 105 communicating with the processor 104, and a wireless communication device 106 communicating with the processor 104. The non-transitory memory device 105 includes a face recognition module 107 (e.g., a face recognition engine) configured to identify one or more individuals in the images captured by the cameras 101, and a computer vision module 108 (e.g., a computer vision engine) configured to identify the execution of one or more services and / or the occurrence of one or more events in the images captured by the cameras 101.

[0037] System 100 can be used in resorts or other locations, such as valet parking areas of restaurants or hotels. In one or more embodiments, system 100 can be applied to integrated resorts, which are large entertainment complexes combining various facilities such as hotels, restaurants, retail stores, and entertainment venues. Host or server 102 can be located at the resort or other location, or remotely.

[0038] As used herein, the term "processor" includes any combination of hardware, firmware, memory, and software for processing data or digital signals. The hardware of a controller may include, for example, a microcontroller, an application-specific integrated circuit (ASIC), a general-purpose or special-purpose central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), and programmable logic devices such as field-programmable gate arrays (FPGAs). In the processors used herein, each function is either performed by hardware configured (i.e., hardwired) to perform that function or by more general-purpose hardware (e.g., a CPU) configured to execute instructions stored in a non-transitory storage medium or memory. A processor may comprise two or more processors; for example, a processor may include two processors (an FPGA and a CPU) interconnected on a PCB. Non-transitory memory devices may be any suitable type or kind of memory, such as non-transitory read / write memory (e.g., flash memory, hard disk drive (HDD), or solid-state drive (SDD)), non-transitory read-only memory (ROM) (e.g., programmable read-only memory (PROM) or erasable programmable read-only memory (EPROM)), or combinations thereof.

[0039] Wireless communication device 106 can communicate with electronic device 103 using any suitable wireless communication protocol, such as Bluetooth communication protocol, Wi-Fi communication protocol (e.g., IEEE 802.11x standard), or equivalent wireless communication protocol that allows secure wireless communication with electronic device 103.

[0040] In one or more embodiments, electronic device 103 may be a tablet computer, computer monitor, virtual reality headset, augmented reality headset, smartwatch, smartphone, headset, any other suitable type or kind of electronic device, or a combination thereof. As described in more detail below, electronic device 103 is configured to generate an alarm (e.g., an audio alarm, such as a tone or voice message, and / or a visual alarm, such as a text-based alarm or graphic) in response to one or more services performing below the relevant service standard.

[0041] Now for reference Figure 3In the depicted embodiments, the face recognition module 107 stored on the non-transitory memory device 105 includes a face recognition database 109. In one or more embodiments, the face recognition database 109 includes a table with multiple fields, including a face image field 110 and a personal identifier (identifier) ​​field 111. The face image field 110 includes multiple face images (i.e., each entry under the face image field 110 includes at least one face image), and the personal identifier (identifier) ​​field 111 includes multiple names or customer IDs associated with the face images (i.e., each entry under the personal identifier field 111 includes a name and / or customer ID number). In other words, each entry in the face recognition database 109 includes one or more face images of a particular individual and the associated personal identifier (e.g., name and / or customer ID) of that particular individual. In one or more embodiments, each face image may be (or may include) a vector map (i.e., a depth map) spanning multiple points on the face image.

[0042] In one or more embodiments, the face recognition database 109 further includes a personal preference field 112, which includes multiple preferences associated with a face image in the face image field 110 and an associated personal identifier in the personal identifier field 111, such as food preferences (e.g., vegetarian) and / or beverage preferences (e.g., red wine or white wine). In one or more embodiments, personal preferences may be an individual's preferred room type in a hotel, a preferred seating location or table in a restaurant. In one or more embodiments, each entry in the personal preference field 112 may include one or more preferences associated with an individual, although in one or more embodiments, only some entries in the preference field may include one or more associated preferences (i.e., some entries in the personal preference field 112 may be empty). In one or more embodiments, the face recognition database 109 (which includes a field 110 containing a face image, a field 111 containing personal identification information, and a field 112 containing personal preferences) may be a lookup table. In one or more embodiments, entries in the personal preference field 112 are editable (e.g., read-write operations can be performed relative to entries under the personal preference field 112).

[0043] In operation, face recognition module 107 is configured to access face recognition database 109 to identify individuals in videos and / or images captured by one or more cameras 101. In one or more embodiments, face recognition module 107 is configured to receive an image of an individual's face captured by one or more cameras 101 and compare that image with face images stored in face image field 110 of face recognition database 109. In one or more embodiments, face recognition module 107 may be configured to generate a vector map (e.g., a depth map) across multiple points in the face image captured by one or more cameras 101 and compare that vector map with the vector map of the face image stored in face recognition database 109. In one or more embodiments, face recognition module 107 is configured to affirmatively identify an individual in response to a comparison returning a match above a threshold (e.g., about 80% or about 90%). In one or more embodiments, the face recognition module 107 may utilize a trained Siamese artificial neural network to identify individuals in images captured by one or more cameras 101 (e.g., the face recognition module 107 may utilize two identical artificial subnetworks, each with the same weights between nodes in the layers, to process images captured by one or more cameras 101 and face images in the face recognition database 109, and generate feature embeddings, which are then compared using a distance function to determine the similarity between face images, thereby identifying individuals in images captured by (multiple) cameras 101).

[0044] Now for reference Figure 4In the depicted embodiments, the computer vision module 108 stored on the non-transitory memory device 105 includes a computer vision database 113. In one or more embodiments, the computer vision database 113 includes a table with multiple fields, including an event field 114, a service task field 115, and a metric field 116. The event field 114 includes multiple events (i.e., each entry under the event field 114 includes a different event, such as a guest arriving, a table being cleared or otherwise becoming available, a guest preparing to order, a customer finishing a drink, a customer finishing their meal, or a customer leaving the restaurant). The service task field 115 includes multiple services associated with the events listed in the event field 114, such as greeting guests, seating guests, accepting food and / or drink orders, delivering ordered food, replenishing drinks, fetching bills, clearing tables, and monitoring diners. The metric field 116 includes multiple service metrics or standards associated with the service tasks listed under the service task field 115 (e.g., a threshold time to complete a given service, the frequency of performing a given service, the number of times a service is performed). In one or more embodiments, the metric field 116 in the computer vision database 113 may be adjusted (or determined) based on personal preferences within the personal preference field 112 in the face recognition database 109. Therefore, in one or more embodiments, the performance metric / standard may be personalized rather than uniform for each individual.

[0045] In one or more embodiments, the computer vision module 108 is an artificial neural network (e.g., a convolutional neural network (CNN)) trained on an image and / or video corpus depicting various reception services. In one or more embodiments, the computer vision module 108 may be configured to detect the execution of restaurant services provided by service personnel, such as: (i) detecting when a guest is seated and ready to order; (ii) detecting when a guest needs another drink; (iii) detecting when a guest attempts to attract the attention of a service personnel; (iv) detecting when a guest has finished eating; (v) detecting when a guest leaves the table; and / or (vi) detecting when the table is set and ready for the next guest. Furthermore, in one or more embodiments, the computer vision module 108 may be configured to calculate the time between any of the listed events described above and / or calculate the number of the listed events described above occurring within a specified time period. In one or more embodiments, system 100 may be configured to send a signal to electronic device 103 via wireless communication device 106, the signal being configured to cause electronic device 103 to generate an alarm and / or display information calculated or determined by computer vision module 108 based on information calculated or determined by computer vision module 108, such as displaying the timing of the events listed above based on service criteria, displaying service performance relative to associated service criteria, and / or notifying an individual in response to the satisfaction of certain thresholds or performance rules.

[0046] In one or more embodiments, the computer vision module 108 may be configured to determine, directly or indirectly, the execution of a service or the occurrence of an event. For example, in one or more embodiments, the computer vision module 108 may be configured to determine that a customer has finished their drink by detecting the liquid level in a glass (i.e., directly measuring the amount of beverage remaining) or by detecting the angle and / or height of the glass relative to the customer when the person sips from the glass (i.e., indirectly measuring the amount of beverage remaining). For example, the computer vision module 108 may be configured to determine that a customer has finished their drink in response to the angle and / or height of the glass exceeding a threshold angle and / or threshold height.

[0047] In one or more embodiments, the computer vision module 108 may be configured to determine or detect one or more service problems (e.g., by comparing images captured by one or more cameras 101 with known service problems stored in the computer vision database 113), and the face recognition module 107 may be configured to identify (e.g., by comparing images captured by one or more cameras 101 with face images stored in the face recognition database 109) affected by the service problems determined by the computer vision module 108. Furthermore, in one or more embodiments, the system 100 may cause the display of the electronic device 103 to display information about the service problems and the identifiers (e.g., names, customer IDs, and / or face images) of the individuals affected by the service problems. In one or more embodiments, the information about the service problems and the identifiers of the individuals affected by the service problems may be stored on the computing device 102.

[0048] In one or more embodiments, system 100 may be configured such that a display of electronic device 103 displays a leaderboard including a list of individual identities (e.g., names, customer IDs, and / or facial images) identified by face recognition database 109 and associated service performance relative to relevant performance metrics determined by computer vision module 108 (e.g., a comparison of service events determined by computer vision module 108 with performance metrics associated with those service events). In one or more embodiments, the list in the leaderboard may be ordered hierarchically based on the performance of services relative to associated service metrics (e.g., the fastest service (and the associated individuals performing that service) may be listed at the top, with the remaining services (and associated individuals) listed in descending order, or the slowest service (and the associated individuals performing that service) may be listed at the top, with the remaining services (and associated individuals) listed in ascending order). In one or more embodiments, the leaderboard may rank individuals (and their associated service performance relative to performance metrics) in any other suitable order, such as alphabetical order by last name or numerical order by customer ID.

[0049] In one or more embodiments, the computer vision module 108 may be configured to determine or detect activities that indicate that a resort guest needs assistance (e.g., by comparing images captured by one or more cameras 101 with known activities that indicate that a guest needs assistance stored in the computer vision database 113).

[0050] In one or more embodiments, system 100 is configured to notify resort staff (e.g., by generating an alarm, such as a visual, audio, and / or haptic alarm, on electronic device 103) in response to the computer vision module 108’s identification of certain individuals (e.g., VIP guests).

[0051] In one or more embodiments, computer vision module 108 may be configured to determine or detect one or more service events (e.g., by comparing images(s) captured by one or more cameras 101 with known service events stored in computer vision database 113), face recognition module 107 may be configured to identify(s) individuals(s) affected by service events determined by computer vision module 108 (e.g., by comparing images(s) captured by one or more cameras 101 with face images stored in face recognition database 109), and system 100 may be configured to track and store these service events concerning specific individuals occurring at various locations within the resort on computing device 102.

[0052] In one or more embodiments, the face recognition database 109 and the computer vision database 113 may be stored on the same non-transitory storage device or different non-transitory storage devices (e.g., the face recognition database 109 and the computer vision database 113 may be stored on the same or different servers, or on the same or different hosts).

[0053] In one or more embodiments, computing device (e.g., host or server) 102 may be configured to provide access to information via an application programming interface (API) regarding service events determined by computer vision module 108, individuals identified by face recognition module 107, face recognition database 109 (or portions thereof, such as personal preferences), and / or computer vision database 113 (or portions thereof, such as service standards / metrics).

[0054] Figure 5A The operation of a system 100 according to an embodiment of this disclosure when applied in a restaurant is described. Figure 5A As shown, system 100 is configured to track the seating time of (multiple) guests, the time when a service staff member first visits a guest, the number of times a service staff member (e.g., a waiter) visits a guest at a table, the time when a guest prepares to order, the time when a service staff member completes the order, the time when a table becomes vacant (i.e., the time when (multiple) guests finish eating and leave the table), and the time when the table is reset after (multiple) guests leave.

[0055] Figure 5B This is a view of a dashboard displayed on a graphical user interface of an electronic device (e.g., a tablet computer) according to one embodiment of this disclosure. Figure 5B As shown, for each table occupied by one or more customers (e.g., tables 10 and 11), the GUI displays a circle divided into multiple sectors (e.g., a pie chart), with each sector associated with a different service provided by the server. Figure 5BAs shown, the circle is divided into six (6) sectors, representing six (6) different services provided by the service staff: (i) resetting the table after the previous group of customers has finished eating (“Table Reset”); (ii) greeting the customer (“Initial Greeting”); (iii) the customer prepares to order (“Preparing to Order”); (iv) the customer’s food order is accepted (“Order Accepted”); (v) the food is delivered to the table by the service staff (“Food Delivered”); and (vi) the table is cleaned for the next group of customers (“Table Cleaned”). In one or more embodiments, for each service milestone, the time of completion of that milestone is indicated in the associated sector. In one or more embodiments, the service milestone may not be time (e.g., a service milestone may be the number of times a particular event occurs or the frequency of that event), and in those embodiments, the GUI may display the associated criterion in the associated sector of the circle.

[0056] Furthermore, in one or more embodiments, for each service milestone, the corresponding sector of the circle is indicated by color or shading to indicate whether the service milestone was completed within the associated service standard (e.g., service standard time). For example, when a service milestone has not yet been completed, the sector can be transparent or white; when a service milestone is completed according to the service standard (e.g., within the service standard time), the sector is green; when a service milestone is not completed according to the service standard but the deviation from the standard is less than 10%, the sector is yellow; and when a service milestone is less than 10% from the service standard, the sector is red. In one or more embodiments, the sector can be colored with any other color to indicate whether the service standard has been met. Furthermore, although the circle is divided into six sectors in the illustrated embodiment, in one or more embodiments, the circle can be divided into any other suitable number of sectors depending on the number of service milestones tracked by the system.

[0057] Although the graphic in the illustrated embodiment is a circle divided into sectors, in one or more embodiments, the graphic may have any other suitable shape or multiple shapes, such as a square, rectangle, triangle, etc. For example, in one or more embodiments, the shape of the graphic may correspond to the shape of the dining table or seating area where the customer is seated (e.g., a rectangle is used for a rectangular table, and a circle is used for a round table).

[0058] Figures 5C-5DA dashboard displayed on a graphical user interface (GUI) of an electronic device (e.g., a tablet computer) according to one embodiment of the present disclosure is depicted. In the illustrated embodiment, the dashboard displays a graphical representation of a restaurant layout, including seating areas (e.g., tables and booths), a kitchen area, a bar area, and a sushi counter. Furthermore, in one or more embodiments, for each table occupied by one or more customers, the GUI displays a graphic indicating the performance of various services provided by the service staff for that table. For example, in the illustrated embodiment, the GUI displays a graphic indicating the various services performed by the service staff, the time taken to complete these services, and whether these services were performed within the associated service standards. In one or more embodiments, the graphic may be... Figure 5B The segmented circles depicted in the image.

[0059] Figure 6 This is a flowchart illustrating a task of a computer implementation method 200 for identifying an individual and tracking the execution of services involving that individual, according to an embodiment of the present disclosure.

[0060] In the illustrated embodiment, method 200 includes task 210: capturing images (e.g., video and / or still images) from one or more cameras. The cameras may be located in a resort (e.g., an integrated resort) or other locations, such as the valet parking area of ​​a restaurant or hotel.

[0061] In the illustrated embodiment, method 200 further includes task 220: using a face recognition module to analyze the image captured by the camera in task 210 to identify individuals in the image.

[0062] In the illustrated embodiment, method 200 further includes task 230: using a computer vision module to analyze images captured by a camera in task 210 to identify / recognize the occurrence of one or more events (e.g., one or more actions performed by a customer at the integrated resort, such as a customer finishing their drink).

[0063] In the illustrated embodiment, method 200 further includes task 240: using a computer vision module to analyze images captured by a camera in task 210 to identify the execution of one or more service tasks (e.g., replenishing a beverage) associated with an event identified in task 230.

[0064] Although in the illustrated embodiment, task 220, which analyzes images using the face recognition module, is performed before tasks 230 and 240, which analyze images using the computer vision module, in one or more embodiments, these tasks 220, 230, and 240 may be performed in any other suitable order. For example, in one or more embodiments, tasks 230 and 240, which analyze images using the computer vision module, may be performed before task 220, which analyzes images using the face recognition module.

[0065] In the illustrated embodiment, method 200 further includes task 250: generating an alert on an electronic device (e.g., a tablet computer, smartphone, and / or AR / VR glasses) in response to a service task identified in task 240 performing below a corresponding service metric. Task 250 may include referencing service metrics stored in a computer vision database. The alert generated in task 250 may be an audio alert, a visual alert, an audiovisual alert, or a haptic alert.

[0066] The face recognition module and / or computer vision module according to the embodiments of the present invention described herein can be implemented using any suitable hardware, firmware (e.g., application-specific integrated circuits), software, or a combination of software, firmware, and hardware. For example, the face recognition module and computer vision module can be stored and executed on a single integrated circuit (IC) chip or separate IC chips. Furthermore, the face recognition module and / or computer vision module can be implemented on flexible printed circuit films, tape-on-a-carrier packages (TCPs), and / or printed circuit boards (PCBs). Additionally, the face recognition module and / or computer vision module can run processes or threads on one or more processors in one or more computing devices, executing computer program instructions and interacting with other system components to perform the various functions described herein. The computer program instructions are stored in memory, which can be implemented in the computing device using standard storage devices such as random access memory (RAM). The computer program instructions can also be stored in other non-transitory computer-readable media, such as CD-ROMs, flash drives, etc. Furthermore, those skilled in the art will recognize that, without departing from the scope of exemplary embodiments of the present invention, the functions of various computing devices can be combined or integrated into a single computing device, or the functions of a particular computing device can be distributed across one or more other computing devices.

[0067] Although the invention has been described in detail with reference to embodiments, the embodiments described herein are not intended to be exhaustive or to limit the scope of the invention to the exact forms disclosed. Those skilled in the art will understand that changes and modifications can be made to the described structures, assembly, and operation methods without substantially departing from the principles, spirit, and scope of the invention. While some embodiments of this disclosure have been disclosed herein, the disclosure is not limited thereto, and its scope is defined by the claims and their equivalents.

Claims

1. A computer system configured to facilitate hospitality services within a resort, the computer system comprising: Multiple cameras are configured to capture images of at least a portion of the resort; A computing device that electronically communicates with the plurality of cameras, the computing device including a processor and a non-transitory memory device communicating with the processor; The non-transitory memory device includes computer-readable instructions that, when executed by the processor, cause the computing device to: Receive images captured by at least one of the plurality of cameras; and The computer vision module is used to determine the occurrence of a service event in an image captured by at least one of the plurality of cameras; and An electronic device that communicates electronically with the computing device, wherein the electronic device is configured to display information about the service event.

2. The computer system according to claim 1, wherein, When executed by the processor, the computer-readable instructions also cause the computing device to: The computer vision module is used to determine the execution status of the service related to the service event; and The execution status of the service is compared with the service standard stored in the non-transitory memory device; The electronic device is configured to display information about the service events and the service standards.

3. The computer system according to claim 2, wherein, The computing device is configured to provide access to the information via an application programming interface, or API.

4. The computer system according to claim 2, wherein, The electronic device is selected from the group consisting of tablet computers, computer monitors, virtual reality headsets, augmented reality headsets, smartwatches, smartphones, and headsets.

5. The computer system according to claim 1, wherein, The electronic device is configured to generate an alarm in response to the service failing to meet the service standard, and the alarm is either an audio alarm or a visual alarm.

6. The computer system according to claim 1, wherein, When executed by the processor, the instructions also cause the computing device to use a face recognition module to identify at least one individual in the images captured by at least one of the plurality of cameras.

7. The computer system according to claim 6, wherein, The electronic device is a display device, and the display device is configured to display a graphical user interface, the graphical user interface including a graphical representation of the layout of the portion of the resort, the location identifier of the at least one individual in the layout, and service standards associated with the at least one individual.

8. The computer system according to claim 7, wherein, The display device is also configured to display the identity of the at least one individual.

9. The computer system according to claim 8, wherein, The graphical user interface also includes a leaderboard, which contains multiple individual identities and multiple associated performance metrics.

10. The computer system according to claim 6, wherein, The non-transitory memory device includes a face recognition database, which includes multiple face images, multiple identifiers associated with the multiple face images, and multiple preferences associated with the multiple identifiers.

11. The computer system according to claim 10, wherein, The non-transitory storage device includes a computer vision database containing multiple events, multiple service tasks associated with the multiple events, and multiple service standards associated with the multiple service tasks.

12. The computer system according to claim 11, wherein, At least one of the multiple service standards is based on one of the multiple preferences.

13. The computer system according to claim 1, wherein, The computer vision module includes a convolutional neural network.

14. The computer system according to claim 1, wherein, When executed by the processor, the computer-readable instructions also cause the computing device to use the computer vision module to determine the occurrence of an event involving the at least one individual.

15. The computer system according to claim 14, wherein, The events are selected from the group consisting of the guest arriving at the restaurant, the table becoming available, the guest preparing to order, the guest finishing their drink, the guest finishing their meal, and the guest leaving the restaurant.

16. The computer system according to claim 1, wherein, The service standard is selected from a group consisting of duration, frequency, and number of occurrences.

17. The computer system according to claim 1, wherein, The service event is an interaction between one or more team members of the resort and one or more guests of the resort.

18. The computer system according to claim 1, wherein, The aforementioned portion of the resort is selected from the group consisting of a restaurant, reception area, meeting space, meeting room, guest room service area, swimming pool area, retail store and valet parking area.

19. The computer system according to claim 18, wherein, The service event refers to a service provided at at least one of the following: the restaurant, the front desk, the meeting space, the reception room, the guest room service area, the pool area, the retail store, and the valet parking area.

20. The computer system according to claim 6, wherein, The non-transitory memory device stores the preferences of the at least one individual, and wherein the computer-readable instructions, when executed by the processor, cause the display of the electronic device to display the name of the at least one individual and the preferences of the at least one individual in response to the face recognition module recognizing the at least one individual.

21. The computer system according to claim 20, wherein, The preferences include the dietary preferences of the at least one individual.

22. The computer system according to claim 20, wherein, The preferences include individual preferences selected from groups consisting of hotel room type and restaurant seating.

23. The computer system according to claim 1, wherein, The resort is a comprehensive resort.

24. The computer system according to claim 1, wherein, The services are selected from the group consisting of: guiding guests to empty tables in the restaurant, greeting guests already seated at the restaurant table, staff visiting guests seated at the restaurant table, staff refilling empty beverage glasses, staff clearing tables after the meal, staff delivering bills to guests, and staff turning over tables in the restaurant.

25. The computer system according to claim 24, wherein, The service standards are selected from the following groups: time spent seated at empty tables, time spent greeting guests, number of times staff visit seated guests, time spent refilling empty beverage glasses, time spent receiving food and / or drinks after a guest has ordered, time spent clearing tables, time spent delivering bills, and time spent turning tables over for the next guest.

26. The computer system according to claim 1, wherein, When executed by the processor, the computer-readable instructions also cause the computing device to: The computer vision module is used to detect at least one service problem in the images captured by at least one of the plurality of cameras; The facial recognition module is used to identify at least one individual in an image captured by at least one of the plurality of cameras, and the at least one individual is associated with the at least one service question; as well as Information about the at least one service problem and the identifier of the at least one individual associated with the at least one service problem are displayed on the display of the electronic device.

27. The computer system according to claim 1, wherein, When executed by the processor, the computer-readable instructions also cause the computing device to: The computer vision module is used to detect behaviors that indicate a guest at the resort needs assistance; and An alarm is generated on the electronic device, the alarm being configured to notify the resort staff that the guest needs assistance.

28. The computer system according to claim 1, wherein, When executed by the processor, the computer-readable instructions also cause the computing device to: The facial recognition module is used to detect at least one VIP guest in an image captured by at least one of the plurality of cameras; and An alarm is generated on the electronic device, which notifies the resort staff of the presence of at least one VIP guest.

29. The computer system according to claim 1, wherein, When executed by the processor, the computer-readable instructions also cause the computing device to: The computer vision module is used to detect multiple service events in the images captured by at least one of the plurality of cameras; The face recognition module is used to identify an individual in an image captured by at least one of the plurality of cameras, and the individual is associated with the plurality of service events; as well as The computing device tracks and stores the multiple service events as the individual moves within the resort.

Images