Server device, unpaid customer detection notification system, information processing method, and computer program

Abstract

To solve the problem that, in a gate system described in Patent Document 1, an alarm sound is emitted by a gate, causing the alarm to reverberate throughout the store and potentially making other customers feel uncomfortable.SOLUTION: A system acquires face feature amounts of settled customers, stores the acquired face feature amounts of the settled customers in a storage medium, acquires face feature amounts of passing customers, determines whether or not the face feature amounts of the passing customers match any of the face feature amounts of the settled customers stored in the storage medium, and, when it is determined that the face feature amounts of the passing customers do not match any of the face feature amounts of the settled customers, generates an unsettled customer detection notification indicating the detection of a customer suspected of not having settled his / her account, and transmits the generated unsettled customer detection notification to an employee output device.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a server device, an unpaid customer detection and notification system, an information processing method, and a computer program. [Background technology]

[0002] In recent years, self-ordering and fully self-service POS (Point of Sales) registers have become increasingly common, leading to an increase in stores reducing staff and moving to unmanned dining areas.

[0003] The gate system described in Patent Document 1 includes a gate that is linked to a cash register system. Patent Document 1 describes a technology that causes an alarm to sound at the gate if a customer does not pay properly. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-060107 Summary of the Invention [Problem to be solved by the invention]

[0005] The gate system described in Patent Document 1 sounds an alarm at the gate, which can reverberate throughout the store and cause discomfort to other customers.

[0006] The present invention aims to provide a server device, an unpaid customer detection and notification system, an information processing method, and a computer program that can sound an alarm in the customer seating area to notify employees of an abnormality without causing discomfort to other customers. [Means for solving the problem]

[0007] In one embodiment, the server device comprises a feature acquisition means for acquiring facial features of customers who have paid and facial features of passing customers; a memory control means for storing the acquired facial features of customers who have paid in a storage medium; a judgment means for determining whether the facial features of the passing customer match any of the facial features of customers who have paid stored in the storage medium; a notification generation means for generating an unpaid customer detection notification indicating the detection of a customer suspected of not paying if it is determined that the facial features of the passing customer do not match any of the facial features of customers who have paid; and a transmission means for transmitting the generated unpaid customer detection notification to an employee output device.

[0008] In another aspect that achieves the above object, an unpaid customer detection and notification system according to one embodiment includes a server device according to any one of the aspects, and an employee output device that receives an unpaid customer detection notification.

[0009] In yet another finding that achieves the above-mentioned object, an information processing method according to one embodiment acquires facial features of a customer who has paid, stores the acquired facial features of the customer who has paid in a storage medium, acquires facial features of a passing customer, determines whether the facial features of the passing customer match any of the facial features of the customers who have paid stored in the storage medium, and if it is determined that the facial features of the passing customer do not match any of the facial features of the customers who have paid, generates an unpaid customer detection notification indicating the detection of a customer who is suspected of not paying, and sends the generated unpaid customer detection notification to an employee output device.

[0010] In yet another finding that achieves the above-mentioned object, a computer program according to one embodiment causes a computer to perform the following processes: acquiring facial features of customers who have paid; storing the acquired facial features of customers who have paid in a storage medium; acquiring facial features of passing customers; determining whether the facial features of the passing customers match any of the facial features of customers who have paid stored in the storage medium; if it is determined that the facial features of the passing customers do not match any of the facial features of customers who have paid, generating an unpaid customer detection notification indicating the detection of a customer who is suspected of not paying; and sending the generated unpaid customer detection notification to an employee output device. [Effects of the Invention]

[0011] The present invention provides a server device, an unpaid customer detection and notification system, an information processing method, and a computer program that can notify employees of abnormalities without causing discomfort to other customers. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of a server system 1000 according to the present disclosure. [Figure 2] 1 is a block diagram showing an example of the configuration of a server device 100 according to the present disclosure. [Figure 3] 10 is a flowchart showing an example of a processing operation of the server device 100 according to the present disclosure. [Figure 4] 10 is a flowchart showing an example of a processing operation of a determining unit 130 according to the present disclosure. [Figure 5] 1 is a block diagram showing an example of the configuration of a server device 100 according to the present disclosure. [Figure 6] 10 is a flowchart showing an example of a processing operation of the server device 100 according to the present disclosure. [Figure 7] 1 is a block diagram showing an example of the configuration of a server device 100 according to the present disclosure. [Figure 8] 10 is a flowchart showing an example of a processing operation of the server device 100 according to the present disclosure. [Figure 9] 1 is a block diagram showing an example of the configuration of a server device 10 according to the present disclosure. [Figure 10] 2 is a diagram illustrating an example of a hardware configuration of a server device 100. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0013] An overview of one embodiment of the present invention will be described with reference to the drawings. Note that the reference numerals in the drawings attached to this overview are attached to each element for convenience as an example to facilitate understanding, and are not intended to limit the present invention to the illustrated form. Furthermore, connecting lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows are used to schematically indicate the flow of the main signal (data) and do not exclude bidirectionality.

[0014] First Embodiment <Configuration of the First Embodiment> FIG. 1 is a block diagram showing an example of a schematic diagram of a server system 1000 according to the first embodiment. As shown in FIG. 1, the server system 1000 according to this embodiment includes a server device 100 and an employee output device 400. To the server device 100, cameras 200-1 to 200-n (collectively referred to as cameras 200), full-self-service POS terminals 300-1 to 300-p (collectively referred to as full-self-service POS terminals 300), gates 500-1 to 500-q (collectively referred to as gates 500), cameras 600-1 to 600-r (collectively referred to as cameras 600), and a storage medium 700 are communicably connected via a communication network 800. Note that n, p, q, and r may be the same number or different numbers.

[0015] Camera 200 is, for example, a video camera. Camera 200 is installed in a position where it can capture an image of at least the face of a customer performing a checkout operation at full-self-service POS terminal 300. Specifically, camera 200 may be installed on the ceiling or wall of the store, or around full-self-service POS terminal 300. Camera 200 generates face image data representing the captured image.

[0016] The full-self-service POS terminal 300 is a device operated by a customer from reading a barcode to paying. In this embodiment, the full-self-service POS terminal 300 is connected to the server device 100 via a communication network 800. When a customer completes payment, the full-self-service POS terminal 300 causes the camera 200 to take a photograph. The camera 200 then captures the face of the customer who has completed payment at the full-self-service POS terminal 300. The full-self-service POS terminal 300 acquires facial image data of the customer who has completed payment generated by the camera 200. The full-self-service POS terminal 300 then performs a process to extract facial features of the customer from the acquired facial image data of the customer who has completed payment. The full-self-service POS terminal 300 transmits facial feature data indicating the extracted facial features to the server device 100. Note that facial feature data is a quantity indicating features corresponding to points, lines, or surfaces of facial features extracted or acquired from a facial image using a known feature extraction method.

[0017] The employee output device 400 is connected to the server device 100 via the communication network 800. The employee output device 400 is an employee device that outputs an unpaid customer detection notice, which is a notice indicating that an unpaid customer who is suspected of not completing payment has been detected. The employee output device 400 is, for example, a printer that prints out customer order information or a display that displays customer order information. The employee output device 400 is installed, for example, in a kitchen, a warehouse, a workshop, etc.

[0018] The gate 500 is connected to the server device 100 via a communication network 800. The gate 500 is installed on a route through which customers who are about to leave the store pass. The gate 500 includes, for example, a motion sensor installed in a position where it can detect a passing customer who is about to leave the store. When the motion sensor detects a customer (i.e., a passing customer), the gate 500 causes the camera 600 to take an image. The gate 500 also performs a process of extracting facial features of the passing customer from the facial image data of the passing customer generated by the camera 600. The gate 500 transmits facial feature data indicating the extracted facial features to the server device 100. The type of the gate 500 is not particularly limited. The method of detecting a passing customer by the gate 500 is not particularly limited.

[0019] The camera 600 is, for example, a camera capable of capturing still images. The camera 600 is installed, for example, in a position where it can capture an image of at least the face of a passing passenger. The camera 600 generates facial image data representing the captured image. The camera 600 may be installed, for example, at the gate 500.

[0020] The storage medium 700 is connected to the server device 100 via a communication network 800. The storage medium 700 stores data on customers who have paid and data on customers who have not paid. The data on customers who have paid is data indicating facial feature data of customers who have paid for the day, extracted from facial image data of customers who have paid for the day. The data on customers who have not paid is data in which facial feature data of passing customers who have been determined to be customers who have not paid is stored as facial feature data of customers who have not paid.

[0021] 2 is a block diagram showing an example of the configuration of the server device 100 according to the first embodiment. The server device 100 includes a feature acquisition unit 110, a storage control unit 120, a determination unit 130, a notification generation unit 140, and a transmission unit 150.

[0022] The feature acquisition unit 110 acquires facial feature data of customers who have paid and transmitted by the full-self-service POS terminal 300, and facial feature data of customers who have passed through and transmitted by the gate 500.

[0023] The storage control unit 120 stores the facial feature amount data of the customer who has already paid, acquired by the feature amount acquisition unit 110, in the storage medium 700 as data of the customer who has already paid.

[0024] The determination unit 130 acquires facial feature amount data of a passing customer from the feature amount acquisition unit 110. The determination unit 130 determines whether the facial feature amount indicated by the facial feature amount data of the passing customer matches any of the facial feature amounts indicated by the facial feature amount data of a customer who has already paid and is stored in the storage medium 700.

[0025] The matching process by the determination unit 130 is performed using, for example, various known methods. Specifically, the determination unit 130 calculates the similarity between the facial feature values ​​of a passing customer and the facial feature values ​​of each paid customer stored as paid customer data in the storage medium 700. The determination unit 130 compares each calculated similarity with a threshold. The determination unit 130 determines that a combination of the facial feature values ​​of a passing customer and the facial feature values ​​of a paid customer whose similarity is equal to or greater than a predetermined threshold is matched. The threshold value can be set as appropriate.

[0026] The number of facial features of a customer who has already paid and is the target of matching processing by the determination unit 130 may be one or more.

[0027] If the judgment unit 130 determines that the facial features of the passing customer do not match the facial features of any of the customers who have paid, the notification generation unit 140 generates an unpaid customer detection notification, which is a notification indicating that an unpaid customer has been detected.

[0028] The sending unit 150 sends the unpaid customer detection notice generated by the notice generating unit 140 to the employee output device 400. The sent unpaid customer detection notice is output by the employee output device 400. The employee output device 400 may output the unpaid customer detection notice by printing it, by displaying it on a display screen, by sound, or by any combination of these.

[0029] <Example of operation in the first embodiment> 3 is a flowchart showing an example of the processing operation of the server device 100 in the first embodiment. Here, the employee output device 400 is assumed to be a printer.

[0030] First, the feature acquisition unit 110 acquires facial feature data of a customer who has already paid from the full-self-service POS terminal 300 (step S101). Next, the memory control unit 120 stores the facial feature data of the customer who has already paid in the storage medium 700 as paid customer data (step S102). The feature acquisition unit 110 also acquires facial feature data of a passing customer from the gate 500 (step S103). Next, the determination unit 130 determines whether the facial feature data indicated by the acquired facial feature data of the passing customer matches any of the facial feature data indicated by the facial feature data of a paying customer stored in the storage medium (step S104). If the determination unit 130 determines that the facial feature values ​​of the passing customer do not match the facial feature values ​​of any paying customers (No in step S104), the notification generation unit 140 generates an unpaid customer detection notification including an instruction to output an alarm receipt indicating the detection of the unpaid customer from the employee output device 400 (step S105). The transmission unit 150 transmits the generated unpaid customer detection notification to the employee output device 400 (step S106).

[0031] The employee output device 400 then outputs the transmitted unpaid customer detection notification. As an example, the unpaid customer detection notification is configured to output an alarm receipt with characters printed on it indicating that an unpaid customer has been detected at the gate 500. The unpaid customer detection notification may also be configured to output a mark or the like on the alarm receipt.

[0032] FIG. 4 is a flowchart showing an example of the processing operation of the determination unit 130 in the first embodiment. First, the determination unit 130 acquires facial feature data of passing customers from the feature acquisition unit 110 (step S107). The determination unit 130 acquires facial feature data of customers who have paid for the day from the storage medium 700 (step S107). The determination unit 130 calculates the similarity between the facial feature of the passing customers and each facial feature of each paid customer (step S108). The determination unit 130 compares each calculated similarity with a threshold (step S109). If there is no combination of the facial feature of the passing customers and the facial feature of the paid customers for which the calculated similarity is equal to or greater than the predetermined threshold, the determination unit 130 determines that the facial feature of the passing customers does not match the facial feature of the paid customers (No in step S109) and proceeds to step S105. If there is a combination of facial features of a passing customer and facial features of a customer who has paid for their payment, for which the calculated similarity is greater than or equal to a predetermined threshold (Yes in step S109), the judgment unit 130 determines that the acquired facial features of the passing customer are consistent with the facial features of the customer who has paid for their payment, and terminates the processing.

[0033] <Advantages of the First Embodiment> The server device 100 of this embodiment includes a feature acquisition unit 110, a memory control unit 120, a determination unit 130, a notification generation unit 140, and a transmission unit 150. According to this embodiment, when an unpaid customer is detected based on the facial feature data of a paid customer and the facial feature data of a passing customer acquired by the feature acquisition unit 110, the result is notified to the employee output device 400, so that an employee can be notified of the abnormality without causing discomfort to other customers.

[0034] <Second embodiment> In the first embodiment described above, if it is determined that the facial features of a passing customer do not match those of any paying customers, a non-payment customer detection notice is generated and sent to the employee output device. In the second embodiment, if it is determined that the facial features of a passing customer do not match those of any paying customers, it is further determined whether the facial features of the passing customer match those of any stored non-payment customers. If it is determined that the facial features of the passing customer match those of any non-payment customers, it counts the number of times the facial features of the passing customer have been determined to match those of non-payment customers. The second embodiment differs from the first embodiment in that a non-payment customer detection notice including the counting results is generated and sent to the employee output device. The following description of the second embodiment will focus on the configuration different from the first embodiment, and the same configuration as the first embodiment will be assigned the same reference numerals and will not be described again.

[0035] Of the terms used in the second embodiment, the terms used in the first embodiment are used in the same meaning as the terms used in the first embodiment, unless otherwise specified.

[0036] <Configuration example of the second embodiment> 5 is a block diagram showing an example of the configuration of the server device 100 in the second embodiment. The server device 100 includes a feature acquisition unit 110, a memory control unit 120, a determination unit 130, a notification generation unit 140, a transmission unit 150, an unpaid customer determination unit 160 (e.g., equivalent to unpaid customer determination means), an unpaid customer storage unit 170 (e.g., equivalent to unpaid customer storage means), and a matching count counting unit 180 (e.g., equivalent to matching counting means).

[0037] If the determination unit 130 determines that the facial feature amount of the passing customer does not match the facial feature amount of any paying customer, the unpaid customer determination unit 160 acquires the facial feature amount data of each unpaid customer stored in the unpaid customer data in the storage medium 700. The unpaid customer determination unit 160 determines whether the facial feature amount of the passing customer matches any of the facial feature amounts indicated by the facial feature amount data of unpaid customers. The matching process by the unpaid customer determination unit 160 is performed using, for example, various known methods.

[0038] The unpaid customer storage unit 170 obtains the result from the unpaid customer determination unit 160. If the unpaid customer determination unit 160 determines that the facial feature amount of the passing customer does not match the facial feature amount of any unpaid customer, the unpaid customer storage unit 170 stores the facial feature amount data of the passing customer in the storage medium 700 as facial feature amount data of an unpaid customer.

[0039] If the unpaid customer determination unit 160 determines that the facial feature amount of a passing customer matches any of the facial feature amounts of unpaid customers, the match counting unit 180 obtains information on the number of matches corresponding to the facial feature amount of the unpaid customer that matches the facial feature amount of the passing customer from the unpaid customer data in the storage medium 700. The match counting unit 180 updates the value of the number of matches indicated by the obtained information on the number of matches to a value incremented by one. The number of matches is counted by updating the value of the number of matches indicated by the information on the number of matches.

[0040] The unpaid customer data stored in the storage medium 700 shown in FIG. 1 includes the number of matches, which is the number of times that the facial feature values ​​of a passing customer and an unpaid customer are determined to match.

[0041] <Example of operation in the second embodiment> Fig. 6 is a flowchart showing an example of the processing operation of the server device 100 in the second embodiment. In the flowchart shown in Fig. 6, steps S101 to S106 are the same as those in the first embodiment.

[0042] If the determination unit 130 determines in step S104 that the facial feature amount of the passing customer does not match the facial feature amount of any of the paying customers (No in step S104), the unpaid customer determination unit 160 determines whether the facial feature amount of the passing customer matches any of the facial feature amounts of the unpaid customers (step S201). Next, if the unpaid customer determination unit 160 determines that the facial feature amount of the passing customer does not match the facial feature amount of any of the unpaid customers (No in step S201), the unpaid customer storage unit 170 stores the facial feature amount data of the passing customer as facial feature amount data of the unpaid customer in the unpaid customer data in the storage medium 700 (step S202), and proceeds to the processing of step S105. In step S105, the same processing as in the first embodiment is performed.

[0043] If the unpaid customer determination unit 160 determines that the facial feature amount data of the passing customer matches any of the facial feature amount data of unpaid customers (Yes in step S201), the match counting unit 180 acquires information on the number of matches corresponding to the facial feature amount of the unpaid customer that matches the facial feature amount of the passing customer from the storage medium 700 (step S203). The match counting unit 180 updates the value of the number of matches indicated by the information on the number of matches acquired from the storage medium 700 by incrementing it by one (step S204). The notification generation unit 140 generates an unpaid customer detection notification including the updated value of the number of matches in the processing of step S204 (step S205). <Effects of the second embodiment> The server device 100 of this embodiment includes an unpaid customer determination unit 160, an unpaid customer storage unit 170, and a match count counting unit 180. According to this embodiment, if the unpaid customer determination unit 160 detects a passing customer as an unpaid customer based on the facial features of the passing customer and the facial features of the unpaid customer, it generates an unpaid customer detection notice including the number of matches and notifies the employee output device 400. With this configuration, it is possible to notify employees of the number of matches of unpaid customers without causing discomfort to other customers. <Third embodiment> While the first and second embodiments described above are intended to deal with customers when they leave the store, the third embodiment is intended to deal with customers when they arrive. The third embodiment differs from the first and second embodiments in that it is configured to determine whether a customer entering the store is a suspicious person, and if a suspicious person is detected, notify the employee output device 400 of the detection of the suspicious person. Note that a suspicious person is a customer who has been detected in the past as an unpaid customer and has reconciled a predetermined number of times or more. The following description of the third embodiment will focus on the configuration that differs from the first and second embodiments, and the same configuration as the previous embodiments will be assigned the same reference numerals and will not be described again.

[0044] Of the terms used in the third embodiment, terms that have also been used in the previous embodiments are used in the same sense as the terms used in the first and second embodiments, unless otherwise specified.

[0045] <Configuration example of the third embodiment> 7 is a block diagram showing an example of the configuration of the server device 100 in the third embodiment. The server device 100 includes a feature acquisition unit 110, a memory control unit 120, a determination unit 130, a notification generation unit 140, a transmission unit 150, an unpaid customer determination unit 160, an unpaid customer memory unit 170, a match counting unit 180, a customer feature acquisition unit 190 (e.g., corresponding to a customer feature acquisition means), and a suspicious person determination unit 210 (e.g., corresponding to a suspicious person determination means).

[0046] The gate 500 may be installed, for example, on a path through which customers entering the store pass. The gate 500 may include, for example, a motion sensor installed at a position where it can detect customers entering the store. When the motion sensor detects a customer, the gate 500 causes the camera 600 to take an image. The gate 500 also performs a process to extract facial features of the customer from the facial image data of the customer generated by the camera 600. The gate 500 transmits facial feature data indicating the extracted facial features to the server device 100. The type of the gate 500 is not particularly limited. The gate 500 may be installed in a location where it can detect customers entering the store and customers passing through.

[0047] The camera 600 is, for example, a camera capable of capturing still images. The camera 600 is installed, for example, in a position where it can capture an image of at least the face of the customer. The camera 600 generates facial image data representing the captured image. The camera 600 may be installed, for example, at the gate 500.

[0048] The unpaid customer determination unit 160 determines whether the facial feature values ​​indicated by the facial feature value data of the customer acquired from the customer feature value acquisition unit 190 match any of the facial feature values ​​of unpaid customers. The matching process by the unpaid customer determination unit 160 is performed using, for example, various known methods.

[0049] The customer feature amount acquisition unit 190 acquires facial feature amount data of the customer from the gate 500.

[0050] If the non-checkout customer determination unit 160 determines that the facial feature amounts of the customer match those of any of the non-checkout customers, the suspicious individual determination unit 210 obtains information on the number of matches corresponding to the facial feature amounts of the non-checkout customer that was determined to match the facial feature amounts of the customer from the storage medium 700. If the obtained number of matches is equal to or greater than a predetermined number, the suspicious individual determination unit 210 determines that the customer is a suspicious individual.

[0051] The suspicious individual determination unit 210 may determine whether a customer is a suspicious individual without performing the processing of the unpaid customer determination unit 160. As an example, the facial feature amount data of an unpaid customer that corresponds to a predetermined number of matches or more may be flagged as suspicious individual data in advance, or may be stored separately from the unpaid customer data in the storage medium 700. The suspicious individual determination unit 210 may then determine whether a customer is a suspicious individual based on whether the facial feature amount of the customer matches any of the facial feature amounts in the suspicious individual data stored in the storage medium 700.

[0052] The predetermined number of times may be determined in advance. The predetermined number of times may be the number of times that unpaid customers have been reconciled per store, or may be the number of times that unpaid customers have been reconciled shared among all stores.

[0053] The notification generating unit 140 generates a suspicious person detection notification indicating that the customer is a suspicious person.

[0054] The transmission unit 150 transmits the generated alert person detection notification to the employee output device 400.

[0055] <Example of operation in the third embodiment> FIG. 8 is a flowchart showing an example of the processing operation of the server device 100 in the third embodiment.

[0056] First, the feature acquisition unit 110, the storage control unit 120, and the determination unit 130 perform the same processes as in the first embodiment from step S101 to step S104.

[0057] In step S301, the customer feature acquisition unit 190 acquires facial feature data of the customer from the gate 500 (step S301). The non-payment customer determination unit 160 determines whether the facial feature data of the customer matches any of the facial feature data of non-payment customers (step S302). If the non-payment customer determination unit 160 determines that the facial feature data of the customer matches any of the facial feature data of non-payment customers (Yes in step S302), the suspicious individual determination unit 210 determines whether the customer is a suspicious individual (step S303). If the suspicious individual determination unit 210 determines that the customer is a suspicious individual (Yes in step S303), the notification generation unit 140 generates a suspicious individual detection notification including an instruction to cause the employee output device 400 to output that a suspicious individual has been detected (step S304). The transmission unit 150 transmits the generated alert person detection notification to the employee output device 400.

[0058] The employee output device 400 then outputs the transmitted warning person detection notification. As an example, the warning person detection notification is configured to output an alarm receipt with text printed on it indicating that a warning person has been detected at the gate 500. The unpaid customer detection notification may also be configured to output a mark or the like on the alarm receipt.

[0059] If the non-payment customer determination unit 160 determines that the customer is not a person to be suspected (No in step S303), the suspected person determination unit 210 ends the process.

[0060] <Advantages of the third embodiment> The server device 100 of this embodiment includes a customer feature acquisition unit 190 and a suspicious individual determination unit 210. According to this embodiment, the server device is configured to determine whether a customer is a suspicious individual based on the facial features indicated by the customer's facial feature data acquired by the customer feature acquisition unit 190 and the facial features of customers who have not yet paid, and if the customer is a suspicious individual, generate a suspicious individual detection notification and notify the employee output device 400. With this configuration, it is possible to notify an employee of the detection of a suspicious individual when the customer arrives at the store without causing discomfort to other customers.

[0061] <Fourth embodiment> A fourth embodiment of the present invention will be described with reference to the drawings.

[0062] <Configuration of the Fourth Embodiment> 9 is a block diagram showing an example of the configuration of the server device 10 in the fourth embodiment. The server device 10 includes a feature acquisition unit 410 (e.g., corresponding to feature acquisition means), a storage control unit 420 (e.g., corresponding to storage control means), a determination unit 430 (e.g., corresponding to determination means), a notification generation unit 440 (e.g., corresponding to notification generation means), and a transmission unit 450 (e.g., corresponding to transmission means).

[0063] The feature amount acquiring unit 410 acquires facial feature amounts of customers who have already paid and facial feature amounts of customers who are passing through.

[0064] The storage control unit 420 stores the acquired facial feature amounts of the customer who has completed payment in a storage medium.

[0065] The determination unit 430 determines whether the facial feature amount of the passing customer matches any of the facial feature amounts of customers who have already paid and are stored in the storage medium.

[0066] If the notification generating unit 440 determines that the facial feature amount of the passing passenger does not match the facial feature amount of any passenger who has already paid, it generates an unpaid passenger detection notification indicating that a passenger suspected of not paying has been detected.

[0067] The transmitting unit 450 transmits the generated unpaid customer detection notice to the employee output device. <Example of operation in the fourth embodiment> An example of the processing operation of the server device 10 in the fourth embodiment will be described.

[0068] First, the feature acquisition unit 410 acquires the facial features of the customer who has already paid and the facial features of the passing customer. Next, the memory control unit 420 stores the acquired facial features of the paying customer in a storage medium. The judgment unit 430 determines whether the facial features of the passing customer match any of the facial features of the paying customers stored in the storage medium. If the notification generation unit 440 determines that the facial features of the passing customer do not match any of the facial features of the paying customers, it generates an unpaid customer detection notification indicating the detection of a customer suspected of not paying. The transmission unit 450 transmits the generated unpaid customer detection notification to an employee output device. <Effects of the Fourth Embodiment> The server device 10 of this embodiment includes a feature acquisition unit 410, a memory control unit 420, a determination unit 430, a notification generation unit 440, and a transmission unit 450. According to this embodiment, when an unpaid customer is detected based on the facial feature data of a paid customer and the facial feature data of a passing customer acquired by the feature acquisition unit 410, the result is notified to the employee output device, so that an employee can be notified of the abnormality without causing discomfort to other customers.

[0069] The above embodiments may be implemented in appropriate combinations.

[0070] In addition, if the judgment unit 130 determines that the facial features of a passing customer or a visiting customer match any of the facial features of a customer who has already paid, the transmission unit 150 may further include means for transmitting a signal to open the flapper gate of gate 500.

[0071] <Modification> The notification generation unit 140 may generate a notification of detection of an unpaid customer that includes an instruction to output, at regular intervals, a display screen indicating the detection of an unpaid customer and a display screen with reduced brightness, alternately on the display of the employee output device 400. This configuration has the effect of increasing the likelihood that an employee will notice the detection of an unpaid customer by flashing the display screen.

[0072] The notification generator 140 may generate a notification of an unpaid customer detection that includes an instruction to output a notification sound indicating the detection of an unpaid customer. This configuration has the effect of enabling employees to notice that an unpaid customer has been detected without paying attention to the employee output device 400.

[0073] The type of notification sound is not particularly limited, and may be, for example, a voice or an alarm sound.

[0074] The notification generation unit 140 may generate a notification of detection of an unpaid customer that includes an instruction to turn on a lamp on the employee output device 400 to indicate that an unpaid customer has been detected. This configuration has the effect of turning on the lamp on the employee output device 400, thereby increasing the likelihood that an employee will notice that an unpaid customer has been detected.

[0075] The notification generation unit 140 may obtain information about the time when an unpaid customer was detected from the gate 500, and generate an unpaid customer detection notification that includes information about the detection time, which is the time when the gate 500 detected the unpaid customer. This configuration has the effect of helping employees identify unpaid customers.

[0076] The notification generation unit 140 may obtain, from gate 500, information indicating the location of the gate at which the unpaid customer was detected, and generate an unpaid customer detection notification including the information indicating the location of the gate at which the unpaid customer was detected. The notification generation unit 140 may also obtain, from gate 500, information identifying the device at the gate at which the unpaid customer was detected, and generate an unpaid customer detection notification including the information identifying the device at the gate at which the unpaid customer was detected. Such a configuration has the effect of assisting staff in identifying unpaid customers.

[0077] The notification generator 140 may generate a notification of unpaid customer detection that includes information about the unpaid customer that is inferred from the facial features of the unpaid customer. The information about the unpaid customer may be, for example, information about the customer's age and gender that is inferred from the facial features of the unpaid customer. This configuration has the effect of helping employees identify unpaid customers.

[0078] The notification generating unit 140 may use any combination of the above-mentioned information on the detection time, information indicating the gate position, information identifying the gate device, and information on the unpaid customer.

[0079] The facial feature amount is not limited to the facial features of a person, and may be any feature amount that can identify a person. For example, the facial feature amount may be a feature amount extracted or acquired from the shape of an accessory worn by the person.

[0080] The camera 200 may be a camera that captures still images.

[0081] The camera 200 may be installed in a position where it can capture an image of the face of a customer who has completed payment from the front. The installation location of the camera 200 is set appropriately depending on the internal structure of the store and the imaging range of the camera 200. The camera 200 may be installed either built into the full-self-service POS terminal 300 or on top of the full-self-service POS terminal 300. In this way, by capturing an image of the face of a customer who has completed payment from the front, detailed facial features of the customer who has completed payment can be extracted, allowing for more accurate judgment.

[0082] The number of cameras installed may correspond to the number of full-self-service POS terminals 300. The cameras 200 may extract feature amounts from facial image data of passing customers.

[0083] The type, installation location, and number of full-self-service POS terminals 300 are not particularly limited.

[0084] The number of employee output devices 400 is not particularly limited.

[0085] The location and number of gates 500 are determined appropriately depending on the internal structure of the store.

[0086] The camera 600 may be a video camera. The camera 600 may be installed, for example, in a position where it can capture a frontal image of the face of a customer passing through the gate 500. The camera 600 may extract features from facial image data of the passing customer.

[0087] <Hardware configuration example> Next, a description will be given of the hardware of each device constituting the server device 100 according to the present disclosure.

[0088] The server device 100 can be configured by an information processing device (so-called computer), and has the configuration exemplified in Fig. 10. For example, the server device 100 has a processor 911, a memory 912, an input / output interface 913, a communication interface 914, etc. The components such as the processor 911 are connected by an internal bus or the like, and are configured to be able to communicate with each other.

[0089] However, the configuration shown in Fig. 10 is not intended to limit the hardware configuration of the server device 100. The server device 100 may include hardware not shown. Furthermore, the number of processors 911 and the like included in the server device 100 is not intended to be limited to the example shown in Fig. 10, and for example, the server device 100 may include multiple processors 911.

[0090] The processor 911 is a programmable device such as a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP). Alternatively, the processor 911 may be a device such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The processor 911 executes various programs including an operating system (OS).

[0091] The memory 912 is a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), etc. The memory 912 stores an OS program, application programs, and various data.

[0092] The input / output interface 913 is an interface for a display device and an input device (not shown). The display device is, for example, a liquid crystal display, etc. The input device is, for example, a device that accepts user operations such as a keyboard or a mouse.

[0093] The communication interface 914 is a circuit, module, etc. that communicates with other devices. For example, the communication interface 914 includes a wireless communication circuit, a NIC (Network Interface Card), etc.

[0094] The functions of the server device 100 are realized by various processing modules. The processing modules are realized, for example, by the processor 911 executing a program stored in the memory 912. The program can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. That is, the present invention can also be embodied as a computer program product. The program can be downloaded via a network or updated using a storage medium storing the program. The processing modules can also be realized by a semiconductor chip.

[0095] The full-self service POS terminal 300, storage medium 700, etc. can also be configured by information processing devices in the same way as the server device 100, and their basic hardware configurations are no different from those of the server device 100, so a description thereof will be omitted.

[0096] The server device 100 is equipped with a computer, and the computer executes a program to realize the functions of the server device 100. The server device 100 also executes a control method for the server device 100 by the program.

[0097] [Appendix 1] a feature acquisition means for acquiring facial feature values ​​of customers who have already paid and facial feature values ​​of customers who are passing through; a storage control means for storing the acquired facial feature amount of the customer who has completed payment in a storage medium; a determination means for determining whether the facial feature amount of the passing passenger matches any of the facial feature amounts of passengers who have already paid and are stored in the storage medium; a notification generating means for generating a notification of detection of an unpaid customer indicating that a customer suspected of not paying has been detected when it is determined that the facial feature amount of the passing customer does not match the facial feature amount of any of the customers who have paid; a transmitting means for transmitting the generated unpaid customer detection notice to an output device for employee use; A server device comprising:

[0098] [Appendix 2] The server device described in 1, wherein the notification generating means generates the unpaid customer detection notification, causing the employee output device to output the contents of the unpaid customer detection notification in a format different from when the customer placed an order.

[0099] [Appendix 3] The server device described in Appendix 2, wherein the notification generating means generates the unpaid customer detection notification by causing the employee output device to print the contents of the unpaid customer detection notification in a format different from that at the time the customer placed their order.

[0100] [Appendix 4] The server device described in Appendix 2, wherein the notification generating means generates the unpaid customer detection notification, which displays the contents of the unpaid customer detection notification on the employee output device in a manner different from when the customer places an order.

[0101] [Appendix 5] The server device described in Appendix 2, wherein the notification generating means generates the unpaid customer detection notification by outputting the contents of the unpaid customer detection notification to the employee output device using a sound different from that used when a customer places an order.

[0102] [Appendix 6] an unpaid customer determination means for determining whether the facial feature amount of the passing customer matches any of the facial feature amounts of unpaid customers stored in the storage medium when it is determined that the facial feature amount of the passing customer does not match any of the facial feature amounts of paid customers; an unpaid customer storage means for storing the facial feature values ​​of the passing customer in the storage medium as the facial feature values ​​of an unpaid customer when it is determined that the facial feature values ​​of the passing customer do not match the facial feature values ​​of any unpaid customer; 2. The server device according to claim 1, comprising:

[0103] [Appendix 7] and a match counting means for counting the number of times that the facial feature values ​​of the passing passengers and the facial feature values ​​of the unpaid passengers match, depending on the result of the determination by the unpaid passenger determining means. Equipped with The server device according to claim 6, wherein the notification generating means generates the unpaid customer detection notification including the counting result.

[0104] [Appendix 8] A server device according to any one of Supplementary Note 1 to Supplementary Note 7; the employee output device for receiving the notification of the detection of the unpaid customer; An unpaid customer detection and notification system comprising:

[0105] [Appendix 9] Acquire facial features of customers who have completed payment, storing the acquired facial feature amount of the customer who has completed payment in a storage medium; Acquire the facial features of passing passengers, determining whether the facial feature amount of the passing passenger matches any of the facial feature amounts of passengers who have already paid and are stored in the storage medium; If it is determined that the facial feature amount of the passing passenger does not match the facial feature amount of any of the passengers who have paid, an unpaid passenger detection notice is generated indicating that a passenger suspected of not paying has been detected; transmitting the generated unpaid customer detection notification to an employee output device; 1. An information processing method comprising:

[0106] [Appendix 10] On the computer, A process of acquiring facial features of a customer who has completed payment; a process of storing the acquired facial feature amount of the customer who has completed payment in a storage medium; A process of acquiring facial features of passing passengers; a process of determining whether the facial feature amount of the passing passenger matches any of the facial feature amounts of passengers who have already paid and are stored in the storage medium; a process of generating a notification of detection of an unpaid customer indicating that a customer suspected of not paying has been detected when it is determined that the facial feature amount of the passing customer does not match the facial feature amount of any of the customers who have paid; a process of transmitting the generated unpaid customer detection notice to an employee output device; A computer program for executing

[0107] [Appendix 11] Furthermore, a customer feature amount acquisition means for acquiring a facial feature amount of a customer; a means for determining whether or not a customer is a person requiring caution using the results of the counting; Equipped with The server device described in Appendix 7, characterized in that the notification generation means generates a suspicious person detection device indicating that the customer is a suspicious person when it is determined that the customer is a suspicious person.

[0108] The disclosures of the above-cited prior art documents are incorporated herein by reference. Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments. Those skilled in the art will understand that these embodiments are merely illustrative and that various modifications are possible without departing from the scope and spirit of the present invention. In other words, the present invention naturally includes various modifications and alterations that may be made by those skilled in the art in accordance with the entire disclosure, including the claims, and the technical concepts thereof.

[0109] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 7 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 8, 9, and 10 in the same dependent relationship as Supplementary Notes 2 to 7. Furthermore, not limited to Supplementary Notes 1, 8, 9, and 10, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments. [Explanation of symbols]

[0110] 10 Server device 100 Server device 110 Feature acquisition unit 120 Memory control unit 130 Judgment Department 140 Notification generator 150 Transmitter 160 Unpaid Customer Judgment Department 170 Unpaid customer storage section 180 Matching counting unit 190 Customer feature acquisition unit 200 cameras 210 Vigilance Person Judgment Department 300 Full-self POS terminals 400 Employee Output Devices 410 Feature acquisition unit 420 Memory control unit 430 Judgment Department 440 Notification generator 450 Transmitter 500 gates 600 cameras 700 Storage medium 800 Communications Network 911 Processor 912 memory 913 Input / Output Interface 914 Communication Interface 1000 Server System

Claims

1. a feature acquisition means for acquiring facial feature values ​​of customers who have already paid and facial feature values ​​of customers who are passing through; a storage control means for storing the acquired facial feature amount of the customer who has completed payment in a storage medium; a determination means for determining whether the facial feature amount of the passing passenger matches any of the facial feature amounts of passengers who have already paid and are stored in the storage medium; a notification generating means for generating a notification of detection of an unpaid customer indicating that a customer suspected of not paying has been detected when it is determined that the facial feature amount of the passing customer does not match the facial feature amount of any of the customers who have paid; a transmitting means for transmitting the generated unpaid customer detection notice to an output device for employee use; A server device comprising:

2. 2. The server device according to claim 1, wherein the notification generating means generates the unpaid customer detection notification by causing the employee output device to output the contents of the unpaid customer detection notification in a format different from that when the customer places an order.

3. 3. The server device according to claim 2, wherein the notification generating means generates the unpaid customer detection notification by causing the employee output device to print the contents of the unpaid customer detection notification in a format different from that at the time of the customer's order.

4. 3. The server device according to claim 2, wherein the notification generating means generates the unpaid customer detection notification by displaying the contents of the unpaid customer detection notification on the employee output device in a format different from that when the customer places an order.

5. The server device according to claim 2, wherein the notification generating means generates the unpaid customer detection notification by causing the employee output device to output the contents of the unpaid customer detection notification using a sound different from that used when a customer places an order.

6. an unpaid customer determination means for determining whether the facial feature amount of the passing customer matches any of the facial feature amounts of unpaid customers stored in the storage medium when it is determined that the facial feature amount of the passing customer does not match any of the facial feature amounts of paid customers; an unpaid customer storage means for storing the facial feature values ​​of the passing customer in the storage medium as the facial feature values ​​of an unpaid customer when it is determined that the facial feature values ​​of the passing customer do not match the facial feature values ​​of any unpaid customer; 2. The server device according to claim 1, further comprising:

7. and a match counting means for counting the number of times that the facial feature values ​​of the passing passengers and the facial feature values ​​of the unpaid passengers match, depending on the result of the determination by the unpaid passenger determining means. Equipped with 7. The server device according to claim 6, wherein the notification generating means generates the unpaid customer detection notification including the counting result.

8. The server device according to any one of claims 1 to 7; the employee output device for receiving the notification of the detection of the unpaid customer; An unpaid customer detection and notification system comprising:

9. Acquire facial features of customers who have completed payment, storing the acquired facial feature amount of the customer who has completed payment in a storage medium; Acquire the facial features of passing passengers, determining whether the facial feature amount of the passing passenger matches any of the facial feature amounts of passengers who have already paid and are stored in the storage medium; If it is determined that the facial feature amount of the passing passenger does not match the facial feature amount of any of the passengers who have paid, an unpaid passenger detection notice is generated indicating that a passenger suspected of not paying has been detected; transmitting the generated unpaid customer detection notification to an employee output device; 1. An information processing method comprising:

10. On the computer, A process of acquiring facial features of a customer who has completed payment; a process of storing the acquired facial feature amount of the customer who has completed payment in a storage medium; A process of acquiring facial features of passing passengers; a process of determining whether the facial feature amount of the passing passenger matches any of the facial feature amounts of passengers who have already paid and are stored in the storage medium; a process of generating a notification of detection of an unpaid customer indicating that a customer suspected of not paying has been detected when it is determined that the facial feature amount of the passing customer does not match the facial feature amount of any of the customers who have paid; a process of transmitting the generated unpaid customer detection notice to an employee output device; A computer program for executing

Images