Investigation device, investigation method, and program

Abstract

To efficiently investigate matters regarding use of a service by a user in a service providing space.SOLUTION: An investigation device includes a video acquisition unit that acquires respective videos obtained by one or more imaging devices disposed to shoot a prescribed service providing space, a tracking unit that tracks the position of a user in the service providing space on the basis of the videos acquired by the video acquisition unit, an inference unit that infers a service use state of the user on the basis of a relationship between the position of the user tracked by the tracking unit and a prescribed position range which is defined in the service providing space, and an information generating unit that generates, for each user, service use state information which indicates the service use state inferred by the inference unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a survey device, a survey method, and a program. [Background technology]

[0002] There is known a technique for tracking a person moving in a managed area by using a plurality of cameras arranged in the managed area (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-57885 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in spaces where services are provided (service providing spaces) such as financial institutions, government offices, post offices, medical institutions, etc., it is preferable to be able to efficiently investigate matters relating to users' use of services.

[0005] In consideration of the above-mentioned problems, an object of the present invention is to make it possible to efficiently investigate matters related to a user's use of services in a service provision space. [Means for solving the problem]

[0006] One aspect of the present invention that solves the above-mentioned problems is a survey device that includes an image acquisition unit that acquires images obtained by each of one or more imaging devices configured to image a specified service provision space, a tracking unit that tracks a user's position in the service provision space based on the images acquired by the image acquisition unit, an estimation unit that estimates the user's service usage status based on the relationship between the user's position tracked by the tracking unit and a specified positional range defined in the service provision space, and an information generation unit that generates service usage status information for each user that indicates the service usage status estimated by the estimation unit.

[0007] One aspect of the present invention is a survey method in a survey device, the survey method including: an image acquisition step in which an image acquisition unit acquires images obtained by each of one or more imaging devices configured to image a specified service provision space; a tracking step in which a tracking unit tracks a position of a user in the service provision space based on the images acquired by the image acquisition step; an estimation step in which an estimation unit estimates the user's service usage status based on the relationship between the position of the user tracked by the tracking step and a specified positional range defined in the service provision space; and an information generation step in which an information generation unit generates, for each user, service usage status information indicating the service usage status estimated by the estimation step.

[0008] One aspect of the present invention is a program for causing a computer serving as an investigation device to function as an image acquisition unit that acquires images obtained by each of one or more imaging devices configured to image a specified service provision space, a tracking unit that tracks the position of a user in the service provision space based on the images acquired by the image acquisition unit, an estimation unit that estimates the user's service usage status based on the relationship between the position of the user tracked by the tracking unit and a specified positional range defined in the service provision space, and an information generation unit that generates, for each user, service usage status information indicating the service usage status estimated by the estimation unit. Effect of the Invention

[0009] According to the present invention, it is possible to efficiently investigate matters relating to the use of services by users in a service provision space. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a survey system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of an outline of a service usage status survey according to the present embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a survey device according to the present embodiment. [Figure 4] FIG. 4 is a diagram showing an example of imaging area information in the present embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of service usage status information in the present embodiment. [Figure 6] 11 is a diagram illustrating an example of a processing procedure executed by the investigation device in the present embodiment in relation to generating service usage status information. [Figure 7] 11 is a diagram showing an example of service usage status information corresponding to multiple business cases in the present embodiment. FIG. [Figure 8] 11 is a diagram showing an example of service usage status information corresponding to a re-entry case in the present embodiment. FIG. [Figure 9] 11 is a diagram showing an example of service usage status information corresponding to an uncompleted business exit case in the present embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <Embodiment> The investigation system of this embodiment will be described below. The investigation system of this embodiment is a system for investigating the usage of services by customers (examples of users) in facilities (examples of service provision spaces) that have counters for each purpose (service), such as a financial institution's branch, a government office, a post office, etc. In the following explanation, an example will be given in which the investigation system of this embodiment is configured to investigate the usage of services by customers at each branch of a financial institution.

[0012] Fig. 1 shows an example of the configuration of a survey system according to this embodiment. The survey system in Fig. 1 shows an example in which a survey device 100 surveys the service usage status of users at each store ST of one or more financial institutions. The store ST shown in Fig. 1 may be, for example, a branch office of a specific financial institution.

[0013] In the store ST, a plurality of imaging devices 200 are provided. The plurality of imaging devices 200 are arranged such that a predetermined imaging position, imaging direction, and imaging range are determined so as to image a predetermined area in the store ST, respectively. An integrated imaging range obtained by integrating the imaging ranges of the plurality of imaging devices 200 may be set to include a spatial range in the store ST in which customers may be present. In this case, the integrated imaging range may include a space corresponding to the entire area of ​​the store ST, or may exclude spaces in the entire area of ​​the store ST in which it is possible that customers may not be present. In the video captured by each of the imaging devices 200, for example, pixels are associated with coordinates indicating a position within the store ST, so that a position within the store ST in the video can be identified. The number of imaging devices 200 arranged in the store ST may be any number equal to or greater than one, as long as it can cover a required spatial range within the store ST.

[0014] The video storage device 300 is installed in a store ST, and is connected to each of the imaging devices 200 arranged in the same store ST via a data interface or a communication line. The video storage device 300 stores the video captured by each of the imaging devices 200 arranged in the same store ST. That is, the video storage device 300 accumulates the video captured by the imaging devices 200 in the store ST. The video storage device 300 stores, for example, an imaging device ID that uniquely identifies the imaging device 200 and the video output from the imaging device 200 in association with each other. In addition, a time (time stamp) is added to the video.

[0015] As described above, each of the multiple stores ST includes a predetermined number of imaging devices 200 and video storage devices 300. The number of imaging devices 200 in a store ST may be changed as appropriate depending on the size of the space in the corresponding store ST, the location of the counter, the location of the equipment, and the like.

[0016] The investigation device 100 investigates the service usage status of customers at each store ST. When investigating the service usage status of customers at one store ST, the investigation device 100 acquires video stored in the video storage device 300 of the corresponding store ST and analyzes the acquired video to investigate the service usage status of each corresponding customer who used the corresponding store ST. The survey device 100 may be, for example, a server provided at the headquarters of a financial institution that the store ST deals with, or may be a cloud server provided on a network. The survey device 100 may integrate the service usage status of customers for each store ST to obtain survey results on the service usage status of customers corresponding to, for example, all financial institutions in the same group.

[0017] The information output terminal 400 is, for example, a terminal used by a financial institution's staff member, and is communicatively connected to the survey device 100. The information output terminal 400 outputs the survey results transmitted by the survey device 100 by displaying, printing, or the like. A plurality of information output terminals 400 may be provided in the survey system as appropriate.

[0018] An example of an outline of a service usage status survey for one customer CS in a store ST will be described with reference to Fig. 2. The figure shows a plan view of the interior of the store ST. In the store ST, imaging devices 200 are disposed at predetermined positions. Each of the imaging devices 200 captures an image of a predetermined spatial range in the store ST.

[0019] The figure shows an example of a store ST equipped with a numbered ticket issuing machine 500 and two waiting benches BC (BC-1, BC-2), and six counters LS (LS-1 to LS-6) corresponding to numbers 1 to 6. Of the counters LS-1 to LS-6, counters LS-1 and LS-2 each correspond to "deposits," counters LS-3 and LS-4 each correspond to "insurance," and counters LS-5 and LS-6 each correspond to "loans." The waiting bench BC is an example of a waiting place, and the window LS is an example of a service providing place.

[0020] In this embodiment, a presence identification area (an example of a location range) is set corresponding to a location such as a waiting bench BC or a window LS. A presence identification area is a location range in which a customer is considered to be present in the corresponding location. Specifically, the survey device 100 sets the presence identification area by storing a map (store map) of the store ST to which information on the location (coordinate) range indicating the presence identification area is added. Specifically, in the figure, an example is shown in which presence identification areas AR1 (AR1-1, AR1-2) are set corresponding to waiting benches BC (BC-1, BC-2). Also shown is an example in which a presence identification area AR2 (AR2-1 to AR2-6) is set corresponding to each counter LS (LS-1 to LS-6). The presence identification area AR2 corresponding to the counter LS is associated with a corresponding service type. Specifically, "deposits" is associated with the presence identification areas AR2-1 and AR2-2, "insurance" is associated with the presence identification areas AR2-3 and AR2-4, and "loans" is associated with the presence identification areas AR2-5 and AR2-6.

[0021] In the figure, for example, a case is shown in which a customer CS acts in accordance with the business at a store ST as follows. A customer CS enters the store ST through the entrance ENT. After entering the store ST through the entrance ENT, the customer CS is constantly being imaged by one of the imaging devices 200 depending on the location within the store ST until the customer CS completes his / her business and leaves through the exit EXT.

[0022] A customer CS who enters the store ST moves to the ticket issuing machine 500 as shown by arrow A. At the ticket issuing machine 500, the customer CS specifies the type of service to be used at the store ST according to his / her purpose and performs an operation to issue a numbered ticket. The ticket issuing machine 500 issues a numbered ticket in response to the operation. The numbered ticket shows the numbered ticket number accepted by the counter LS corresponding to the specified service type purpose. The staff at the counter LS attend to customers in ascending order of the numbered ticket numbers. In this case, the service type designated by the customer CS corresponds to "insurance," and the ticket number on the issued ticket indicates the order in which the customer will be served by staff at the insurance counters LS-3 and LS-4.

[0023] The customer CS holds the numbered ticket issued by the numbered ticket issuing machine 500 and waits at the service window LS until it is his / her turn to be served. Therefore, the customer CS sits on a waiting bench and waits. As shown by the arrow B in the figure, the customer CS moves to waiting bench BC-1 out of waiting benches BC-1 and BC-2, sits on waiting bench BC-1, and waits his / her turn.

[0024] The customer CS's numbered ticket number is now available at window 4. Therefore, the customer CS moves to window 4 as shown by arrow C. At window 4, the customer CS is attended to by a person in charge according to his / her need.

[0025] When customer CS has completed his / her business at window No. 4, he / she leaves store ST through exit EXT as shown by arrow D.

[0026] For customers CS who use services while moving about in store ST as described above, the survey device 100 obtains the following information (service usage status information) as survey results regarding service usage, based on the images captured by the imaging device 200. First, the survey device 100 obtains, as service usage status information, the time when the customer CS enters the store ST through the entrance ENT (entrance time).

[0027] In this case, since the customer CS was seated on the waiting bench BC-1 and waiting, the survey device 100 determines that the state in which the position (customer position) identified for the customer CS was included in the presence identification area AR1-1 corresponding to the waiting bench BC-1 was longer than a certain period of time. In other words, the survey device 100 determines that the customer waited on the waiting bench BC-1 until receiving the service (receiving assistance at the counter). In this case, the survey device 100 obtains, as the service usage status information, the waiting time until the customer CS was served at the counter. The waiting time may be obtained based on the time the customer position was included in the presence identification area AR1-1. In addition, at this time, the survey device 100 may obtain, as the service usage status information, information indicating that the waiting location of the customer CS is the waiting bench BC-1, based on the fact that the customer position was included in the presence identification area AR1-1 corresponding to the waiting bench BC-1.

[0028] In this case, the customer CS waits on waiting bench BC-1, and is then attended to by a staff member at counter LS-4. In this case, the investigation device 100 determines that the customer CS has used the service type "insurance" corresponding to counter LS-4, based on the fact that the customer CS's position has been in presence identification area AR2-4 corresponding to counter LS-4 for a certain period of time or more. Therefore, the investigation device 100 obtains information indicating that the service type used by the customer CS is "insurance" as service usage status information. In addition, at this time, the survey device 100 may obtain information indicating that the service usage location (service counter) where the customer CS received assistance is counter LS-4 as service usage status information, based on the fact that the customer's location was included in the presence identification area AR2-4 corresponding to counter LS-4.

[0029] When the customer CS finishes his / her business at the counter LS-4, he / she exits through the exit EXT. The investigation device 100 obtains, as the service usage status information, the time (exit time) corresponding to the timing at which the customer CS exits through the exit EXT and disappears from the image captured by the imaging device 200.

[0030] Furthermore, the survey device 100 may estimate the gender, age, height, and the like as biometric attributes (customer attributes) of the customer CS based on the feature amount of the customer CS personally recognized from the image captured by the imaging device 200.

[0031] In this way, the survey device 100 can obtain information such as entry time, waiting time, waiting location, service type, service window, exit time, and customer attributes corresponding to a single service use by a customer at the store ST as service usage status information. The service usage status information thus obtained in response to one service use by a customer is treated as the survey results regarding the customer's service usage at the store ST. The accumulated survey results in response to such service usage can be used as a reference for, for example, a financial institution in charge to grasp the degree of usage and congestion status of each service type and to establish or reorganize a store. At this time, the person in charge may use the information output terminal 400 to check the contents of the service usage status information obtained from the survey device 100.

[0032] An example of the functional configuration of the investigation device 100 will be described with reference to Fig. 3. The investigation device 100 includes, as hardware, a central processing unit (CPU), read only memory (ROM), random access memory (RAM), and storage devices such as hard disk drives (HDDs) and solid state drives (SSDs). The functions of the investigation device shown in the figure are realized by a CPU included in the hardware of the investigation device 100 executing a program.

[0033] The investigation device 100 in FIG. 3 includes a communication unit 101, a control unit 102, and a storage unit 103. The communication unit 101 communicates with the video storage device 300 of the store ST.

[0034] The control unit 102 executes various controls in the investigation device 100. The control unit 102 includes an image acquisition unit 121, a tracking unit 122, an estimation unit 123, and an information generation unit 124. The video acquisition unit 121 acquires video stored in the video storage device 300 in the store ST. The tracking unit 122 individually recognizes and tracks users present in the store ST based on the video acquired by the video acquisition unit 121. The tracking unit 122 is adapted to recognize a change in the position of the user in the store ST in response to the movement of the user. The estimation unit 123 estimates the service usage status of the user based on the relationship between the position of the user tracked by the tracking unit 122 and the presence identification areas (AR1, AR2, etc.) defined in the store ST. The information generating unit 124 generates service usage status information indicating the service usage status of each user estimated by the estimation unit 123. The information generating unit 124 causes the service usage status information storage unit 133 to store the generated service usage status information.

[0035] The storage unit 103 stores various types of information related to the survey device 100. The storage unit 103 includes a store map storage unit 131, an imaging area information storage unit 132, and a service usage status information storage unit 133.

[0036] The store map storage unit 131 stores data of a map (store map) showing the interior of each store ST. The store map shows the space inside the store ST using coordinates. The space inside the store ST shown by the store map may be, for example, two-dimensional as seen from a planar direction inside the store ST, or may be three-dimensional. The space shown by the store map shows the layout of the counters, facilities, etc. in the store ST. Furthermore, the store map may show coordinate ranges of presence identification areas (AR1, AR2) set corresponding to waiting benches BC and counters LS, as shown in Fig. 2. In this case, each of the set presence identification areas may be associated with service type identification information indicating the corresponding service type.

[0037] The imaging area information storage unit 132 stores imaging area information. The imaging area information is provided for each store ST, for example, and is information indicating coordinates of spatial portions within the store ST included in images captured by the imaging devices 200 arranged in the store ST.

[0038] 4 shows an example of imaging area information corresponding to one store ST. The imaging area information corresponding to one store ST is associated with a store ID that uniquely identifies the corresponding store ST. In addition, the imaging area information corresponding to one store ST is structured such that area coordinates are stored in association with each imaging device ID that uniquely identifies the imaging device 200 arranged in the corresponding store ST. The area coordinates are associated with pixels of the image of the imaging device 200 identified by the corresponding imaging device ID and the coordinates of a position in the store ST included in the image.

[0039] Returning to Fig. 3 for the explanation, the service usage status information storage unit 133 stores the service usage status information generated by the information generating unit 124. One piece of service usage status information is information indicating the service usage status according to one service usage by a customer in the store ST.

[0040] 5 shows an example of service usage status information corresponding to one service usage by a customer. The service usage status information in this figure includes fields for the store ID, customer attributes, and service usage status. The store ID field stores the store ID indicating the corresponding store ST.

[0041] The customer attribute field stores information indicating attributes (customer attributes) of the corresponding customer estimated by the estimation unit 123. Specifically, in the same figure, gender, age, and height are stored as customer attributes. The gender, age, and height as customer attributes are information obtained by estimating by the estimation unit 123 based on the video. For this reason, the age and height may have a certain range, such as 20s, 160 cm to 165 cm.

[0042] The service usage status information field stores information indicating the customer's service usage status according to the corresponding service usage. Specifically, the figure shows an example in which the service usage status field has fields for storing information on entry time, exit time, remarks, purpose (service type), counter used, service time (counter usage start time, counter usage end time), waiting time, waiting location, and reason for waiting time.

[0043] The entry time field stores the time (entry time) when the customer enters the store ST in response to the use of the corresponding service. The entry time may be expressed by date and time. The field of the exit time stores the time (exit time) when the customer who has finished using the corresponding service exits the store ST. The exit time may be expressed by date and time.

[0044] The remarks field stores remarks, which are information that should be added regarding the use of the corresponding service.

[0045] The service type field stores the details of the service type used by the customer under the corresponding service usage. The field of the service window stores information indicating the service window (service window) used by the customer when using the corresponding service. The counter use start time field stores the time (counter use start time) when the customer started using the counter (the counter clerk dealt with the customer). The counter use start time may be expressed as a date and time. The field of the end time of counter use stores the time (end time of counter use) when the customer finished using the counter (the counter clerk's response to the customer). The end time of counter use may be expressed by date and time.

[0046] The waiting time field stores the time (waiting time) that a customer waits until using a corresponding service at the store ST. The waiting time may be, for example, the time (waiting time) that a customer CS stays in the presence identification area AR1-1 of the waiting bench BC-1. The waiting place field stores information about the place (waiting place) where the customer waited during the waiting time. The field of the waiting time occurrence reason stores the reason why the response waiting time occurred (reason for the waiting time occurrence).

[0047] 5 includes customer attributes based on biological characteristics such as gender, age, height, etc. as information about the customer himself / herself, but does not include personal information. In this case, the estimation unit 123 recognizes a person as a customer from the video, but does not need to identify the recognized person. In this manner, in this embodiment, by obtaining service usage status information according to a customer's use of a service without identifying the individual customer, it is possible to protect the personal information of the customer. On the other hand, when it is permitted to identify an individual as a customer in investigating the service usage status, the estimation unit 123 may perform personal recognition on the customer recognized from the video. In this case, the estimation unit 123 may identify the individual by comparing biometric information such as a facial photograph, fingerprint, voiceprint, iris, etc. of the customer stored in a customer database owned by the financial institution with biometric information obtained by the video or an authentication device installed in the store ST. In addition, the information generation unit 124 may obtain personal information corresponding to the identified individual from the personal information stored in the customer database and include it in the corresponding service usage status information.

[0048] An example of a processing procedure executed by the survey device 100 in relation to the generation of service usage status information will be described with reference to the flowchart in Fig. 6. In the same figure, an example of processing for generating one piece of service usage status information in response to one service usage by a customer will be given.

[0049] Step S100: In the survey device 100, the video acquisition unit 121 acquires video of the store ST for which service usage status information is to be generated. At this time, the video acquisition unit 121 requests video from the video storage device 300 of the target store ST. When requesting video, the video acquisition unit 121 may specify, for example, the timestamp period of the video to be acquired. The video acquisition unit 121 acquires the video transmitted from the video storage device 300 in response to the request. The acquired video includes multiple videos acquired by each of the imaging devices 200 arranged in the corresponding store ST.

[0050] Step S102: The estimation unit 123 analyzes the video captured in step S100 and identifies one target customer.

[0051] Step S104: The estimation unit 123 estimates customer attributes (gender, age, height, etc.) for the customer identified in step S102. The estimation unit 123 may estimate the customer attributes using, for example, a trained model trained to output biometric characteristics such as a person's gender, age, height, etc. from an input video.

[0052] Step S106: The tracking unit 122 executes tracking of the customer identified in step S102 using the video acquired in step S100. Specifically, the tracking unit 122 may detect the position (customer position) of the customer identified in step S102 in the store ST from the time the customer enters through the entrance ENT until the customer leaves through the exit EXT after completing the service, and obtain tracking information associated with a time (time stamp) for each detected position.

[0053] Step S108: The estimation unit 123 estimates the entry time of the target customer. For example, the estimation unit 123 compares the tracking information obtained in step S106 with the store map of the corresponding store ST stored in the store map storage unit 131, and detects the situation in which the customer moves from the entrance ENT to enter the store ST. The estimation unit 123 may take the time indicated by the time stamp corresponding to the detected situation as the entry time estimation result.

[0054] Step S110: Furthermore, the estimation unit 123 estimates the waiting time that the target customer waited before using the service and the waiting location where the customer waited corresponding to the waiting time. In this case, the estimation unit 123 recognizes a state in which the customer position indicated in the tracking information continues to be present for a certain period of time or more within a presence identification area corresponding to a waiting place indicated on a store map of the corresponding store ST (for example, presence identification area AR1 corresponding to a waiting bench BC), as a waiting state of the customer. The estimation unit 123 may calculate the time that such a waiting state continues, and use the calculated time length as an estimation result of the waiting time. The estimation unit 123 may obtain the waiting time as, for example, the time from when the customer enters the store ST to when the customer ends the waiting state and moves from the waiting place. The estimation unit 123 may also include, in the waiting time, the time during which the customer waits after being served by a staff member at the counter but then returns to the waiting place and waits because the staff member interrupts the service at the counter for some reason. Furthermore, the estimation unit 123 may estimate a presence identification area corresponding to a waiting location in which the customer's position was included according to the waiting time estimated as described above as the waiting location where the customer was waiting. Furthermore, the estimation unit 123 may estimate the reason for the occurrence of the waiting time. When estimating the waiting time, the estimation unit 123 may analyze the video acquired by the video acquisition unit 121 using a technology classified as Explainable AI. For example, when the estimation unit 123 recognizes that there are many customers in the store ST as a result of analyzing the video acquired by the video acquisition unit 121, the estimation unit 123 may determine that the reason for the occurrence of the waiting time is that there are many visitors. Furthermore, when the estimation unit 123 recognizes that there is a closed counter LS among the counters LS corresponding to the service type used by the customer as a result of analyzing the video acquired by the video acquisition unit 121, the estimation unit 123 may determine that the reason for the occurrence of the waiting time is that there is a closed counter.

[0055] Step S112: The estimation unit 123 also estimates the type of service used this time by the target customer and the service window corresponding to the use of the service type. In this case, the estimation unit 123 uses the corresponding store map and the tracking information to identify a presence identification area in which the customer position of the target customer has been continuously present for a certain period of time or more among the presence identification areas corresponding to the counter LS in the corresponding store map. The estimation unit 123 identifies a service type associated with the identified presence identification area. The service type identified in this way is obtained as an estimation result. The estimation unit 123 may also identify a counter associated with the identified presence identification area on the corresponding store map. The estimation unit 123 may use the identified counter as an estimation result of the currently used counter.

[0056] Step S114: The estimation unit 123 also estimates the time during which the target customer used the service counter estimated in step S112. In this case, the estimation unit 123 may identify the start time and end time of a period during which the target customer's customer position was continuously present in the presence identification area corresponding to the service counter estimated in step S112, and acquire the identified start time and end time as the service counter usage start time and service counter usage end time, respectively, which are the time during which the service counter was used.

[0057] Step S116: The estimation unit 123 also estimates the exit time when the target customer finishes using the service and exits the store ST. In this case, the estimation unit 123 uses the corresponding store map and tracking information to identify the time corresponding to the timing when the customer exits the store ST from the exit EXT after using the counter LS. The estimation unit 123 outputs the time identified in this way as an estimated result of the exit time.

[0058] Step S118: The information generation unit 124 generates service status usage information (FIG. 5) that stores the store ID of the target store ST, the customer attributes estimated in step S104, the entry time estimated in step S108, the waiting time and waiting location estimated in step S110, the service type and counter estimated in step S112, the counter usage time estimated in step S114, and the exit time estimated in step S116.

[0059] Step S120: The information generating unit 124 stores the service status use information generated in step S118 in the service use status information storage unit 133.

[0060] In the example described above with reference to Fig. 5, the target customer enters the store ST, completes one business (service type), and then leaves the store ST. However, as an example of a customer actually completing business at the store ST, there are other examples, such as a case where a customer enters the store ST, completes multiple business, and then leaves the store ST (multiple business case). There are also examples where a customer enters the store ST, completes one or more business, exits the store ST, and then re-enters the store ST within a certain time period or within a time limit within the same business day to complete one or more business (re-entry case). The investigation system of this embodiment may be configured to also handle multiple business cases and re-entry cases as described above.

[0061] When dealing with a multiple business case, the investigation device 100 executes the process up to step S114 until the target customer tracked in step S106 finishes his / her business and leaves one counter LS. After this, if the target customer does not leave the store ST but remains in the store ST and is in a presence identification area AR2 corresponding to a different counter LS from the one he / she used previously, the investigation device 100 determines that the target customer is in a state where he / she will use the counter LS for the next business. In this case, if the target customer is in the presence identification area AR1 corresponding to the waiting bench BC between the time when the target customer finishes the previous business and the time when the target customer uses the counter LS for the next business, the waiting time and the waiting location may be estimated by the process of step S110. When the survey device 100 determines that the target customer is ready to use the counter LS in response to the next matter, it may execute the processing of steps S112 and S114 in response to the next matter, thereby estimating the service type, counter to be used, counter usage time, etc., corresponding to the next matter. In this way, the survey device 100 can track the target customer in response to multiple business cases and obtain service usage status information for each business. Then, when the target customer finishes the last business and leaves the store ST through the exit EXT, the exit time may be obtained by processing in step S116.

[0062] FIG. 7 shows an example of service usage status information corresponding to a multiple-business case in which a target customer has completed two business matters. In the figure, an example is shown in which service status usage information corresponding to the first business in the first row and service status usage information corresponding to the second (last) business in the second row are stored for a common store ID and customer attributes. In the case of a multiple business case, multiple business are completed in response to one entry into the store ST. For this reason, in the service status usage information, the entry time is stored in the service usage information corresponding to the first business, and is not stored in the service usage information corresponding to the second or subsequent business. In addition, the exit time is stored in the service usage information corresponding to the last business, and is not stored in the service usage information corresponding to the business before the last. In addition, in the service status usage information corresponding to the second business in the figure, information related to waiting (waiting time, waiting location, and reason for waiting) corresponds to the situation in which the customer waits after finishing the first business until the second business is accepted at the service window.

[0063] Furthermore, when dealing with a re-entry case, the investigation device 100, in response to identifying a customer who entered the store ST from the entrance ENT in step S102, determines whether the identified customer is a customer who has completed business at the store ST within a past predetermined time period that corresponds to the time limit (a re-entering customer). For this purpose, the investigation device 100 may, for example, refer to video captured by the imaging device 200 within the past predetermined time period and determine whether or not there is a customer who has the same features as the customer identified in the current step S102 among the customers who are determined to have completed business at the store ST. If it is determined that the customer is a re-entering customer, the survey device 100 may track the re-entering customer in step S106, and may execute the processes in step S112 and after in accordance with the behavior of the re-entering customer. Then, when the target customer completes his / her last business and leaves the store ST through the exit EXT, the exit time may be acquired in step S116.

[0064] FIG. 8 shows an example of service usage status information corresponding to a re-entry case, in which a customer completes one business, exits the store ST, re-enters the store ST within the time limit, and completes a second business. In the same figure, an example is shown in which, for a common store ID and customer attributes, service status usage information corresponding to the first matter on the first line and service status usage information corresponding to the second (last) matter on the second line are stored. In this re-entry example, since the customer left the store ST once after completing the first business, the remarks in the service status usage information corresponding to the first business on the first line indicate "temporary exit." In other words, it indicates that the target customer left the store ST once after completing the first business. In addition, the remarks of the service status usage information corresponding to the second business on the second line indicate that it is a "return." In other words, it indicates that the second business was completed by the target customer who had completed the first business and left the store ST, but returned to the store ST again to complete it. In this case, the entry time and the exit time are stored in the service usage status information corresponding to the first and second business, respectively. That is, the service usage status information in the first row stores the entry time when the target customer enters the store ST to complete the first business, and the exit time when the target customer exits the store ST after completing the first business. In addition, the service usage status information in the second row stores the entry time when the target customer enters the store ST to complete the second business, and the exit time when the target customer exits the store ST after completing the second business.

[0065] As an example of a store ST, a customer may enter the store ST to complete some business, but may then leave without completing the business at the store ST, for example because the waiting time is too long or because they have to complete another business outside the store ST. The investigation system of this embodiment may be configured to also handle cases such as the above (cases of leaving without completing business). 9 shows an example of the service status usage information corresponding to the case of leaving without completing the business. The remarks field in the service status usage information in the figure indicates that the customer entered the store ST but left without completing the business. In this case, valid information does not need to be stored for each of the items of service type, service window, and response time. In addition, in the case of leaving without completing business, the user may be waiting before leaving the store ST. Therefore, when the user waits, information may be stored in each of the items of the waiting time, the waiting place, and the reason for the waiting time in the service status usage information. In addition, in the case of a re-entry, there may be a case where a customer leaves the store ST after completing his / her business, but re-enters the store ST and leaves without completing his / her business. In response to such a case, the service status usage information as the re-entry case may include service status usage information of contents corresponding to the case of leaving without completing the business.

[0066] A modification of this embodiment will now be described. The estimation unit 123 may obtain, for example, the purpose (service type) selected by the customer through operation of the numbered ticket issuing machine 500 as an estimation result of the service type in the service status use information. In addition, the estimation unit 123 may determine the waiting time as, for example, a time from the time when the ticket is issued by the ticket issuing machine 500 to a time when the corresponding ticket number is called up and displayed on a call display device (not shown) installed at the counter LS.

[0067] In the above embodiment, the investigation device 100 is configured to acquire video temporarily stored in the video storage device 300 of the store ST and estimate items corresponding to the service status utilization information. In contrast, the investigation device 100 may be configured to acquire video captured by the imaging device 200 of the store ST in real time, and estimate items corresponding to the service status utilization information in real time. In this case, the video storage device 300 of the store ST may be omitted.

[0068] In this embodiment, the space in which the service is provided may be a government office, a post office, or a medical institution, as described above. In this way, depending on the space in which the service is provided, the counter LS in the above embodiment may be, for example, a room or a floor.

[0069] Furthermore, in this embodiment, the space in which the service is provided is not limited to indoor spaces such as the above-mentioned financial institution stores, but may be outdoors. An example of a space where outdoor services are provided is an event venue where many temporary stores are exhibited outdoors. In this case, visitors to the event venue are users, and the user's purpose may be to use the services provided by the temporary stores (for example, to purchase products, cooking items, etc. sold by the temporary stores). In this case, the temporary stores set up at the event venue correspond to the window LS in the above embodiment, and a predetermined area in front of each temporary store becomes the presence identification area AR2. An area corresponding to tables for eating and drinking set up at the event venue may also be the presence identification area AR2. In this case, the imaging device 200 may be provided at the entrance and exit of the event venue and at various places within the event venue. In this case, the result of the user's use of the services of the temporary stores at the event venue may be managed as the service usage status. In this case, the entry time and exit time in the service usage information correspond to the time of entry into the event venue and the time of exit from the event venue, respectively. The service type and service window in the service usage information may be replaced with the store type (e.g., accessory sales, food and drink sales, etc.) and store name of the corresponding temporary store, respectively. The service time in the service usage information may be replaced with the start time when the corresponding customer was served at the corresponding temporary store and the end time when the service ended. The waiting time in the service usage information may be the time spent waiting in a queue at the corresponding temporary store.

[0070] For example, the investigation device 100 may be configured such that predetermined functions are distributed among a plurality of devices to which the functions are assigned, and the plurality of devices cooperate with each other via communication to execute processing.

[0071] In addition, a program for realizing the functions of the above-mentioned investigation device 100, the video storage device 300, and the information output terminal 400 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform the processing of the above-mentioned investigation device 100, the video storage device 300, and the information output terminal 400. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. In addition, the "computer system" may include multiple computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. Furthermore, the "computer system" may be a classical computer such as a virtual machine virtually constructed by software, or a quantum computer. Moreover, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as HDDs and SSDs built into computer systems. Thus, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM. The recording medium also includes internal or external recording media accessible from a distribution server to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format executable by a terminal device. In other words, the format in which the program is stored in the distribution server does not matter as long as the program can be downloaded from the distribution server and installed in a format executable by a terminal device. Note that the program may be divided into multiple parts, downloaded at different times, and then combined in the terminal device, or the divided programs may be distributed by different distribution servers. Furthermore, the term "computer-readable recording medium" refers to a storage medium that holds the program for a certain period of time, such as a volatile memory (RAM) in a computer system that becomes a server or client when the program is transmitted via a network. The program may also be for implementing part of the above-mentioned functions. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

[0072] <Additional Notes> (1) One aspect of this embodiment is a survey device that includes an image acquisition unit that acquires images obtained by each of one or more imaging devices configured to image a specified service provision space, a tracking unit that tracks a position of a user in the service provision space based on the images acquired by the image acquisition unit, an estimation unit that estimates a service usage status of a user based on a relationship between the position of the user tracked by the tracking unit and a specified positional range defined in the service provision space, and an information generation unit that generates, for each user, service usage status information indicating the service usage status estimated by the estimation unit.

[0073] (2) One aspect of this embodiment is the survey device described in (1), wherein the service usage status includes a service type used by the user, and the estimation unit may estimate, as the service type used by the user, a service type corresponding to a location range in which the user tracked by the tracking unit has been present for a predetermined period of time or more, among location ranges corresponding to service provision locations defined in the service provision space corresponding to each service type.

[0074] (3) One aspect of this embodiment is the survey device described in (1) or (2), wherein the service usage status includes a service usage time during which the user used a service in the service provision space, and the estimation unit may estimate the service usage time based on a time during which the location of the user tracked by the tracking unit was within a location range of a service provision location defined in the service provision space.

[0075] (4) One aspect of this embodiment is a survey device according to any one of (1) to (3), wherein the service usage status includes a waiting time required for the user to be able to use the service, and the estimation unit may estimate the waiting time based on a time during which the location of the user tracked by the tracking unit was within a location range defined as a waiting area in the service provision space.

[0076] (5) One aspect of this embodiment is the survey device described in any one of (1) to (4), wherein the estimation unit estimates a predetermined biometric characteristic of a user based on the image acquired by the image acquisition unit, and the information generation unit includes the estimated biometric characteristic in service usage status information as an attribute of the corresponding user.

[0077] (6) One aspect of this embodiment is a survey method in a survey device, the survey method including: an image acquisition step in which an image acquisition unit acquires images obtained by each of one or more imaging devices configured to image a specified service provision space; a tracking step in which a tracking unit tracks a position of a user in the service provision space based on the images acquired by the image acquisition step; an estimation step in which an estimation unit estimates the user's service usage status based on the relationship between the position of the user tracked by the tracking step and a specified positional range defined in the service provision space; and an information generation step in which an information generation unit generates, for each user, service usage status information indicating the service usage status estimated by the estimation step.

[0078] (7) One aspect of this embodiment is a program for causing a computer, as an investigation device, to function as an image acquisition unit that acquires images obtained by each of one or more imaging devices configured to image a specified service provision space, a tracking unit that tracks the position of a user in the service provision space based on the images acquired by the image acquisition unit, an estimation unit that estimates the user's service usage status based on the relationship between the position of the user tracked by the tracking unit and a specified positional range defined in the service provision space, and an information generation unit that generates, for each user, service usage status information indicating the service usage status estimated by the estimation unit. [Explanation of symbols]

[0079] 100 Survey device, 101 Communication unit, 102 Control unit, 103 Memory unit, 121 Video acquisition unit, 122 Tracking unit, 123 Estimation unit, 124 Information generation unit, 131 Store map memory unit, 132 Image capture area information memory unit, 133 Service usage status information memory unit, 200 Imaging device, 300 Video memory device, 400 Information output terminal, 500 Numbered ticket issuing machine

Claims

1. an image acquisition unit that acquires images obtained by one or more imaging devices that are provided to capture an image of a predetermined service providing space; a tracking unit that tracks a position of a user in a service providing space based on the image acquired by the image acquisition unit; an estimation unit that estimates a service usage status of a user based on a relationship between a position of the user tracked by the tracking unit and a predetermined position range defined in a service provision space; an information generating unit that generates service usage status information indicating the service usage status estimated by the estimation unit for each user; An investigation device comprising:

2. The service usage status includes a type of service used by the user, The estimation unit estimates, as a service type to be used by the user, a service type corresponding to a location range in which the user tracked by the tracking unit has been present for a predetermined period of time or more, among location ranges corresponding to service providing places determined in the service providing space corresponding to each service type. The investigation device according to claim 1 .

3. The service usage status includes a service usage time during which the user used the service in the service providing space, The estimation unit estimates the service use time based on a time during which the position of the user tracked by the tracking unit was present within a position range of a service provision location defined in the service provision space. The investigation device according to claim 1 or 2.

4. The service usage status includes a waiting time required for the user to be able to use the service, The estimation unit estimates the waiting time based on a time during which the position of the user tracked by the tracking unit was within a position range defined as a waiting place in the service providing space. The investigation device according to claim 1 or 2.

5. The estimation unit estimates a predetermined biometric feature of the user based on the image acquired by the image acquisition unit, The information generating unit includes the estimated biometric feature as an attribute of the corresponding user in the service usage status information. The investigation device according to claim 1 or 2.

6. A survey method for a survey device, comprising: an image acquisition step in which an image acquisition unit acquires images obtained by one or more imaging devices provided to capture an image of a predetermined service provision space; a tracking step in which a tracking unit tracks a position of the user in the service providing space based on the image obtained by the image obtaining step; an estimation step of estimating a service usage status of the user based on a relationship between a position of the user tracked in the tracking step and a predetermined position range defined in a service providing space by an estimation unit; an information generating step of generating service usage status information indicating the service usage status estimated by the estimation step for each user by an information generating unit; Research methods including:

7. The computer as a research device an image acquisition unit that acquires images obtained by one or more imaging devices that are provided to capture an image of a predetermined service providing space; a tracking unit that tracks a position of a user in a service providing space based on the image acquired by the image acquisition unit; an estimation unit that estimates a service usage status of a user based on a relationship between a position of the user tracked by the tracking unit and a predetermined position range defined in a service providing space; an information generating unit that generates service usage status information indicating the service usage status estimated by the estimation unit for each user; A program to function as a

Images