Information processing device, control method for information processing device, and storage medium

Abstract

Provided is an information processing device that contributes to improving sales at a store or the like. The information processing device comprises a detection means, a calculation means, a storage means, a selection means, and a notification means. The detection means detects a group comprising a plurality of members who have entered a prescribed facility. The calculation means calculates the current positions of each of the plurality of members by using image data captured by a camera device installed in the prescribed facility. The calculation means calculates a center point of the group on the basis of the calculated current positions of each of the plurality of members. The storage means stores time-series data of the calculated current positions of each of the plurality of members as a line of flow of each of the plurality of members and stores time-series data of the calculated center point of the group as the line of flow of the group. The selection means selects, as a payer, a member assumed to have payment authority from among the plurality of members on the basis of the stored line of flow of each of the plurality of members and the stored line of flow of the group. The notification means notifies a person associated with the prescribed facility of information about the selected payer.

Description

Information processing apparatus, control method for information processing apparatus, and storage medium

[0001] The present invention relates to an information processing apparatus, a control method for an information processing apparatus, and a storage medium.

[0002] There are technologies related to pedestrian flow analysis in stores and the like.

[0003] For example, Patent Document 1 describes that it provides a pedestrian flow analysis method, a pedestrian flow analysis apparatus, and a pedestrian flow analysis system that can determine the order in which people visited stores with a simple configuration and high accuracy. The pedestrian flow analysis apparatus of Patent Document 1 includes an image information acquisition unit, a person identification unit, a store estimation unit, a database, and a visit order determination unit. The image information acquisition unit acquires an appearance image of a person. The person identification unit identifies a person from the appearance image. The store estimation unit identifies a possession owned by the person from the appearance image, and estimates the store where the possession was acquired based on the identified possession. The database stores and associates person information indicating the identified person, store information indicating the estimated store, and time information indicating the time when the appearance image was acquired. The visit order determination unit determines the order in which the person visited the stores based on time-series changes of the stores indicated by the store information stored in the database.

[0004] Japanese Unexamined Patent Publication No. 2021-039784

[0005] There are cases where a group of multiple people visits a store such as a consumer electronics mass retailer. Operators of such stores have a desire to improve store sales through efficient customer service to the person who makes decisions when purchasing products within a group (for example, a family). However, existing systems cannot satisfy this desire.

[0006] It should be noted that Patent Document 1 merely discloses a technology related to pedestrian flow analysis in stores and the like. Therefore, even if the technology disclosed in Patent Document 1 is applied, it is difficult to satisfy the above desire.

[0007] A main object of the present invention is to provide an information processing apparatus, a control method for an information processing apparatus, and a storage medium that contribute to improving sales of stores and the like.

[0008] According to a first aspect of the present invention, an information processing device is provided, comprising: detection means for detecting a group of multiple members who have entered a predetermined facility; calculation means for calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; storage means for storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and for storing the time-series data of the calculated centroid of the group as the movement line of the group; selection means for selecting a member who is assumed to have the authority to make a decision from among the multiple members as a decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and notification means for notifying a person related to the predetermined facility of the information of the selected decision-maker.

[0009] A control method for an information processing device is provided, comprising: a detection step of detecting a group consisting of multiple members that have entered a predetermined facility; a calculation step of calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; a storage step of storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and storing the time-series data of the calculated centroid of the group as the movement line of the group; a selection step of selecting a member who is assumed to have the authority to make a decision from among the multiple members as a decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and a notification step of notifying a person related to the predetermined facility of the information of the selected decision-maker.

[0010] According to a third aspect of the present invention, a computer-readable storage medium is provided that stores a program for a computer mounted on an information processing device to execute: a detection process for detecting a group of multiple members that have entered a predetermined facility; a calculation process for calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; a storage process for storing time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and storing time-series data of the calculated centroid of the group as the movement line of the group; a selection process for selecting a member who is assumed to have the authority to make a decision from among the multiple members as the decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and a notification process for notifying persons related to the predetermined facility of the information of the selected decision-maker.

[0011] According to each aspect of the present invention, an information processing device, a control method for the information processing device, and a storage medium are provided that contribute to increasing sales at stores and the like. However, the effects of the present invention are not limited to those described above. The present invention may produce other effects in lieu of or in conjunction with the effects described above.

[0012] Figure 1 is a diagram illustrating the outline of one embodiment. Figure 2 is a flowchart illustrating the overview of operation of one embodiment. Figure 3 is a diagram illustrating an example of the schematic configuration of an information processing system according to an embodiment of this disclosure. Figure 4 is a diagram illustrating an example of the internal configuration of a store according to an embodiment of this disclosure. Figure 5 is a diagram illustrating an example of the display of a terminal according to an embodiment of this disclosure. Figure 6 is a diagram illustrating an example of the processing configuration of a server device according to an embodiment of this disclosure. Figure 7 is a flowchart illustrating an example of the operation of a group control unit according to an embodiment of this disclosure. Figure 8 is a diagram illustrating the operation of a group control unit according to an embodiment of this disclosure. Figure 9 is a diagram illustrating an example of a group management database according to an embodiment of this disclosure. Figure 10 is a diagram illustrating the operation of a group control unit according to an embodiment of this disclosure. Figure 11 is a diagram illustrating the operation of a group control unit according to an embodiment of this disclosure. Figure 12 is a diagram illustrating the operation of a group control unit according to an embodiment of this disclosure. Figure 13 is a diagram illustrating the operation of a group control unit according to an embodiment of this disclosure. Figure 14 is a diagram illustrating an example of the internal configuration of a shopping mall according to an embodiment of this disclosure. Figure 15 is a diagram illustrating an example of the hardware configuration of a server device according to this disclosure.

[0013] First, an overview of one embodiment will be described. The reference numerals in the drawings attached to this overview are provided for convenience as examples to aid understanding, and this overview is not intended to be limiting in any way. Furthermore, unless otherwise specified, the blocks shown in each drawing represent functional units, not hardware units. The connecting lines between blocks in each drawing include both bidirectional and unidirectional lines. Unidirectional arrows schematically indicate the flow of the main signal (data) and do not exclude bidirectional flow. In this specification and in the drawings, elements that can be similarly described are given the same reference numerals to avoid redundant explanation.

[0014] An information processing device 100 according to one embodiment includes a detection means 101, a calculation means 102, a storage means 103, a selection means 104, and a notification means 105 (see Figure 1). The detection means 101 detects a group of multiple members that have entered a predetermined facility (step S1 in Figure 2). The calculation means 102 calculates the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility (step S2). The calculation means 102 calculates the centroid of the group based on the calculated current positions of each of the multiple members (step S3). The storage means 103 stores the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and stores the time-series data of the calculated centroid of the group as the movement line of the group (step S4). The selection means 104 selects a member who is assumed to have the authority to make a decision from among the multiple members as the decision-maker, based on the stored movement lines of each of the multiple members and the stored movement line of the group (step S5). The notification means 105 notifies the person associated with the designated facility of the information of the selected decision-maker (step S6).

[0015] The information processing device 100 detects a group of multiple individuals entering a designated facility and calculates the movement path of each member of the group and the movement path of the group (the trajectory of the group's center of gravity). Using the movement paths of each member and the group, the information processing device 100 selects a member within the group who is presumed to have the authority to make a payment as the decision-maker. The information processing device 100 notifies a person related to the designated facility (for example, a store employee) of the information of the selected decision-maker. Since the store employee can then know who the decision-maker is, they can provide direct and efficient customer service to that decision-maker. As a result, sales at the store increase.

[0016] Specific embodiments will be described in more detail below with reference to the drawings.

[0017] [First Embodiment] The first embodiment will be described in more detail with reference to the drawings.

[0018] [System Configuration] Figure 3 is a diagram showing an example of the schematic configuration of an information processing system according to the embodiment disclosed herein. As shown in Figure 3, the information processing system includes a server device 10.

[0019] Server device 10 is a server that controls customer information for stores such as supermarkets, convenience stores, consumer electronics retailers, and home improvement stores. Server device 10 may be installed inside the building of the supermarket or other store, or it may be installed on a network (on the cloud).

[0020] As shown in Figure 3, store employees carry a terminal 20. The employees use the terminal 20 to obtain various information from the server device 10 and input various information into the server device 10.

[0021] At least one camera device 30 is installed inside the store (see Figure 4). Figure 4 shows an example of the interior of a consumer electronics store. As shown in Figure 4, multiple camera devices 30 are installed in various locations within the store.

[0022] The camera device 30 is, for example, a device capable of measuring the distance to a subject, such as a depth camera or a stereo camera.

[0023] For example, the camera device 30 periodically or at predetermined intervals captures a predetermined range and acquires a camera image (an image consisting of RGB (Red Green Blue) pixels) and a depth image (an image including depth information from the camera to the subject).

[0024] The camera device 30 transmits the acquired image data (RGB image, depth image) and camera ID to the server device 10.

[0025] The camera ID is an ID used to identify each camera device 30 installed within the store. The camera ID can be the MAC (Media Access Control) address or IP (Internet Protocol) address of the camera device 30.

[0026] The server device 10 stores the received image data (RGB image, depth image), the date and time of reception, and the camera ID in association with each other.

[0027] The devices shown in Figures 3 and 4 are interconnected. Specifically, the server device 10, the terminal 20, and the camera device 30 are connected by wired or wireless communication means and are configured to communicate with each other.

[0028] The configuration of the information processing system shown in Figure 3 is illustrative and not intended to limit its configuration. For example, the system may include multiple server devices 10. Load balancing and redundancy may be achieved by using multiple server devices 10.

[0029] [Outline Operation] Next, the outline operation of the information processing system according to the first embodiment will be described.

[0030] <Group Detection> The server device 10 detects groups of customers who have come to the store in groups of multiple people. For example, the server device 10 detects multiple customers who have come to the store as a family. The server device 10 detects groups consisting of multiple customers based on image data obtained from the camera device 30.

[0031] <Selection of decision-makers> When a group is detected, the server device 10 calculates the movement of each member constituting the group. Specifically, the server device 10 tracks the movement of each member using image data obtained from the camera device 30.

[0032] More specifically, the server device 10 detects the current position of each member. Furthermore, the server device 10 calculates the group's centroid (the coordinates of the group's centroid) from the current position of each member. The server device 10 calculates the group's centroid as the center point of each member's current position.

[0033] The server device 10 calculates the movement paths of each member and the group by repeatedly calculating the current position of each member and the group's center of gravity. The group's movement path is the trajectory of the group's center of gravity.

[0034] In this way, the server device 10 tracks the movement of the group by calculating the movement of each member of the group and the group's movement path.

[0035] When a predetermined period of time has elapsed since a group of customers visited the store, the server device 10 selects a member from among the members of that group who is presumed to have the authority to pay for the goods (the member who makes the decision regarding whether or not to allow the purchase of the goods). In the following explanation, the member who is presumed to have the authority to pay for the goods will be referred to as the "decision-maker."

[0036] Specifically, the server device 10 sets the member who is moving along the route closest to the group's movement route as the decision-maker.

[0037] Here, it is assumed that members who are not the payers will scatter throughout the store, picking up various items that the group has decided to purchase in advance, and then returning to the group. On the other hand, it is assumed that the payers will often be accompanying the group. Based on these assumptions, the server device 10 calculates the group's movement patterns and identifies members who are moving along the group's movement patterns as the payers.

[0038] <Notification to store staff> Once a payer is selected from among the group members, the server device 10 notifies the store staff of the payer's information. For example, the server device 10 sends the payer's facial image to the terminal 20 held by the store staff.

[0039] Upon receiving information about the payer, the terminal 20 displays the received information. For example, the terminal 20 displays information as shown in Figure 5. This display allows the store clerk to identify the payer in the group and to provide customer service based on the premise that the payer has significant authority when purchasing goods.

[0040] Next, we will describe the details of each device included in the information processing system according to the first embodiment.

[0041] [Server Device] FIG. 6 is a diagram showing an example of the processing configuration (processing modules) of a server device 10 according to an embodiment disclosed in the present application. Referring to FIG. 6, the server device 10 includes a communication control unit 201, an image data control unit 202, a group control unit 203, and a storage unit 204.

[0042] The communication control unit 201 is a means for controlling communication between the server device and other devices. For example, the communication control unit 201 receives data (packets) from the terminal 20. The communication control unit 201 also transmits data to the terminal 20. The communication control unit 201 delivers data received from other devices to other processing modules. The communication control unit 201 transmits data acquired from other processing modules to other devices. In this way, other processing modules transmit and receive data to and from other devices via the communication control unit 201. The communication control unit 201 has a function as a receiving unit that receives data from other devices, and a function as a transmitting unit that transmits data to other devices.

[0043] The image data control unit 202 is a means for performing control related to image data received from a camera device 30.

[0044] The image data control unit 202 stores image data received from the camera device 30 for each camera device 30 (for each camera ID). The image data control unit 202 stores the reception time of the image data, the camera ID, and the image data (RGB image, depth image) in association with each other.

[0045] The group control unit 203 is a means for performing control related to a group of multiple people visiting a store.

[0046] FIG. 7 is a flowchart showing an example of the operation of the group control unit 203 according to an embodiment disclosed in the present application. The operation of the group control unit 203 will be described with reference to FIG. 7.

[0047] First, the group control unit 203 detects a group of multiple people visiting the store (step S101).

[0048] Specifically, the group control unit 203 acquires image data captured by the camera device 30 installed to be capable of capturing the entrance from among the stored image data periodically or at a predetermined timing. For example, the group control unit 203 refers to table information that stores camera IDs in association with installation locations of the camera devices 30, and acquires image data captured by the camera device 30 installed to be capable of capturing the entrance.

[0049] The group control unit 203 attempts to extract a plurality of face regions (face images) from the acquired image data. Since existing techniques can be used for the face image extraction processing by the group control unit 203, detailed description thereof will be omitted. For example, the group control unit 203 may extract a face image (face region) from the image data using a learning model trained by a CNN (Convolutional Neural Network). Alternatively, the group control unit 203 may extract a face image using a technique such as template matching.

[0050] When a plurality of face images cannot be extracted from the image data, the group control unit 203 does not perform any special processing and terminates the processing.

[0051] When a plurality of face images can be extracted from the image data, the group control unit 203 sets a group having a plurality of customers appearing in the image data as members. For example, as shown in FIG. 8, if three persons are captured in the image data obtained by capturing the entrance, the group control unit 203 sets a group consisting of the three persons.

[0052] The group control unit 203 assigns a group ID to the detected group. Further, the group control unit 203 assigns a member ID to each member of the group. The group control unit 203 stores the group ID, member IDs, face images of each member, and the like in a group management database (see FIG. 9).

[0053] As shown in Figure 9, the group management database stores the group registration date and time, group ID, and information about each member (member ID, face image). Note that the group management database shown in Figure 9 is an example and is not intended to limit the items to be stored. For example, feature quantities generated from each member's face image may also be registered in the group management database.

[0054] Once the group information is stored in the group management database, the group control unit 203 tracks the movement of each member (step S102).

[0055] Specifically, the group control unit 203 uses image data obtained from the camera device 30 to calculate the current position of each group member belonging to the group.

[0056] Specifically, the group control unit 203 reads face images from each entry of the same group stored in the group management database.

[0057] Furthermore, the group control unit 203 extracts at least one face image (face region) from each of the most recent and multiple image data (RGB images) that were captured at substantially the same time.

[0058] The group control unit 203 sets one face image read from the group management database as the matching side and multiple face images extracted from the image data as the registration side, and performs a one-to-many matching (N is a positive integer, the same applies hereafter). By performing the one-to-many matching, the group control unit 203 identifies a face image from among the multiple face images extracted from the image data that is substantially the same as the face image of a member stored in the group management database.

[0059] The group control unit 203 identifies image data including the face image (face region) of a group member through one-to-many matching, and then estimates the current location of that member using information from the camera device 30 that transmitted the identified image data.

[0060] More specifically, the group control unit 203 calculates the position information of the group members using the installation information of the camera device 30 that photographed the group members, internal parameters, the user's coordinates in the RGB image, and depth information in the depth image.

[0061] The group control unit 203 calculates the coordinates (X coordinate, Y coordinate) of a coordinate system with a point in the store as the origin (hereinafter referred to as the store coordinate system) as the position information of the members.

[0062] Furthermore, the installation information for the camera device 30 includes the coordinates of the location where the camera device 30 is installed (X and Y coordinates in the store coordinate system) and the installation direction of the camera device 30. Internal parameters include the focal length of the camera device 30, etc. The user's coordinates in the RGB image are the coordinates of the user's face area (center of the face area) in the screen coordinate system. The depth information in the depth image is the depth (distance) to the user's face area in the screen coordinate system.

[0063] The group control unit 203 acquires location information (coordinates in the store coordinate system) of the subject (group member) by applying a predetermined calculation formula to the installation information of the camera device 30, etc.

[0064] Since existing technologies can be applied to calculate the subject's location information from image data, a more detailed explanation will be omitted.

[0065] The group control unit 203 calculates the current location of each member by repeating the above process for each member stored in the group management database. The group control unit 203 stores (appends) the calculated current location along with the calculation time in the movement path field of the group management database.

[0066] After calculating the current position of each member, the group control unit 203 further calculates the group's centroid.

[0067] Specifically, the group control unit 203 calculates the centroid of the group by adding the coordinates (X coordinate, Y coordinate) of each member and dividing the sum by the number of people in the group. For example, consider a case where the group has three members and the coordinates of each member at time T are (X1, Y1), (X2, Y2), and (X3, Y3). In this case, the centroid of the group at time T (Xg, Yg) is calculated as {(X1 + X2 + X3) / 3} and {(Y1 + Y2 + Y3) / 3}.

[0068] The group control unit 203 stores the calculated centroid (center point) of the group in the movement path field of the group management database. In Figure 9, the value entered in the fourth row from the top of the movement path field is the centroid of the group.

[0069] As shown in Figure 9, the group management database records the current location of each member and the group's centroid at each point in time. In other words, the group management database stores time-series data on each member's current location and time-series data on the group's centroid. The time-series data of current location and centroid stored in the group management database constitutes the movement paths of each member and the group as a whole.

[0070] In this way, the group control unit 203 can obtain the movement paths of the group members and the group by calculating the current position of each member and the center of gravity of the group periodically or at predetermined intervals.

[0071] The group control unit 203 repeatedly calculates the movement of each member and the group for a predetermined period (step S103, No. branch in Figure 7). For example, the group control unit 203 tracks the group's movement for a predetermined period, such as 3 minutes or 5 minutes, after the group enters the store.

[0072] After calculating the movement paths of each member and group for a predetermined period (step S103, Yes branch), the group control unit 203 selects the decision-maker (step S104).

[0073] Specifically, the group control unit 203 sets the member who is moving along the route closest to the group's route as the decision-maker.

[0074] Specifically, the group control unit 203 calculates the distance between each member's position and the group's centroid at each time point. Furthermore, the group control unit 203 calculates the sum of the calculated distances between each member's position and the group's centroid. The group control unit 203 selects the member with the smallest sum of distances as the decision-maker.

[0075] For example, consider the case where three members move as shown in Figure 10. In Figure 10, the solid lines show the actual movement (path) of each member, and the dotted lines show the trajectory (path) of the group's center of gravity.

[0076] The group control unit 203 calculates the distance between each member's current position and the group's centroid at each of the times T1, T2, and T3. For each member, the group control unit 203 calculates the sum of the distances calculated at each of the times T1 to T3 and selects the member with the smallest sum as the decision-maker. In other words, the group control unit 203 selects the member with the smallest sum of distances between the movement paths of each of the stored members and the movement path of the stored group as the decision-maker. In the example in Figure 10, the member in white is selected as the decision-maker.

[0077] Once a payer is selected, the group control unit 203 notifies the store clerk of the payer's information (notification of payer's information; step S105 in Figure 7). For example, the group control unit 203 transmits the payer's facial image to the terminal 20 held by the store clerk.

[0078] Alternatively, the group control unit 203 may process the image data showing each member of the group (for example, image data obtained by taking pictures with the camera device 30 installed at the entrance) to clearly indicate the approver, and then transmit the processed image data to the terminal 20. For example, the group control unit 203 may transmit the image data shown in Figure 11 to the terminal 20.

[0079] Thus, the group control unit 203 includes the functions of a detection means, a calculation means, a selection means, and a notification means. The detection means detects a group consisting of multiple members that has entered a predetermined facility (for example, a store such as a consumer electronics retailer). The calculation means uses image data captured by at least one camera device 30 installed in the predetermined facility to calculate the current position of each of the multiple members, and calculates the centroid of the group based on the calculated current positions of each of the multiple members. The time-series data of the calculated current positions of each of the multiple members is stored as the movement lines of each of the multiple members, and the time-series data of the calculated centroid of the group is stored as the movement line of the group. The selection means selects a member who is assumed to have the authority to make a decision from among the multiple members as the decision-maker, based on the stored movement lines of each of the multiple members and the stored movement line of the group. The notification means notifies a person related to the predetermined facility (for example, a store employee) of the information of the selected decision-maker.

[0080] The memory unit 204 is a means for storing information necessary for the operation of the server device 10. The memory unit 204 functions as a storage means that stores the time-series data of the current position of each of the multiple members calculated as the movement path of each of the multiple members, and stores the time-series data of the centroid of the group calculated as the movement path of the group.

[0081] <Terminal> Examples of terminals 20 include smartphones, mobile phones, game consoles, tablets, and other portable terminal devices. Terminal 20 can be any device or equipment as long as it can receive operations from store employees and communicate with the server device 10, etc. The configuration and operation of terminal 20 are obvious to those skilled in the art, so a detailed explanation is omitted.

[0082] Next, a modified example of the first embodiment will be described.

[0083] <Modification 1> If there are two customers who have come to the store, the server device 10 does not need to select a payer from the group of two people. This is because the distance between the current position of each of the two members and the center of gravity of the group is equidistant, and it is not possible to determine which member is moving according to the group's movement path.

[0084] Thus, the group control unit 203 (selection means) may select a decision-maker if the number of members forming the group is three or more. By not selecting a decision-maker from a group consisting of two customers, the load on the server device 10 is reduced.

[0085] <Modification 2> The server device 10 may calculate a variance value for the movement paths of each group member and detect the group based on the calculated variance value.

[0086] Specifically, the group control unit 203 calculates the movement paths of each member and the group as a whole for a predetermined period (for example, 1 minute) from the arrival of multiple customers (groups). After the predetermined period has elapsed, the group control unit 203 sets the movement path of each member (current position at each time) as the target data and the movement path of the group (centroid of the group at each time) as the average value and calculates the variance value.

[0087] The group control unit 203 determines that the group setting is correct if each member's variance value is smaller than a predetermined value. In this case, the group control unit 203 continues to track the group's movement and selects an approver.

[0088] If the variance value of each member is greater than a predetermined value, the group control unit 203 determines that the setting of the group consisting of each member is incorrect.

[0089] The fact that each member has a large variance suggests that they were not moving in the same direction or to the same place. In other words, it is highly likely that the multiple individuals initially identified as members of the group simply entered the establishment at the same time.

[0090] In this case, the group control unit 203 cancels the group setting. For example, the group control unit 203 may delete the entry in the group management database.

[0091] Thus, the group control unit 203 (detection means) may calculate a variance value for each of the movement paths of multiple members stored during a predetermined period after the group enters a predetermined facility. The group control unit 203 may determine that a group consisting of multiple members has been detected if each of the calculated variance values ​​is smaller than a predetermined value. As a result, the server device 10 can detect groups more accurately.

[0092] <Variation 3> The server device 10 may terminate tracking the movement of the group including the payer after notifying the store clerk of the payer's information. Alternatively, the server device 10 may continue tracking the movement of the group including the payer even after notifying the store clerk of the payer's information.

[0093] For example, the group control unit 203 may collect information such as the group's destination (the stores the group will visit).

[0094] Alternatively, the group control unit 203 may periodically or at predetermined intervals re-elect the payer after notifying the store clerk of the payer's information, and if it determines that the payer has changed, it may notify the store clerk of the information of the re-elected payer.

[0095] <Modification 4> The group control unit 203 may be configured to identify members by extracting a body image (body image) instead of a face image (face region) and comparing it with the body image stored in the group management database. Alternatively, the group control unit 203 may perform the comparison using a full-body image that includes the face image instead of a body image.

[0096] As described above, the server device 10 according to the first embodiment detects a group of multiple people entering the store and calculates the movement of each member of the group and the movement of the group as a whole. Using the movement of each member and the movement of the group as a whole, the server device 10 selects a member who is assumed to have payment authority within the group as the payer. The server device 10 notifies the store staff of the information of the selected payer (for example, a photograph of the payer). Since the store staff can know who the payer is, they can provide direct and efficient customer service to that payer. As a result, sales at the store increase.

[0097] [Second Embodiment] Next, a second embodiment will be described in detail with reference to the drawings.

[0098] In the second embodiment, the departure of a member from a group or the splitting of a group will be described. More specifically, the server device 10 may detect the departure of a member or the splitting of a group before or after the selection of a decision-maker.

[0099] The following will focus on explaining the differences between the first and second embodiments.

[0100] Furthermore, the server device 10 according to the second embodiment will continue to track the movement of the group even after selecting a payer from the group (after notifying the store clerk, etc., of the payer's information).

[0101] First, let me explain the departure of a member from the group.

[0102] When the group control unit 203 calculates the current position of the group members and the centroid of the group, it also calculates the variance value for each group member. If the calculated variance value remains above a predetermined threshold for a predetermined period of time or longer, the group control unit 203 determines that the member with the large variance value has left the group.

[0103] For example, if a family of five enters a store and walks around the store together, and then one member goes to a different location than the other four, that member will be determined to have left the group.

[0104] In this case, the group control unit 203 may delete the entry of the member who has left the group from the group management database. The group control unit 203 may also calculate the current position and the centroid of the group for the remaining members of the group.

[0105] If a decision-maker has already been selected, the group control unit 203 may re-select a new decision-maker based on the current positions of each member after their departure and the group's centroid. Furthermore, if the previously selected decision-maker and the newly selected decision-maker are different, the group control unit 203 may transmit information about the re-selected decision-maker to the terminal 20.

[0106] Next, I will explain the group's split.

[0107] When the group control unit 203 calculates the current position of the group members and the centroid of the group, it also calculates the distance between each group member.

[0108] For example, consider a case where a group of six members, A through F, splits. In this case, the group control unit 203 calculates the distance between member A and B, the distance between member A and C, the distance between member A and D, the distance between member A and E, the distance between member A and F, the distance between member B and C, and so on.

[0109] The group control unit 203 sets members whose calculated distance from each other is below a predetermined threshold as a provisional group. For example, if the distance between members A and C is small and the distance between members D and F is small, the group control unit 203 sets up a provisional group consisting of members A and C and a provisional group consisting of members D and F.

[0110] If the state of the temporary group that was set continues for a predetermined period (for example, 1 minute), the group control unit 203 determines that the original group has split. In the example above, the original group consisting of members A to F is determined to have split into a group consisting of members A to C and a group consisting of members D to F.

[0111] For example, as shown in Figure 12, one group is formed from time T1 to T2. Then, when the group splits into two at time T2, the group control unit 203 determines that one group has split into two groups at time T3.

[0112] When a group splits, the group control unit 203 updates the group management database to reflect the split groups. In the example above, the group control unit 203 deletes the entry for the original group and adds entries for the two new groups created by the split to the group management database.

[0113] Next, we will explain the operation of the group control unit 203 when the decision-maker has not been selected before the group splits.

[0114] The group control unit 203 tracks the movement of each group after the split. After a predetermined period has elapsed since the split, the group control unit 203 selects decision-makers from each group. The group control unit 203 notifies store employees, etc., of the information of the selected decision-makers.

[0115] For example, in the above example, the group control unit 203 selects an approver from a group consisting of members A to C, and selects an approver from a group consisting of members D to F.

[0116] As explained above, the group control unit 203 may select an approver from a group with three or more members.

[0117] Thus, the group control unit 203 may also function as a split detection means, which detects a split in a group using time-series data of the current location of each of the stored members (the movement path of each member). Furthermore, the group control unit 203 (selection means) may select a decision-maker from the group formed by the split. As a result, the server device 10 can notify store employees and others of the new decision-maker without fail.

[0118] Next, we will explain the operation of the group control unit 203 when the decision-makers have already been selected before the group splits.

[0119] When one group splits into multiple groups, the group control unit 203 estimates the "attributes" of each of the split groups. More specifically, the group control unit 203 estimates the group attributes based on factors such as the presence or absence of common attributes among the members constituting each group.

[0120] For example, the group control unit 203 estimates the attributes of each group using artificial intelligence (AI). For example, the group control unit 203 inputs image data of each member to the AI ​​along with a prompt such as, "Please estimate the group's attributes based on the attributes and commonalities of each person in the image data." The group control unit 203 may also estimate the attributes of each group using a pre-trained machine learning model.

[0121] Generative AI (for example, a large-scale language model) analyzes image data to output group attributes such as "male group" or "female group." However, if there are no commonalities among the group members, the generative AI may output a result such as "group attribute unknown."

[0122] The group control unit 203 determines a policy regarding the selection of decision-makers for each of the multiple groups newly created by the splitting of a group, depending on whether the group includes decision-makers who have already been selected and whether the group's attributes have been estimated.

[0123] For example, let's consider a scenario where the original group, consisting of members A through F, splits into two groups: Group 1, consisting of members A through C, and Group 2, consisting of members D through F. Furthermore, let's assume that member A has already been elected as the decision-maker for the original group.

[0124] If the group attributes of group 1 are estimated, the group control unit 203 re-selects a decision-maker from among members A to C belonging to group 1. As a result of the re-selection, member A may be selected as the decision-maker again, or member B or C may be selected as the decision-maker.

[0125] If the group attributes of group 1 are unknown, the group control unit 203 will continue to use member A as the decision-maker, since member A, who belongs to group 1, was selected as the decision-maker for the original group.

[0126] If the group attributes of group 2 have been estimated, the group control unit 203 will select a new approver from among members D to F belonging to group 2.

[0127] If the group attributes of group 2 are unknown, the group control unit 203 may select a new approver from among members D to F belonging to group 2, or it may not select an approver.

[0128] Thus, the group control unit 203 (selection means) may not select a decision-maker from the group to which the decision-maker selected before the split belongs, among the multiple groups created by the split of the group, but may instead inherit the decision-maker selected in the group before the split. As a result, the processing load on the server device 10 is reduced.

[0129] Alternatively, the group control unit 203 (selection means) may estimate the attributes of each of the multiple groups resulting from the splitting of the group. The group control unit 203 may select decision-makers from the groups whose attributes have been estimated, and may not select decision-makers from the groups whose attributes have not been estimated. The server device 10 reduces the load by not selecting decision-makers as needed.

[0130] In other words, the group control unit 203 attempts to estimate the group attributes for each group after the split. For groups for which the group attributes can be estimated, the group control unit 203 selects a new decision-maker from the new group after the split. For groups for which the group attributes can be estimated, the group control unit 203 determines that a new group has been formed that will act according to the group attributes, and selects a decision-maker from that new group.

[0131] In contrast, for groups whose group attributes are unknown and to which the decision-maker selected in the original group belongs, the group control unit 203 will not select a new decision-maker, instead inheriting the already selected decision-maker. If the attributes of the new group resulting from the group split cannot be calculated (i.e., the group members have no commonalities or relationships), the group control unit 203 will determine that there is no change in the decision-maker and will inherit the already selected decision-maker.

[0132] Furthermore, for groups whose group attributes are unknown and to which the decision-maker selected in the original group does not belong, the group control unit 203 may or may not select a new decision-maker from that group.

[0133] The group control unit 203 may decide, depending on the actions of the group after the split, whether to take over the already elected decision-makers or to elect new decision-makers.

[0134] More specifically, the group control unit 203 may determine, based on the actions of the group to which the original group's decision-maker belongs, whether to take over the original decision-maker or select a new one, given that the group's group attributes have been estimated.

[0135] For example, the group control unit 203 identifies the sales floors visited by the group whose group attributes have been calculated. More specifically, the group control unit 203 sets the sales floors visited by the group as those where the group's center of gravity does not move over a predetermined period of time. The group control unit 203 identifies the sales floors visited by the group by referring to table information that stores the location information (coordinates in the store coordinate system) of each sales floor.

[0136] The group control unit 203 determines whether there is a relationship between the group attributes and the sales floor the group visited. For example, if a "male group" visits the men's clothing section, the group control unit 203 determines that there is a relationship between the group attributes and the sales floor the group visited.

[0137] The group control unit 203 may also refer to table information that stores group attributes and store information (store information related to the group attributes) in association, and determine whether or not the above relationship exists. Alternatively, the group control unit 203 may use a generating AI (large-scale language model) to determine whether or not there is a relationship between the group attributes and the store visited.

[0138] If there is a relationship between group attributes and sales floor, the group control unit 203 will be confident that the group is acting separately from other groups and will select a new decision-maker from that group. In other words, because it can determine whether the group split in order to engage in new purchasing behavior, the group control unit 203 will select a new decision-maker from that group.

[0139] Conversely, if there is no relationship between group attributes and sales locations, the group control unit 203 inherits the decision-makers selected in the original group. In other words, the group control unit 203 does not select new decision-makers from the newly formed group after the split.

[0140] Thus, the group control unit 203 (selection means) may determine whether each member belonging to the group whose attributes have been estimated is acting in accordance with the estimated attributes. If each member belonging to the group whose attributes have been estimated is acting in accordance with the estimated attributes, the group control unit 203 may select a decision-maker from the group whose attributes have been estimated. Alternatively, if each member belonging to the group whose attributes have been estimated is not acting in accordance with the estimated attributes, the group control unit 203 may not select a decision-maker from the group whose attributes have been estimated. If the group whose attributes have been estimated includes a decision-maker selected in the group before the split, the group control unit 203 may take over the decision-maker selected in the group before the split. Through this operation of the group control unit 203, information on a more reliable decision-maker is notified to store staff, etc. That is, if the group control unit 203 determines that the group is acting in accordance with the estimated group attributes, it assumes that the estimated group attributes have a high degree of reliability and selects a new decision-maker from that group.

[0141] Next, a modified example according to the second embodiment will be described.

[0142] <Modification 1> If one group splits into multiple groups, the server device 10 may select a new decision-maker from the group with the largest number of members. For example, if a group with 3 members and a group with 4 members are newly formed, the group control unit 203 may select a decision-maker for the group with 4 members.

[0143] <Modification 2> When the server device 10 notifies the store clerk or other person of the payer's information, it may also notify the reliability of the payer's decision result.

[0144] For example, when the group control unit 203 notifies information about a decision-maker selected from a group that has taken action according to the estimated group attributes, it may notify that the decision result regarding that decision-maker is "high confidence."

[0145] More specifically, the group control unit 203 (notification means) may notify a person related to a predetermined facility (for example, a store employee) of the confidence level regarding the result of the decision-maker selection. In this case, the group control unit 203 may set the confidence level regarding the selection of a decision-maker who was selected when each member belonging to a group whose attributes were estimated was acting in accordance with the estimated attributes, higher than the confidence level regarding the selection of a decision-maker from other groups. As a result, the employee can provide more efficient customer service.

[0146] <Modification 3> Even if a member leaves the group and the group disbands, the server device 10 may track the movements of each member. Furthermore, the server device 10 may detect the gathering of the departing members (reformation of the group).

[0147] The group control unit 203 calculates the distance between each member who has left the group, and determines that the members who have left have reunited if the distance between each member falls below a predetermined value. When the members who have left have reunited and the group has been reformed, the group control unit 203 identifies the member that the multiple members have decided to reunite with.

[0148] For example, consider a group consisting of members A, B, and C, where the three members are acting separately (see Figure 13). If, after the group disbands, member A's travel route is the shortest, and the distance between members A, B, and C is less than or equal to a predetermined value, the group control unit 203 identifies member A as the member that members B and C have met up with.

[0149] When multiple members identify a member to whom they have gathered, the group control unit 203 may recognize the identified member as the decision-maker of the reformed group. That is, the group control unit 203 may treat the movement of the member to whom the other members have gathered as the movement of the group and select that member as the decision-maker.

[0150] As described above, the server device 10 according to the second embodiment detects group splits, etc., and selects a decision-maker for the newly formed group. The server device 10 grasps group splits, etc., in real time and selects a decision-maker as needed. As a result, store employees, etc., can find out who the new decision-makers are, and can provide appropriate customer service.

[0151] [Third Embodiment] Next, a third embodiment will be described in detail with reference to the drawings.

[0152] In the third embodiment, we will describe tracking the movement of a group across multiple stores (selection of the payment person).

[0153] The following will focus on explaining the differences between the first to third embodiments.

[0154] The information processing system disclosed in this application can also be applied to shopping malls consisting of multiple stores. For example, the server device 10 may perform group movement tracking and payment selection for customers of a shopping mall where multiple stores A to D are operating, as shown in Figure 14.

[0155] Furthermore, camera devices 30 are placed in various locations within the shopping mall and its stores, as shown in Figure 14.

[0156] In this scenario, a group shopping at a shopping mall may have different payers for each store they visit. The server device 10 takes this possibility into consideration when selecting a payer.

[0157] Specifically, the group control unit 203 detects a group of customers when multiple customers enter a shopping mall. The group control unit 203 tracks the movement of the group and selects a payer when the group enters a store. The group control unit 203 then notifies the store staff of the store the group entered of the information of the selected payer.

[0158] When a group leaves a store, the group control unit 203 initializes the movement paths of each member and the group as a whole. The group control unit 203 initializes the movement paths of each member and the group as a whole, and tracks the group's movement from the time they leave a store until they enter another store.

[0159] The group control unit 203 selects a decision-maker based on the movement between stores (the movement of each member and the group between stores) when the group enters another store. The group control unit 203 notifies the store staff of the store the group entered of the information of the selected decision-maker.

[0160] For example, in Figure 14, when a group enters a shopping mall and then enters store A, the group control unit 203 selects a payer based on the movement of each member and the group from the time they enter the shopping mall until they enter store A. The group control unit 203 then notifies the store staff of store A of the information of the selected payer.

[0161] When the group leaves store A and moves towards store D, the group control unit 203 selects a decision-maker for the group based on the movement of each member and the group from store A to store D. The group control unit 203 then notifies the staff of store D of the information of the selected decision-maker.

[0162] As described above, the server device 10 according to the third embodiment re-selects a payer for a group each time the group leaves a store in a facility where multiple stores are operating, such as a shopping mall. That is, each time the group leaves a store, the server device 10 initializes the group's center of gravity and tracks the group's movement. When the group enters a store, the server device 10 selects a new payer based on the movement between stores (the movement of members and the group, etc.) and notifies the store staff of the information of the new payer. As a result, even if the payer changes chronologically depending on the stores the group visits, the server device 10 can notify the store staff of accurate payer information.

[0163] Next, we will describe the hardware of each device that makes up the information processing system. Figure 15 shows an example of the hardware configuration of the server device 10.

[0164] The server device 10 can be configured as an information processing device (a so-called computer), and has the configuration illustrated in Figure 15. For example, the server device 10 includes a processor 311, memory 312, input / output interface 313, and communication interface 314, etc. The components of the processor 311, etc. are connected by an internal bus or the like and are configured to communicate with each other.

[0165] However, the configuration shown in Figure 15 is not intended to limit the hardware configuration of the server device 10. The server device 10 may include hardware not shown, and it may not have to include the input / output interface 313 if necessary. Also, the number of processors 311 etc. included in the server device 10 is not intended to be limited to the example in Figure 15; for example, the server device 10 may include multiple processors 311.

[0166] The processor 311 is a programmable device such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or DSP (Digital Signal Processor). Alternatively, the processor 311 may be a device such as an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). The processor 311 executes various programs, including an operating system (OS).

[0167] Memory 312 can be RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc. Memory 312 stores the OS program, application programs, and various data.

[0168] The input / output interface 313 is an interface for a display device or input device (not shown). The display device is, for example, a liquid crystal display. The input device is, for example, a device that accepts user input such as a keyboard or mouse.

[0169] The communication interface 314 is a circuit, module, etc., that communicates with other devices. For example, the communication interface 314 may include a NIC (Network Interface Card), etc.

[0170] The functions of the server device 10 are realized by various processing modules. These processing modules are realized, for example, by the processor 311 executing a program stored in the memory 312. The program can also be recorded on a computer-readable storage medium. The storage medium can be a non-transitory material such as semiconductor memory, hard disk, magnetic recording medium, or optical recording medium. In other words, the present invention can also be embodied as a computer program product. Furthermore, the program can be downloaded via a network or updated using the storage medium on which the program is stored. Moreover, the processing module may be realized by a semiconductor chip.

[0171] The server device 10, which is an information processing device, is equipped with a computer, and its functions can be realized by having the computer execute a program. Furthermore, the server device 10 executes a control method for the server device 10 using this program.

[0172] [Modification] The configuration and operation of the information processing system described in the above embodiment are illustrative examples and are not intended to limit the system configuration.

[0173] In the above embodiment, the server device 10 described a case where multiple people captured in image data obtained by photographing the entrance of a store are set into a group. However, the server device 10 may also set a group if it can calculate group attributes from the group (multiple people). For example, the server device 10 may set a group if it can obtain group attributes such as "group of men," "group of women," or "family with children" by inputting the image data into the generation AI.

[0174] The above embodiment describes a case in which facial information (facial images) is used to calculate the user's movement path. However, the server device 10 may use other information instead of or in addition to facial information to calculate the user's movement path (track the user). For example, the group control unit 203 may use characteristics such as the characteristics of the clothing the user is wearing or the user's posture to understand the user's movement path.

[0175] In the above embodiment, the case in which the group management database is configured inside the server device 10 was described, but the database may be built on an external database server or the like. In other words, some functions of the server device 10 may be implemented on another server. More specifically, it is sufficient that the "group control unit (group control means)" etc. described above is implemented in any device included in the system.

[0176] The form of data transmission and reception between each device (server device 10, terminal 20, etc.) is not particularly limited, but the data transmitted and received between these devices may be encrypted. Since user facial images and other information are transmitted and received between these devices, it is desirable that encrypted data be transmitted and received in order to properly protect this information.

[0177] In the flowcharts (sequence diagrams) used in the above description, multiple processes are shown in order, but the execution order of the processes performed in the embodiment is not limited to the order in which they are shown. In the embodiment, the order of the illustrated processes can be changed to the extent that it does not impede the content, for example, by executing each process in parallel.

[0178] The embodiments described above are explained in detail to facilitate understanding of the disclosure, and it is not intended that all the configurations described above are necessary. Furthermore, when multiple embodiments are described, each embodiment may be used individually or in combination. For example, it is possible to replace parts of the configuration of one embodiment with those of another embodiment, or to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of one embodiment with those of another.

[0179] As described above, the industrial applicability of the present invention is clear, and it is particularly suitable for application to information processing systems that grasp group behavior and provide information useful to stores.

[0180] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0181] [Note 1] An information processing device comprising: detection means for detecting a group of multiple members that have entered a predetermined facility; calculation means for calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; storage means for storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and for storing the time-series data of the calculated centroid of the group as the movement line of the group; selection means for selecting a member who is presumed to have the authority to make a decision from among the multiple members as a decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and notification means for notifying a person related to the predetermined facility of the information of the selected decision-maker.

[0182] [Note 2] The information processing device according to Note 1, wherein the selection means selects the member who has the smallest sum of distances between the movement paths of each of the stored members and the movement paths of the stored group from among the multiple members to be the decision-maker.

[0183] [Note 3] The information processing device according to Note 1, wherein the detection means calculates a variance value relating to the movement of each of the multiple members stored during a predetermined period after the group enters the predetermined facility, and determines that the group consisting of the multiple members has been detected if each of the calculated variance values ​​is smaller than a predetermined value.

[0184] [Appendix 4] The information processing apparatus according to any one of Appendix 1 to 3, further comprising a split detection means for detecting a split of the group using time-series data of the current positions of each of the stored members, wherein the selection means selects the decision-maker from the group resulting from the split of the group.

[0185] [Note 5] The information processing device described in Note 4, wherein the selection means does not select the decision-maker from the group to which the decision-maker selected before the split belongs among the multiple groups resulting from the split of the group, but rather inherits the decision-maker selected in the group before the split.

[0186] [Appendix 6] The information processing apparatus according to Appendix 5, wherein the selection means estimates the attributes of each of the multiple groups resulting from the splitting of the group, selects the decision-maker from the group whose attributes have been estimated, and does not select the decision-maker from the group whose attributes have not been estimated.

[0187] [Note 7] The information processing device according to Note 6, wherein the selection means determines whether each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, selects the decision-maker from the group whose attributes are estimated if each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, and does not select the decision-maker from the group whose attributes are estimated if each member belonging to the group whose attributes are estimated is not acting in accordance with the estimated attributes.

[0188] [Note 8] The information processing device according to Note 7, wherein the notification means notifies a person related to the predetermined facility of the confidence level regarding the result of the selection of the decision-maker, and sets the confidence level regarding the selection of the decision-maker who was selected when each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, higher than the confidence level regarding the selection of the decision-maker who was selected from other groups.

[0189] [Note 9] A control method for an information processing device, comprising: a detection step of detecting a group consisting of multiple members that have entered a predetermined facility; a calculation step of calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; a storage step of storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and storing the time-series data of the calculated centroid of the group as the movement line of the group; a selection step of selecting a member who is assumed to have the authority to make a decision from among the multiple members as the decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and a notification step of notifying a person related to the predetermined facility of the information of the selected decision-maker.

[0190] [Note 10] The control method for the information processing device described in Note 9, wherein the selection step is to select the member who has the smallest sum of distances between the movement paths of each of the stored members and the movement paths of the stored group as the decision-maker.

[0191] [Note 11] The control method for the information processing device described in Note 9, wherein the detection step involves calculating a variance value for the movement paths of each of the multiple members stored during a predetermined period after the group enters the predetermined facility, and determining that the group consisting of the multiple members has been detected if each of the calculated variance values ​​is smaller than a predetermined value.

[0192] [Note 12] A control method for an information processing device according to any one of Notes 9 to 11, further comprising a split detection step of detecting a split in the group using time-series data of the current positions of each of the stored members, wherein the selection step of selecting the decision-maker from the group formed by the split of the group.

[0193] [Note 13] The control method for the information processing device described in Note 12, wherein the selection step does not select the decision-maker from the group to which the decision-maker selected before the split belongs among the multiple groups resulting from the split of the group, but rather inherits the decision-maker selected in the group before the split.

[0194] [Note 14] The control method for the information processing device described in Note 13, wherein the selection step involves estimating the attributes of each of the multiple groups resulting from the splitting of the group, selecting the decision-maker from the group whose attributes have been estimated, and not selecting the decision-maker from the group whose attributes have not been estimated.

[0195] [Note 15] The control method for the information processing device described in Note 14, wherein the selection step involves determining whether each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, selecting the decision-maker from the group whose attributes are estimated if each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, and not selecting the decision-maker from the group whose attributes are estimated if each member belonging to the group whose attributes are estimated is not acting in accordance with the estimated attributes.

[0196] [Note 16] The control method for the information processing device described in Note 15, wherein the notification step involves notifying a person related to the predetermined facility of the confidence level regarding the result of the selection of the decision-maker, and setting the confidence level regarding the selection of the decision-maker who was selected when each member belonging to the group whose attributes were estimated is acting in accordance with the estimated attributes, higher than the confidence level regarding the selection of the decision-maker who was selected from other groups.

[0197] [Note 17] A computer-readable storage medium that stores a program for causing a computer mounted on an information processing device to execute: a detection process for detecting a group of multiple members that have entered a predetermined facility; a calculation process for calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; a storage process for storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and storing the time-series data of the calculated centroid of the group as the movement line of the group; a selection process for selecting a member who is assumed to have the authority to make decisions from among the multiple members as a decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and a notification process for notifying persons related to the predetermined facility of the information of the selected decision-maker.

[0198] [Note 18] The storage medium described in Note 17, wherein the selection process selects the member who has the smallest sum of distances between the movement paths of each of the stored members and the movement paths of the stored group from among the multiple members to be the decision-maker.

[0199] [Note 19] The storage medium described in Note 17, wherein the detection process calculates a variance value relating to the movement of each of the multiple members stored during a predetermined period after the group enters the predetermined facility, and determines that the group consisting of the multiple members has been detected if each of the calculated variance values ​​is smaller than a predetermined value.

[0200] [Note 20] The storage medium according to any one of Notes 17 to 19, wherein a split detection process is further performed to detect a split in the group using time-series data of the current positions of each of the stored members, and the selection process selects the decision-maker from the group formed by the split of the group.

[0201] [Note 21] The storage medium described in Note 20, wherein the selection process does not select the decision-maker from the group to which the decision-maker selected before the split belongs among the multiple groups resulting from the split of the group, but rather inherits the decision-maker selected in the group before the split.

[0202] [Note 22] The storage medium described in Note 21, wherein the selection process involves estimating the attributes of each of the multiple groups resulting from the splitting of the group, selecting the decision-maker from the group whose attributes are estimated, and not selecting the decision-maker from the group whose attributes are not estimated.

[0203] [Note 23] The selection process is as described in Note 22, wherein the storage medium determines whether each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, and if each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, the decision-maker is selected from the group whose attributes are estimated, and if each member belonging to the group whose attributes are estimated is not acting in accordance with the estimated attributes, the decision-maker is not selected from the group whose attributes are estimated.

[0204] [Note 24] The storage medium described in Note 23, wherein the notification process notifies a person related to the predetermined facility of the confidence level regarding the result of the selection of the decision-maker, and sets the confidence level regarding the selection of the decision-maker who was selected when each member belonging to the group whose attributes were estimated is acting in accordance with the estimated attributes, higher than the confidence level regarding the selection of the decision-maker who was selected from other groups.

[0205] [Claim 25] The information processing apparatus according to Appendix 1, wherein the selection means selects the decision-maker when the number of members forming the group is three or more.

[0206] [Claim 26] The control method for the information processing device according to Appendix 9, wherein the selection step is performed when the number of members forming the group is three or more, the decision-maker is selected.

[0207] [Claim 27] ​​The storage medium according to Appendix 17, wherein the selection process selects the approver when the number of members forming the group is three or more.

[0208] Furthermore, some or all of the configurations described in Appendices 2 to 8, which are subordinate to Appendice 1 above, may also be subordinate to Appendices 9 and 17 in the same way as those described in Appendices 2 to 8. Moreover, not limited to Appendices 1, 9 and 17, some or all of the configurations described as appendices may also be subordinate to various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above.

[0209] Furthermore, each disclosure of the above-mentioned prior art documents cited herein is incorporated herein by reference. Although embodiments of the present invention have been described above, the present invention is not limited to these embodiments. It will be understood by those skilled in the art that these embodiments are merely illustrative and that various modifications are possible without departing from the scope and spirit of the present invention. That is, the present invention naturally includes the entire disclosure, including the claims, and various modifications and alterations that can be made by those skilled in the art in accordance with the technical idea.

[0210] 10 Server device 20 Terminal 30 Camera device 100 Information processing device 101 Detection means 102 Calculation means 103 Storage means 104 Selection means 105 Notification means 201 Communication control unit 202 Image data control unit 203 Group control unit 204 Storage unit 311 Processor 312 Memory 313 Input / output interface 314 Communication interface

Claims

1. An information processing device comprising: detection means for detecting a group of multiple members who have entered a predetermined facility; calculation means for calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; storage means for storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and for storing the time-series data of the calculated centroid of the group as the movement line of the group; selection means for selecting a member who is presumed to have the authority to make a decision from among the multiple members as a decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and notification means for notifying a person related to the predetermined facility of the information of the selected decision-maker.

2. The information processing apparatus according to claim 1, wherein the selection means selects the member who has the smallest sum of distances between the movement paths of each of the stored members and the movement paths of the stored group from among the plurality of members to be the decision-maker.

3. The information processing apparatus according to claim 1, wherein the detection means calculates a variance value relating to the movement paths of each of the plurality of members stored during a predetermined period after the group enters the predetermined facility, and determines that the group consisting of the plurality of members has been detected if each of the calculated variance values ​​is smaller than a predetermined value.

4. The information processing apparatus according to any one of claims 1 to 3, further comprising a split detection means for detecting a split of the group using time-series data of the current positions of each of the stored members, wherein the selection means selects the decision-maker from the group resulting from the split of the group.

5. The information processing apparatus according to claim 4, wherein the selection means does not select the decision-maker from the group to which the decision-maker selected before the split belongs among the multiple groups resulting from the split of the group, but rather inherits the decision-maker selected in the group before the split.

6. The information processing apparatus according to claim 5, wherein the selection means estimates the attributes of each of the multiple groups resulting from the splitting of the group, selects the decision-maker from the group whose attributes have been estimated, and does not select the decision-maker from the group whose attributes have not been estimated.

7. The information processing apparatus according to claim 6, wherein the selection means determines whether each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, selects the decision-maker from the group whose attributes are estimated if each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, and does not select the decision-maker from the group whose attributes are estimated if each member belonging to the group whose attributes are estimated is not acting in accordance with the estimated attributes.

8. The information processing apparatus according to claim 7, wherein the notification means notifies a person related to the predetermined facility of the confidence level regarding the result of the selection of the decision-maker, and sets the confidence level regarding the selection of the decision-maker who was selected when each member belonging to the group whose attributes are estimated is acting in accordance with the estimated attributes, higher than the confidence level regarding the selection of the decision-maker who was selected from other groups.

9. A control method for an information processing device, comprising: a detection step of detecting a group of multiple members that have entered a predetermined facility; a calculation step of calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; a storage step of storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and storing the time-series data of the calculated centroid of the group as the movement line of the group; a selection step of selecting a member who is assumed to have the authority to make a decision from among the multiple members as the decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and a notification step of notifying a person related to the predetermined facility of the information of the selected decision-maker.

10. A computer-readable storage medium that stores a program for causing a computer mounted on an information processing device to execute: a detection process for detecting a group of multiple members that have entered a predetermined facility; a calculation process for calculating the current position of each of the multiple members using image data captured by at least one camera device installed in the predetermined facility, and calculating the centroid of the group based on the calculated current positions of each of the multiple members; a storage process for storing the time-series data of the calculated current positions of each of the multiple members as the movement lines of each of the multiple members, and storing the time-series data of the calculated centroid of the group as the movement line of the group; a selection process for selecting a member who is assumed to have the authority to make decisions from among the multiple members as a decision-maker based on the stored movement lines of each of the multiple members and the stored movement line of the group; and a notification process for notifying persons related to the predetermined facility of the information of the selected decision-maker.

Images