Information processing program, information processing method, and information processing device

Abstract

There have been various measures considered for eliminating or alleviating congestion at cash registers, etc. of retail stores. For example, a measure is executed to eliminate or alleviate congestion by estimating a wait time and presenting the same to a person. Secondary congestion may occur when people head to a predetermined place, such as a cash register, all at once when the presented wait time elapses. This information processing program causes a computer to execute processing for: acquiring, upon reception of an output result of a measure, an action selection model for determining whether a target person will act according to the measure; using the acquired action selection model to execute a simulation in a digital twin for determining whether an agent follows a first measure; and selecting a first person, to which the first measure is applied in the real world, on the basis of the simulation result. Due to the foregoing, it is possible to suppress secondary congestion of people, such congestion caused by applying measures to the people.

Description

Information processing program, information processing method, and information processing device

[0001] The present invention relates to an information processing program, an information processing method, and an information processing device.

[0002] A technology called Digital Twin is used to represent objects that exist in the physical space of the real world in a virtual space. For example, Digital Twin uses the Internet of Things (IoT) to collect data in real time from operating factory production lines and equipment, and runs various simulations. Then, for example, by repeatedly notifying the site of the results of the simulation in the virtual space and feeding back the site's situation to the virtual space, it is possible to improve production efficiency and prevent potential accidents.

[0003] International Publication No. 2019 / 123702 Japanese Patent Application Laid-Open No. 2016-122355 Japanese Patent Application Laid-Open No. 2016-143312

[0004] Here, it is conceivable to implement measures using digital twins. For example, in facilities such as retail stores, certain areas such as cash registers can become crowded with people, causing queues of people to wait. For this reason, various measures to alleviate or mitigate congestion are being considered. For example, it is conceivable to implement measures to alleviate or mitigate congestion by estimating waiting times and presenting them to people.

[0005] However, because a person's future actions change depending on the situation in the real world, it is not possible to improve the reproducibility of a person's actions in a virtual space. For this reason, when implementing a policy using a digital twin, it is not possible to select the person to whom the policy should be applied.

[0006] In one aspect, an object is to improve the accuracy of selecting people to whom a policy is applied.

[0007] In one aspect, the information processing program receives the output results of a policy, acquires a behavioral selection model that determines whether a target person will act in accordance with the policy, and uses the acquired behavioral selection model to run a simulation in the digital twin that determines whether an agent will follow a first policy, and causes a computer to execute a process in the real world to select a first person to whom the first policy will be applied based on the results of the simulation.

[0008] In one aspect, it is possible to improve the accuracy of selecting the person to whom the policy is applied.

[0009] FIG. 1 is a diagram showing an example of a conventional waiting time display. FIG. 2 is a diagram showing an example of secondary congestion caused by a conventional waiting time display. FIG. 3 is a diagram showing an example of waiting time display according to this embodiment. FIG. 4 is a diagram showing an example of digital twin construction according to this embodiment. FIG. 5 is a diagram showing an example of the configuration of an information processing system 1 according to this embodiment. FIG. 6 is a diagram showing an example of the configuration of an information processing device 10 according to this embodiment. FIG. 7 is a diagram showing an example of waiting time display control according to this embodiment. FIG. 8 is a diagram showing another example of waiting time display control according to this embodiment. FIG. 9 is a diagram showing an example of the configuration of waiting time display control processing according to this embodiment. FIG. 10 is a flowchart showing an example of the flow of waiting time display control processing according to this embodiment. FIG. 11 is a diagram showing a first calculation example of the waiting time display control processing according to this embodiment. FIG. 12 is a diagram showing a second calculation example of the waiting time display control processing according to this embodiment. FIG. 13 is a flowchart showing an example of the flow of a policy display person identification processing according to this embodiment. FIG. 14 is a diagram explaining an example of the hardware configuration of the information processing device 10. FIG. 15 is a diagram explaining an example of the hardware configuration of the information processing terminal 100.

[0010] Below, examples of the information processing program, information processing method, and information processing device according to the present embodiment will be described in detail with reference to the drawings. Note that the present embodiment is not limited to these examples. Furthermore, each example can be appropriately combined within a consistent range.

[0011] First, secondary congestion caused by conventional wait time display will be specifically described using Figures 1 and 2. Figure 1 is a diagram showing an example of conventional wait time display. As shown in Figure 1, when a queue occurs at the register, for example, an information processing device estimates the wait time (e.g., 10 minutes) based on the number of people waiting at the register and displays this information on signage or the like. In this way, facilities such as retail stores can try to alleviate congestion by delaying customers who are about to line up at the register or the like.

[0012] Fig. 2 shows an example of secondary congestion caused by conventional waiting time notifications. However, even if a waiting time is displayed as shown in Fig. 1, there is a possibility that customers who have been waiting may suddenly head to the cash register after the displayed waiting time has elapsed, causing secondary congestion, as shown in Fig. 2. One of the reasons for this is thought to be that the same waiting time, such as 10 minutes, is displayed to all customers.

[0013] FIG. 3 is a diagram illustrating an example of a waiting time display according to this embodiment. Therefore, in this embodiment, for example, the information processing device 10 predicts the shopping time for each customer and determines whether to display the waiting time and delay the customer. For example, as shown in FIG. 3 , when the waiting time is 10 minutes, customer A, whose shopping time is predicted to be 20 minutes, will continue shopping until the congestion clears. Therefore, the information processing device 10 determines that there is no need to display the waiting time to customer A and delay the customer, and does not display the waiting time to customer A. On the other hand, customer B, whose shopping time is predicted to be 5 minutes, will head to the cash register before the congestion clears. Therefore, the information processing device 10 determines that there is a need to display the waiting time to customer B and delay the customer, and displays the waiting time via the information processing terminal 100 carried by customer B. The shopping time refers to, for example, the time from when a customer enters the store to when they head to the cash register. It can be estimated from customer attributes such as the customer's gender, age or generation, and occupation, or obtained using a store application on the customer's information processing terminal 100. Furthermore, for example, when presenting waiting times to customers, the information processing device 10 can spread out the arrival times of the cashiers by presenting different waiting times for each customer according to the amount of time they have spent shopping. The presented waiting time may be, for example, the estimated time when congestion will be resolved, calculated by adding the waiting time to the current time.

[0014] FIG. 4 is a diagram illustrating an example of digital twin construction according to this embodiment. Furthermore, as shown in FIG. 4 , the information processing device 10 constructs a digital twin, places agents corresponding to real-world customers on the digital twin, and performs simulations in a virtual space, thereby enabling selection of customers to whom waiting times should be presented and determination of the waiting times to be presented. The information processing device 10, for example, reproduces areas of a real-world store in the virtual space of the digital twin. For example, the information processing device 10 reproduces an area where a frozen food section and an area where a vegetable section are located in the virtual space. Furthermore, the information processing device 10, for example, reproduces an area where a cash register is located in the virtual space. Then, the information processing device 10 places, for example, people who exist in the real world as agents in the virtual space that reproduces the real world.

[0015] Next, an information processing system 1 for implementing this embodiment will be described. FIG. 5 is a diagram showing an example of the configuration of the information processing system according to this embodiment. As shown in FIG. 5, the information processing system 1 is a system in which, for example, an information processing device 10 and information processing terminals 100-1 to 100-N (N is any natural number; hereinafter collectively referred to as "information processing terminals 100") are connected to each other via a network 50 so as to be able to communicate with each other. The information processing device 10 is also connected to, for example, camera devices 200-1 to 200-M (M is any natural number; hereinafter collectively referred to as "camera devices 200") so as to be able to communicate with each other via the network 50.

[0016] The network 50 may be wired or wireless, and may be any of a variety of communication networks, such as an intranet used within a facility such as a retail store. The network 50 may not be a single network, but may instead be configured such that an intranet and the Internet are connected via a network device such as a gateway or other device (not shown). The facility, such as a retail store, is not limited to indoors, but may include, for example, outdoors within the premises.

[0017] The information processing device 10 is an information processing device such as a desktop PC (Personal Computer), a notebook PC, or a server computer that is installed in a facility such as a retail store and used by facility staff or managers.

[0018] For example, in order to collect real-world information in real time, the information processing device 10 receives video captured by the camera device 200 from the camera device 200. Strictly speaking, the video captured by the camera device 200 is a plurality of captured images captured by the camera device 200, i.e., a series of frames of a video.

[0019] Furthermore, the information processing device 10 may use, for example, existing object detection technology to detect customers (hereinafter, sometimes simply referred to as "people") staying in a facility such as a retail store from video. Note that object detection such as people from video may involve detecting a predetermined area in the video for each object, such as a bounding box, which is a rectangular area surrounding an object such as a person. Furthermore, the information processing device 10 may also detect, in addition to people, for example, a shopping basket or cart held by the person, and items contained therein, from video.

[0020] Furthermore, the information processing device 10 may, for example, place agents corresponding to people detected from captured images on the digital twin and detect congestion at a predetermined location such as a cash register. Alternatively, the information processing device 10 may, for example, use existing technology to analyze captured images to calculate the crowd density at a predetermined location such as a cash register and detect congestion.

[0021] Furthermore, the information processing device 10 may, for example, execute a simulation in the digital twin to determine whether an agent will comply with a first measure, and select a first person in the real world to whom the first measure will be applied based on the results of the simulation. Here, the measure may be, for example, a presentation of a waiting time for congestion at a predetermined location, such as a cash register, or an estimated time for the congestion to clear. Whether the agent will comply may be determined by whether the agent, in response to the presentation of the waiting time, heads to the predetermined location before the waiting time has elapsed. The simulation may be executed using, for example, a behavior selection model, which is a machine learning model for receiving an output result of the measure and determining whether the target person will act in accordance with the measure. Furthermore, the information processing device 10 may, for example, present the waiting time to the selected first person via the information processing terminal 100.

[0022] 5, the information processing device 10 is shown as a single computer, but may be a distributed computing system configured with multiple computers. Furthermore, the information processing device 10 may be a cloud computer device managed by a service provider that provides cloud computing services.

[0023] The information processing terminal 100 may be, for example, a mobile terminal such as a smartphone or tablet PC carried by a customer staying in a facility such as a retail store, or may be a dedicated terminal rented out within the facility. An application for displaying a waiting time, for example, is pre-installed on the information processing terminal 100. The information processing terminal 100 displays the waiting time received from the information processing device 10 via the application. Alternatively, the information processing terminal 100 may access a website or application on the information processing device 10 for displaying the waiting time, and display the waiting time. In this case, the information processing terminal 100 does not need to have an application for displaying the waiting time installed.

[0024] The camera device 200 is, for example, a surveillance camera installed in a facility such as a retail store. The camera device 200 captures an image of a predetermined location such as a cash register. Depending on the size of the facility, for example, there may be only one camera device 200, rather than multiple camera devices as shown in FIG. 5 . The image captured by the camera device 200 is transmitted to the information processing device 10.

[0025] [Functional configuration of information processing device 10] Next, a functional configuration of the information processing device 10 will be described. Fig. 6 is a diagram showing an example of the configuration of the information processing device 10 according to this embodiment. As shown in Fig. 6, the information processing device 10 has a communication unit 20, a storage unit 30, and a control unit 40.

[0026] The communication unit 20 is a processing unit that controls communication with other information processing devices such as the information processing terminal 100 and the camera device 200, and is, for example, a communication interface such as a network interface card or a USB (Universal Serial Bus) interface.

[0027] The storage unit 30 has the function of storing various data and programs executed by the control unit 40, and is realized by a storage device such as a memory or a hard disk. The storage unit 30 stores video information 31, real world information 32, digital twin information 33, and the like.

[0028] The video information 31 stores, for example, video captured by the camera device 200. The video stored in the video information 31 is video captured by the camera device 200 and transmitted to the information processing device 10, and the video information 31 may also store, for example, an identifier for uniquely identifying the camera device 200 that captured the video, the date and time of capture, and the like.

[0029] The real world information 32 stores real world information obtained from, for example, the information processing terminal 100 or images captured by the camera device 200. The real world information may be, for example, the time each person entered a facility such as a retail store obtained from the information processing terminal 100, the position of each person within the facility obtained by analyzing the image, and attributes of each person such as the gender, age or generation, and occupation of each person.

[0030] Digital twin information 33 stores information about the digital twin, such as agents corresponding to people in the real world, areas on the digital twin corresponding to facilities such as retail stores in the real world, and measures derived from simulations on the digital twin.

[0031] The above information stored in the storage unit 30 is merely an example, and the storage unit 30 can store various information other than the above information.

[0032] The control unit 40 is a processing unit, such as a processor, that controls the entire information processing device 10. The control unit 40 includes a detection unit 41, an acquisition unit 42, a simulation unit 43, and an output unit 44. Each processing unit is an example of an electronic circuit included in the processor or an example of a process executed by the processor.

[0033] The detection unit 41 detects congestion at a predetermined location, such as a cash register, from video captured by the camera device 200, for example. More specifically, the detection unit 41 places agents corresponding to each person detected from the video on the digital twin, and detects congestion at a predetermined location, such as a cash register. Alternatively, the detection unit 41 may use existing technology to analyze the video, calculate the crowd density at a predetermined location, such as a cash register, and detect congestion.

[0034] The acquisition unit 42 acquires, for example, a behavior selection model, which is a machine learning model for determining whether a target person will act in accordance with a measure in response to the output result of the measure. The behavior selection model is generated by training, for example, information regarding at least one of the agent's attributes, the time of entry into the store, the time when the congestion at the register will end, and the number of people waiting at the register as features, and information regarding whether the agent will head to the register by the time the congestion at the register will end as a correct answer label. The agent's attributes include, for example, at least one of the agent's gender, age or generation, and occupation.

[0035] Note that a behavior selection model may be created for each area set in the virtual space, for example. More specifically, for example, a behavior selection model is created to be applied to an agent located in an area in the virtual space that reproduces an area in the real world where a frozen food section is located. Also, a behavior selection model is created to be applied to an agent located in an area in the virtual space that reproduces an area in the real world where a vegetable section is located. In this case, the behavior selection model is trained as a machine learning model using the features of the agent in each area in the virtual space, with information regarding whether the agent will head to the cash register by the closing time as the correct answer label. For example, the machine learning model is trained based on training data including the features of people present in real-world areas corresponding to the areas set in the virtual space.

[0036] The acquisition unit 42 also acquires, for example, video captured inside the facility. For example, the video is captured by the camera device 200, transmitted from the camera device 200 as needed, and stored in the video information 31, and the acquisition unit 42 acquires the video from the video information 31.

[0037] The acquisition unit 42 also acquires real-world information contained in the video by analyzing the acquired video using, for example, existing technology. The real-world information may be, for example, each person detected in the video, the shopping cart or basket held by each person, and the products contained therein. The real-world information may also be, for example, the time each person entered a facility such as a retail store, which is acquired from the information processing terminal 100, the location of each person within the facility, which is acquired by analyzing the video, and attributes of each person, such as the gender, age or generation, and occupation of each person.

[0038] The simulation unit 43, for example, uses the acquired behavior selection model to execute a simulation in the digital twin to determine whether or not the agent will comply with the first measure. Furthermore, the simulation unit 43, for example, selects a first person in the real world to which the first measure will be applied based on the results of the simulation. Here, the measure may be, for example, a presentation of a waiting time for congestion at a predetermined location such as a cash register or an estimated time when the congestion will be resolved, and whether or not the agent will comply may be whether or not the agent, in response to the presentation of the waiting time, heads to the predetermined location before the waiting time has elapsed.

[0039] Furthermore, the process of executing the simulation may include, for example, a process of placing an agent corresponding to each person staying at the facility in the virtual space of the digital twin. Furthermore, the process of executing the simulation may include, for example, a process of executing a simulation to determine whether the agent will comply with a first measure applied to the facility based on the placed agent. Furthermore, the process of selecting a first person may include a process of selecting a first person from among the people staying at the facility based on the results of the simulation. Note that the first person may be selected based on real-world information included in the video acquired by the acquisition unit 42. For example, if the number of items in a person's shopping cart or other real-world information is greater than or equal to a predetermined threshold, the simulation unit 43 may first select another person as the first person because the person may cause secondary congestion.

[0040] In addition, the simulation unit 43, for example, places an agent corresponding to a person in the real world in the virtual space of the digital twin, and sets the attributes of the person, such as the gender, age or generation, and occupation of the corresponding person, for the agent placed in the virtual space.

[0041] Furthermore, the simulation unit 43 calculates the time it takes for an agent to reach a destination, which is a predetermined location such as a cash register, and determines whether or not to apply a measure to the agent using the calculated time and the number of people currently at the destination. Note that the measure to be applied or not determined here may be, for example, presentation of a waiting time for congestion at a predetermined location such as a cash register or an estimated time when the congestion will end, or presentation of a waiting time or estimated time when the congestion will end that differs for each agent.

[0042] The simulation unit 43 also estimates the end time of a predetermined behavior of the agent, for example, according to the attributes of the person associated with the agent. Here, the end time of the predetermined behavior may be, for example, the end time of shopping, and may be estimated from the time the target person entered the store, the shopping time according to the target person's attributes, etc.

[0043] 7 is a diagram illustrating an example of the wait time notification control according to the present embodiment. As shown in Fig. 7, the simulation unit 43 estimates (predicts) for each person the end time of a predetermined behavior (in the example of Fig. 7, the end time of the shopping trip) based on the time the target person enters the store and the attributes of the target person, for example, by performing a simulation in the virtual space of the digital twin.

[0044] The simulation unit 43 also estimates the time when congestion at the destination of the agent will be resolved based on the number of people waiting at the destination, which is a predetermined location such as a cash register, and the throughput for the number of people waiting. Here, the throughput for the number of people waiting may be, for example, the number of people per unit time who leave the queue at a predetermined location such as a cash register on the digital twin.

[0045] The simulation unit 43 also selects a first agent corresponding to the first person to whom the first measure is to be applied, for example, based on the estimated time when the congestion will end and the end time of the agent's predetermined behavior. For example, using the example of FIG. 7 , customer A's predicted shopping stop time (12:20) is later than the scheduled time when the congestion will end (12:10), so there is no need to detain him / her and there is no need to present a waiting time, which is an example of the first measure. Therefore, customer A is not selected as the first person (agent). On the other hand, customer B's shopping stop time (12:05) is earlier than the scheduled time when the congestion will end (12:10), so there is a need to detain him / her and he / she is selected as the first person (agent) to present a waiting time.

[0046] Furthermore, the simulation unit 43 identifies, for example, a person in the real world that corresponds to an agent located in the virtual space in the digital twin.

[0047] The output unit 44 outputs information related to the first measure to, for example, the information processing terminal 100 used by the first person. The information related to the first measure may be, for example, information such as a waiting time for congestion at a predetermined location such as a cash register or an estimated time when the congestion will be resolved. Note that, for example, the information related to the first measure may be different for each first person.

[0048] Furthermore, the output unit 44 outputs information including the wait time at the cash register to, for example, the information processing terminal 100 used by a person corresponding to an agent located in the virtual space of the digital twin. The information including the wait time at the cash register may be different for each person corresponding to an agent.

[0049] 8 is a diagram illustrating another example of the wait time notification control according to the present embodiment. As shown in FIG. 8, for example, information such as a different wait time and an estimated time when congestion will be resolved may be presented to each person selected by the simulation unit 43 as the first person to whom the first measure will be applied. This allows the information processing device 10 to further prevent secondary congestion caused by the presentation of information such as wait times.

[0050] The waiting time notification control for each person will be described in more detail. Fig. 9 is a diagram showing an example of the configuration of the waiting time notification control process according to this embodiment. As shown in Fig. 9, the waiting time notification control process according to this embodiment is composed of three processes: (1) creating a schedule for relieving congestion, (2) assigning customers, and (3) presenting information.

[0051] (1) In creating a congestion relief schedule, for example, the information processing device 10 calculates congestion relief times using the estimated congestion relief time, the register throughput, and the number of customers who have not yet come to the register as input data, and creates a congestion relief schedule. The congestion relief times are, for example, congestion relief times that are presented to customers. Furthermore, as shown in FIG. 9, congestion relief times are calculated for the number of customers who have not yet come to the register, and each may be a different time. Furthermore, at the congestion relief schedule stage, it is not necessary to assign a customer to each congestion relief time.

[0052] Next, in (2) customer allocation, for example, the information processing device 10 inputs the names of all customers who have not yet come to the cash register and the end times of their shopping trips, and assigns each customer to each hold-up release time. Note that the customer name may be, for example, a name such as Mr. A or Mr. B, which is used to uniquely identify the customer, as shown in Figure 9. Note that a specific method for assigning customers to hold-up release times will be described later.

[0053] Next, in (3) information presentation, for example, the information processing device 10 presents the assigned detention release time to the information processing terminal 100 of each customer.

[0054] [Processing Flow] Next, the waiting time presentation control process for each person shown in Fig. 9 will be described in more detail along the flow using Figs. 10 to 12. Fig. 10 is a flowchart showing an example of the flow of the waiting time presentation control process according to this embodiment. The time presentation control process shown in Fig. 10 is executed, for example, when the information processing device 10 detects congestion at a cash register on the digital twin.

[0055] 10, the information processing device 10 creates a congestion relief schedule based on, for example, the estimated congestion relief time, the throughput of the registers, and the number of customers who have not yet come to the registers (step S101). The creation of the congestion relief schedule will be described in more detail with reference to FIG.

[0056] 11 is a diagram showing a first calculation example of the waiting time display control process according to this embodiment. As shown on the left side of FIG. 11, for example, let's assume that the current time in the store is 12:00. Also, let's assume that the cash register throughput is one person per minute, the estimated time for congestion to end is 12:10, ten minutes from the current time, and the number of customers who have not yet come to the cash register is five, A through E.

[0057] The cash register throughput may be dynamically derived by, for example, the information processing device 10 counting the number of people leaving the cash register queue on an hourly basis on the digital twin. The estimated congestion relief time may also be derived by, for example, the information processing device 10 predicting the time when the number of people in the cash register queue will fall below a predetermined threshold based on the number of people in the cash register queue and the cash register throughput on the digital twin.

[0058] In addition, the shopping completion times of customers A through E who have not come to the register can be predicted and derived for each agent corresponding to each person, for example, by the information processing device 10 conducting a simulation in the virtual space of the digital twin, based on the time of entry of the target person, the target person's attributes, and the like. The shopping completion times can be predicted based on various information, such as the time of entry of the target person, the target person's attributes such as gender, age or generation, and occupation, as well as the current time of day, current or future weather, and the status of a sale. For example, if the current time of day is evening and the target person is a housewife, the shopping completion time may be earlier due to dinner preparations, etc. Furthermore, for example, depending on the current or future weather, if it looks like it will rain, the shopping completion time may be earlier, or later due to the target person killing time in the store until the rain stops. Furthermore, for example, if a limited-time sale is being held, the shopping completion time may be later. In the example of FIG. 11, it is assumed that the predicted shopping completion times for customers A to E are 12:05, 12:05, 12:20, 12:00, and 12:00, respectively, as shown on the left side of FIG.

[0059] First, the information processing device 10, for example, uses various information shown on the left side of FIG. 11 as input data and assigns the estimated congestion relief time (12:10) to the congestion relief time in the first line of the congestion relief schedule. Next, the information processing device 10 calculates the waiting time per person from, for example, the cash register throughput (1 minute in the example of FIG. 11), adds this waiting time to the congestion relief time in the first line of the congestion relief schedule (12:10), and assigns the resulting time (12:11) to the congestion relief time in the second line. The information processing device 10 then repeats the same process for the congestion relief times in the third line and beyond, for example, until it has repeated the process one time (the number of customers not yet at the cash registers (five in the example of FIG. 11)). This results in the creation of a congestion relief schedule with no assigned customers, such as the one shown on the right side of FIG. 11.

[0060] 10, the information processing device 10 then creates a customer list by listing all customers who have not come to the register (step S102). For example, in the example of FIG. 11, customers A to E who have not come to the register are listed.

[0061] Next, the information processing device 10 sorts the customer list created in step S102 in ascending order by the predicted shopping completion time of each customer (step S103). FIG. 12 is a diagram showing a second calculation example of the waiting time presentation control process according to this embodiment. The list shown in the lower left of FIG. 12 is an example of a customer list sorted in ascending order by shopping completion time. In the customer list shown in FIG. 12, for example, customers A to E who did not come to the register in the example of FIG. 11 are sorted in ascending order by shopping completion time. The table shown in the upper left of FIG. 12 is an example of a delay release schedule created in step S101.

[0062] 10, the information processing device 10 then assigns each customer to the hold-up release schedule created in step S101 (steps S104 to S106). For example, as shown in FIG. 10, the allocation of each customer to the hold-up release schedule is performed by looping the customer list sorted in ascending order by shopping end time in step S103 from the first customer to the last customer. The allocation of each customer to the hold-up release schedule will be described in more detail with reference to FIG. 12.

[0063] For example, as shown on the right side of Figure 12, the allocation of each customer to the congestion release schedule is performed by assigning the customer name in the congestion release schedule to the customer name in the order starting from the first line of the customer list sorted in ascending order by shopping end time. Note that, for example, in this case, if the shopping end time of the customer to be assigned is later than the congestion release time on the congestion release schedule, as in the case of customer C shown in Figure 12, the customer may be skipped without being assigned to the congestion release schedule, and the next customer may be assigned. In this case, for example, the next customer may include the customer whose assignment was skipped, and the next congestion release time on the congestion release schedule is compared with the shopping end time of the skipped customer to determine whether to assign the customer to the congestion release schedule.

[0064] 10 , after the process of assigning customers up to the last line of the customer list to the delay release schedule has been completed, the information processing device 10 then presents, for example, the delay release time to each assigned customer via the information processing terminal 100 (step S107). After step S107 is executed, the time presentation control process shown in FIG. 10 ends.

[0065] [Regarding the process of identifying a person to whom a policy should be applied] Here, an application example of a process of identifying a person to whom a policy should be applied from among multiple people staying in a facility using a digital twin will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the flow of the policy suggestion person identification process according to this embodiment. Note that the information processing device 10 repeatedly executes the policy suggestion person identification process at predetermined time intervals, for example.

[0066] First, as shown in FIG. 13 , the information processing device 10 detects, for example, people in the real world (step S201). For example, the information processing device 10 identifies the respective positions of multiple people staying in a facility. More specifically, the information processing device 10 uses, for example, existing object detection technology to analyze video captured by the camera device 200 to identify the person's position information and identify the respective positions of multiple people within a predetermined area. Alternatively, for example, the information processing device 10 may identify the respective positions of multiple people within a predetermined area by receiving position information of the information processing terminals 100 used by the people. For example, the information processing device 10 identifies customer A staying in an area where a frozen food section is located. Furthermore, the information processing device 10 identifies customer B staying in an area where a vegetable section is located.

[0067] Next, the information processing device 10 places an agent corresponding to the real-world person detected in step S201 (step S202). More specifically, for example, the information processing device 10 acquires area information within a facility reproduced in the virtual space of the digital twin. Then, for example, based on the position information of the real-world person, the information processing device 10 places agents corresponding to the person at positions in the virtual space corresponding to each of the positions of multiple people existing in the real world. For example, the information processing device 10 places agent A corresponding to customer A staying in an area where a frozen food section is located in an area in the virtual space that reproduces the frozen food section. Furthermore, for example, the information processing device 10 places agent B corresponding to customer B staying in an area where a vegetable section is located in an area in the virtual space that reproduces the vegetable section.

[0068] Next, the information processing device 10 executes a simulation to determine whether the agent deployed in step S202 complies with the policy (step S203). For example, the information processing device 10 executes a simulation to determine whether the agent complies with the policy using the agent's position in the virtual space and a behavior selection model. More specifically, the information processing device 10 first selects one agent to which the policy is to be applied from among multiple agents deployed in the virtual space. Here, the selected agent corresponds to, for example, a first agent corresponding to a first person to whom the first policy is to be applied. Next, the information processing device 10 identifies, for example, the position of the selected agent in the virtual space. Then, the information processing device 10 determines whether the agent complies with the policy by, for example, inputting the feature values ​​of the agent into a behavior selection model associated with the area in which the agent is located. Note that determining whether the agent complies with the policy means, for example, determining whether the agent will move to a destination where a cash register is located within a predetermined time until the cash register congestion ends. For example, the information processing device 10 inputs the feature amounts of agent A into a behavior selection model associated with an area in the virtual space that reproduces an area where a frozen food section is located, and thereby determines whether or not the agent A will comply with the policy. Also, the information processing device 10 inputs the feature amounts of agent B into a behavior selection model associated with an area in the virtual space that reproduces an area where a vegetable section is located, and thereby determines whether or not the agent B will comply with the policy.

[0069] Next, the information processing device 10 identifies, for example, a person to whom the policy should be applied from among a plurality of people existing in the real world (step S204). More specifically, for example, the information processing device 10 identifies, from among a plurality of people existing in the real world, a person corresponding to an agent determined to follow the first policy by the simulation executed in step S203, as a person to whom the policy should be applied. Furthermore, for example, when it is determined that one agent will follow the first policy, the information processing device 10 identifies, from among a plurality of people existing in the real world, a person corresponding to the selected agent as a first person to whom the first policy should be applied.

[0070] Next, the information processing device 10 determines whether the person identified in step S204 is currently present in a predetermined area (step S205). More specifically, for example, the information processing device 10 determines, based on the person's location information, whether the person identified in step S204 is currently present in a predetermined area set as an area for notifying information related to the policy. Here, the predetermined area is, for example, an area including the location where the person was detected in step S201. If it is determined that the person identified in step S204 is not present in the predetermined area (step S205: No), the information processing device 10 returns to, for example, step S203 and repeats the process.

[0071] On the other hand, if it is determined that the person identified in step S204 is present in the predetermined area (step S205: Yes), the information processing device 10 outputs information about the measure to, for example, the information processing terminal 100 used by the person (step S206). More specifically, for example, the information processing device 10 designates the person identified in step S204 as the first person to whom the first measure will be applied, and notifies the information processing terminal 100 used by the person of information about the first measure. This may involve, for example, notifying the information processing terminal 100 used by the first person of information about guidance to the cash register as information about the first measure.

[0072] [Effects] The information processing device 10 can reflect the situation in the real world and improve the reproducibility of people's actions in the virtual space. Furthermore, the information processing device 10 can improve the accuracy of selecting people to whom measures should be applied when implementing measures by using digital twins.

[0073] Furthermore, as described above, the information processing device 10 receives the output results of the measures, acquires a behavioral selection model that determines whether or not the target person will act in accordance with the measures, and uses the acquired behavioral selection model to run a simulation in the digital twin that determines whether or not the agent will follow the first measure, and selects the first person in the real world to whom the first measure will be applied based on the results of the simulation.

[0074] In this way, the information processing device 10 uses a behavior selection model to simulate whether an agent will comply with a measure such as a waiting time notification on the digital twin, and selects a person to whom the measure will be applied. This allows the information processing device 10 to improve the accuracy of selecting a person to whom the measure will be applied.

[0075] In addition, the process of executing a simulation performed by the information processing device 10 includes a process of placing an agent corresponding to each person staying at the facility in the virtual space of the digital twin, and performing a simulation to determine whether the agent will comply with a first measure applied to the facility based on the placed agent, and the process of selecting a first person performed by the information processing device 10 includes a process of selecting the first person from among the people staying at the facility based on the results of the simulation, and the information processing device 10 outputs information related to the first measure to an information processing terminal used by the first person.

[0076] This allows the information processing device 10 to improve the accuracy of selecting people to whom the policy is applied.

[0077] In addition, the information processing device 10 acquires video footage taken inside the facility and analyzes the acquired video footage to acquire the attributes of the people included in the video footage, and for each person staying at the facility, places an agent corresponding to the person in the virtual space of the digital twin and sets attributes for the agent placed in the virtual space.

[0078] This allows the information processing device 10 to improve the accuracy of selecting people to whom the policy is applied.

[0079] Furthermore, the information processing device 10 calculates the time it takes for the agent to reach the destination, and determines whether or not to apply a measure to the agent using the calculated time and the number of people currently staying at the destination.

[0080] This allows the information processing device 10 to improve the accuracy of selecting people to whom the policy is applied.

[0081] Furthermore, the information processing device 10 sets attributes of the corresponding person to the agent, estimates the end time of the agent's specified action according to the set attributes, estimates the time when congestion at the destination will be resolved based on the number of people waiting at the agent's destination and the throughput for the number of people waiting, and selects a first agent corresponding to the first person based on the estimated resolution time and end time.

[0082] This allows the information processing device 10 to improve the accuracy of selecting people to whom the policy is applied.

[0083] In addition, the information processing device 10 identifies a person corresponding to an agent located in the virtual space of the digital twin, and outputs information including the waiting time at the cash register to an information processing terminal used by the identified person.

[0084] This allows the information processing device 10 to improve the accuracy of selecting people to whom the policy is applied.

[0085] In addition, the information processing device 10 acquires video footage taken inside the facility and analyzes the acquired video footage to acquire real-world information contained in the video footage, and the process of selecting a first person executed by the information processing device 10 includes a process of selecting a first person based on the results of a simulation and information.

[0086] This allows the information processing device 10 to improve the accuracy of selecting people to whom the policy is applied.

[0087] In addition, the process of executing a simulation, which is performed by the information processing device 10, includes a process of executing a simulation by inputting the first time when the congestion at the cash register will be resolved into a behavior selection model generated by training the model using information regarding at least one of the agent's attributes including at least one of the agent's gender, age or generation, and occupation, the time when the congestion at the cash register will be resolved, the agent's entry time, and the number of people waiting at the cash register as features, and information regarding whether the agent will head to the cash register by the time when the congestion at the cash register will be resolved as a correct answer label.

[0088] This allows the information processing device 10 to improve the accuracy of selecting people to whom the policy is applied.

[0089] Furthermore, the information processing device 10 places agents corresponding to each of the multiple people at positions in the virtual space that correspond to the respective positions of the multiple people based on position information of the people present in a specified area in the real world, and the process of executing a simulation executed by the information processing device 10 includes a process of selecting one first agent from the multiple agents placed in the virtual space and determining whether or not the first agent will follow a first measure using the position of the first agent in the virtual space and a behavior selection model, and the process of selecting a first person executed by the information processing device 10 includes a process of selecting a person corresponding to the first agent as the first person from the multiple people present in the real world if it is determined that the first agent will follow the first measure, and the information processing device 10 determines whether the identified first person is currently present in the specified area, and if it is determined that the first person is present in the specified area, outputs information regarding the first measure to the information processing terminal 100 used by the first person.

[0090] This enables the information processing device 10 to improve the accuracy of selecting a person to whom a policy should be applied and to present the policy to that person.

[0091] [System] The information, including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings, may be changed as desired unless otherwise specified. Furthermore, the specific examples, distributions, and numerical values ​​described in the embodiments are merely examples and may be changed as desired.

[0092] Furthermore, the specific form of distribution or integration of the components of each device is not limited to that shown in the drawings. That is, all or some of the components may be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions of each device may be realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0093] [Hardware] Fig. 14 is a diagram illustrating an example of the hardware configuration of the information processing device 10. As shown in Fig. 14, the information processing device 10 has a communication interface 10a, a hard disk drive (HDD) 10b, a memory 10c, and a processor 10d. The components shown in Fig. 14 are connected to each other via a bus or the like.

[0094] The communication interface 10a is a network interface card or the like, and communicates with other servers. The HDD 10b stores programs and databases that operate the functions shown in FIG.

[0095] The processor 10d is a hardware circuit that operates a process that executes each function described in FIG. 6 and other figures by reading a program that executes the same processing as each processing unit shown in FIG. 6 from the HDD 10b or the like and expanding it into the memory 10c. That is, this process executes the same functions as each processing unit possessed by the information processing device 10. Specifically, the processor 10d reads a program having the same functions as the detection unit 41, the acquisition unit 42, the simulation unit 43, and the output unit 44 from the HDD 10b or the like. Then, the processor 10d executes a process that executes the same processing as the simulation unit 43 and the like.

[0096] In this way, the information processing device 10 operates as an information processing device that executes operation control processing by reading and executing a program that executes processing similar to that of each processing unit shown in Figure 6. The information processing device 10 can also realize functions similar to those of the above-mentioned embodiment by reading a program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, this embodiment may also be applied in a case where another information processing device executes a program, or where the information processing device 10 and another information processing device cooperate to execute a program.

[0097] 6 can be distributed via a network such as the Internet. This program can be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and can be read out from the recording medium and executed by a computer.

[0098] Fig. 15 is a diagram illustrating an example of the hardware configuration of the information processing terminal 100. As shown in Fig. 15, the information processing terminal 100 includes a communication interface 100a, an HDD 100b, a memory 100c, a processor 100d, an input device 100e, and a display device 100f. The components shown in Fig. 15 are connected to each other via a bus or the like.

[0099] The communication interface 100a is a network interface card or the like, and communicates with other information processing devices. The HDD 100b stores programs and data for operating each function of the information processing terminal 100.

[0100] The processor 100d is a hardware circuit that reads out a program for executing the processing of each function of the information processing terminal 100 from the HDD 100b or the like and loads it into the memory 100c, thereby operating a process for executing each function of the information processing terminal 100. In other words, this process executes the same function as each processing unit of the information processing terminal 100.

[0101] In this way, the information processing terminal 100 operates as an information processing device that executes operation control processing by reading and executing a program that executes processing of each function of the information processing terminal 100. The information processing terminal 100 can also realize each function of the information processing terminal 100 by reading a program from a recording medium using a medium reading device and executing the read program. Note that the program in this other example is not limited to being executed by the information processing terminal 100. For example, this embodiment may also be applied to cases where another information processing device executes a program or where the information processing terminal 100 and another information processing device cooperate to execute a program.

[0102] The program for executing the processing of each function of the information processing terminal 100 can be distributed via a network such as the Internet. The program can be recorded on a computer-readable recording medium such as a hard disk, FD, CD-ROM, MO, or DVD, and can be executed by being read from the recording medium by a computer.

[0103] The input device 100e detects various input operations by the user, such as input operations for a program executed by the processor 100d. The input operations include, for example, a touch operation and inserting an earphone jack into the information processing terminal 100. Here, a touch operation refers to various contact actions on the display device 100f, such as tapping, double tapping, swiping, and pinching. The touch operation also includes, for example, an action of bringing an object, such as a finger, close to the display device 100f. The input device 100e may be, for example, a button, a touch panel, or a proximity sensor.

[0104] The display device 100f displays various visual information under the control of the processor 100d. The display device 100f may be a liquid crystal display (LCD), an organic light emitting diode (OLED), or a so-called organic electroluminescence (EL) display.

[0105] REFERENCE SIGNS LIST 1 Information processing system 10 Information processing device 10a Communication interface 10b HDD 10c Memory 10d Processor 20 Communication unit 30 Storage unit 31 Video information 32 Real world information 33 Digital twin information 40 Control unit 41 Detection unit 42 Acquisition unit 43 Simulation unit 44 Output unit 50 Network 100 Information processing terminal 100a Communication interface 100b HDD 100c Memory 100d Processor 100e Input device 100f Display device 200 Camera device

Claims

1. An information processing program that causes a computer to execute the following processes: obtain a behavioral selection model that receives the output result of a policy and determines whether a target person will act in accordance with the policy; use the obtained behavioral selection model to run a simulation in a digital twin to determine whether an agent will follow a first policy; and select a first person in the real world to which the first policy will be applied based on the results of the simulation.

2. The information processing program according to claim 1, characterized in that the process of executing the simulation includes a process of: placing an agent corresponding to each person staying at the facility in the virtual space of the digital twin, and executing the simulation to determine whether the agent will comply with the first measure applied to the facility based on the placed agent; and the process of selecting the first person includes a process of selecting the first person from the people staying at the facility based on the result of the simulation; and outputting information related to the first measure to an information processing terminal used by the first person.

3. The information processing program according to claim 1, characterized in that it causes the computer to execute the following processes: acquiring video footage taken within a facility; acquiring attributes of people included in the video footage by analyzing the acquired video footage; for each person staying at the facility, placing an agent corresponding to the person in a virtual space in the digital twin; and setting the attributes for the agent placed in the virtual space.

4. The information processing program according to claim 1, characterized in that the computer is caused to execute a process of calculating the time required for the agent to reach the destination, and determining whether or not to apply a policy to the agent using the calculated time and the number of people currently present at the destination.

5. The information processing program according to claim 1, characterized in that the computer is caused to execute the following processes: setting attributes of a corresponding person to the agent; estimating the end time of a specified action of the agent according to the set attributes; estimating the time when congestion at the destination will be resolved based on the number of people waiting at the agent's destination and the throughput for the number of people waiting; and selecting a first agent corresponding to the first person based on the estimated resolution time and end time.

6. The information processing program according to claim 1, characterized in that the computer is caused to execute a process of identifying a person corresponding to the agent located in the virtual space of the digital twin, and outputting information including the waiting time at the cash register to an information processing terminal used by the identified person.

7. The information processing program according to claim 1, further comprising: acquiring video footage taken inside a facility; analyzing the acquired video footage to acquire information about the real world contained in the video footage; and having the computer execute processing; and wherein the processing of selecting the first person includes processing of selecting the first person based on the results of the simulation and the information.

8. The information processing program according to any one of claims 1 to 7, characterized in that the process of executing the simulation includes a process of inputting the first time when the congestion at the cash register is resolved into the behavior selection model generated by training the model using information regarding at least one of the agent's attributes including at least one of the agent's gender, age or generation, and occupation, the time when the congestion at the cash register is resolved, the agent's entry time, and the number of people waiting at the cash register as features, and information regarding whether the agent will head to the cash register by the time when the congestion is resolved as a correct answer label, thereby executing the simulation.

9. The information processing program according to claim 1, further comprising: causing the computer to execute a process of placing each of a plurality of agents corresponding to each of a plurality of people at a position in the virtual space corresponding to each of the plurality of people based on position information of the people present in a specified area of ​​the real world; the process of executing the simulation includes a process of selecting one first agent from the plurality of agents placed in the virtual space, and using the position of the first agent in the virtual space and the behavior selection model to determine whether or not the first agent will follow the first measure; and the process of selecting the first person includes a process of selecting a person corresponding to the first agent as the first person from the plurality of people present in the real world if it is determined that the first agent will follow the first measure; and determining whether or not the identified first person is currently present in the specified area; and if it is determined that the first person is present in the specified area, outputting information regarding the first measure to an information processing terminal used by the first person.

10. An information processing method characterized in that a computer executes the following processes: obtain a behavioral selection model that receives the output result of a policy and determines whether a target person will act in accordance with the policy; use the obtained behavioral selection model to run a simulation in a digital twin that determines whether an agent will follow a first policy; and select a first person in the real world to which the first policy will be applied based on the results of the simulation.

11. An information processing device having a control unit that executes the following processes: acquiring a behavior selection model that receives the output result of a policy and determines whether a target person will act in accordance with the policy; using the acquired behavior selection model, running a simulation in a digital twin to determine whether an agent will follow a first policy; and selecting a first person in the real world to which the first policy will be applied based on the results of the simulation.

Images