Programs, methods, information processing devices, systems

Abstract

To enhance the enjoyment of waiting in line. [Solution] This is a program to be executed by a computer equipped with a processor and memory. The program causes the processor to execute the following steps: a first area in which a queue of customers may form, a second area for generating augmented reality object images arranged at predetermined intervals, acquiring photographic data taken of the first area, detecting the second area based on the acquired photographic data, creating an object image corresponding to the detected second area, and displaying the created object image at a position corresponding to the second area.

Description

Technical Field

[0001] The present disclosure relates to a program, a method, an information processing apparatus, and a system.

Background Art

[0002] In recent years, technologies related to AR (Augmented Reality) and MR (Mixed Reality) have been developing (see Patent Document 1). AR and MR expand the real world that the user sees by adding information to the real space that the user is viewing.

Prior Art Documents

Patent Documents

[0003] [[ID=2l]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In places where people tend to gather, measures are taken everywhere to avoid crowding. For example, in a line waiting to pay, a sign indicating the position of customers is installed on the floor of the waiting space to ensure the distance between customers in the payment line. However, waiting in a line while maintaining a spaced-apart state lacks interest.

[0005] An object of the present disclosure is to enhance the interest when waiting in a line.

Means for Solving the Problems

[0006] This is a program to be executed by a computer equipped with a processor and memory. The program causes the processor to execute the following steps: a first area where a queue of customers may form, with multiple second areas for generating augmented reality object images arranged at predetermined intervals; acquiring photographic data of the first area; detecting the second area based on the acquired photographic data; creating an object image corresponding to the detected second area; and displaying the created object image at a position corresponding to the second area. [Effects of the Invention]

[0007] According to this disclosure, it is possible to enhance the enjoyment of waiting in line. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the overall configuration of System 1. [Figure 2] Figure 1 is a block diagram showing an example configuration of the terminal device 10. [Figure 3] This figure shows an example of the functional configuration of server 20. [Figure 4] This diagram shows the data structure of user information 181. [Figure 5] This diagram shows the data structure of object information 182. [Figure 6] This diagram shows the data structure of environmental information 183. [Figure 7] This is a schematic diagram representing the area around the cash register counter inside a store according to this embodiment. [Figure 8] This flowchart illustrates an example of how terminal device 10 generates object images within a column. [Figure 9] This flowchart shows a specific example of the operation of the display control unit 195 according to this embodiment when displaying an object image. [Figure 10] This is a schematic diagram showing an example of the display of the display 141 according to this embodiment. [Figure 11]It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 12] It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 13] It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 14] It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 15] It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 16] It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 17] It is a flowchart showing an example of the operation when the server 20 generates an object image in a column. [Figure 18] It is a schematic diagram showing a display example of the display 30 according to this embodiment. [Figure 19] It is a schematic diagram showing another example of the area around the cash register in the store according to this embodiment. [Figure 20] It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 21] It is a schematic diagram showing a display example of the display 141 according to this embodiment. [Figure 22] It is a block diagram showing the basic hardware configuration of the computer 90.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described while referring to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0010] <Schematic> The system according to this embodiment is constructed, for example, in the facilities where customers form a line. The system recognizes a predetermined area arranged among the customers in the line and generates an object image so that the customers can grasp that an object exists in the recognized area. As a result, the customers can feel as if they are lined up together with the object, and the time that should be boring waiting in line can be felt enjoyable.

[0011] <1 Configuration Diagram of the Whole System> FIG. 1 is a block diagram showing an example of the overall configuration of the system 1. The system 1 shown in FIG. 1 includes, for example, a terminal device 10, a server 20, a display 30, and a photographing device 41. The terminal device 10, the server 20, the display 30, and the photographing device 41 are communicatively connected via, for example, a network 80.

[0012] In FIG. 1, an example in which the system 1 includes one terminal device 10 is shown, but the number of terminal devices 10 included in the system 1 is not limited to one. The number of terminal devices 10 included in the system 1 may be one or more.

[0013] In FIG. 1, an example in which the system 1 includes one server 20 is shown, but the number of servers 20 included in the system 1 is not limited to one. The server 20 may be composed of a plurality of servers according to the functions it has. Also, the server 20 may be, for example, an aggregate of a plurality of devices regarded as one server. The way of distributing the plurality of functions required to implement the server 20 according to this embodiment for one or more hardware can be appropriately determined in view of the processing capabilities of each hardware and / or the specifications required for the server 20.

[0014] In FIG. 1, an example in which the system 1 includes one display 30 and one photographing device 41 each is shown, but the number of displays 30 and photographing devices 41 included in the system 1 is not limited to one each. The number of displays 30 and photographing devices 41 included in the system 1 may be one or more.

[0015] The terminal device 10 shown in Figure 1 is, for example, an information processing device operated by customers waiting in line. The terminal device 10 can be implemented as a mobile device such as a smartphone or tablet. The terminal device 10 may also be a wearable device such as an HMD (Head Mount Display).

[0016] The terminal device 10 comprises a communication interface 12, an input device 13, an output device 14, memory 15, storage 16, and a processor 19. The input device 13 is a device for receiving input operations from the user (e.g., a touch panel, touchpad, etc.). The output device 14 is a device for presenting information to the user (e.g., a display, speaker, etc.).

[0017] Server 20 is, for example, an information processing device that manages customer information.

[0018] The server 20 is implemented, for example, by a computer connected to the network 80. As shown in Figure 1, the server 20 includes a communication IF 22, an I / O IF 23, memory 25, storage 26, and a processor 29. The I / O IF 23 functions as an interface to an input device for receiving input operations from the user and an output device for presenting information to the user.

[0019] The display 30 displays data according to the control of the server 20. The display 30 is installed, for example, in a facility where rows are formed and displays images of the facility's status. The display 30 is implemented using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0020] The imaging device 41 is a device that receives light using a light-receiving element and outputs it as imaging data. The imaging device 41 is installed so that it can photograph a predetermined location within the facility at a predetermined angle of view. Specifically, the imaging device 41 is installed so that it can photograph a queue waiting for a predetermined process within the facility. The imaging device 41 outputs the imaging data to the server 20.

[0021] Within the facility, multiple areas for generating augmented reality object images are provided at predetermined intervals in areas where queues may form. Examples of areas for generating augmented reality object images are as follows: • Areas marked with a designated marker (floor, wall, designated object, etc.) • Objects with pre-defined shapes or colors.

[0022] A marker may be a code on which identification information is written. For example, a marker may be implemented as a barcode or QR code (registered trademark) on which identification information is written. The information written on the marker may also be address information for reading information about the object from the server 20. Markers are not limited to being attached in one location; multiple markers may be attached in a single area.

[0023] In areas where queues may form, markers may be placed to indicate customer positions, separate from the areas for generating object images. For example, markers may be placed on the floor in areas where queues may form, at predetermined intervals to indicate the formation of queues. If markers are placed on the floor, the areas for generating object images will be placed between the markers. In other words, the markers and areas will be arranged alternately.

[0024] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the terminal device 10 and the server 20, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0025] <1.1 Terminal Device Configuration> Figure 2 is a block diagram showing an example configuration of the terminal device 10 shown in Figure 1. As shown in Figure 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 160, a location information sensor 150, a storage unit 180, and a control unit 190. Each block included in the terminal device 10 is electrically connected, for example, by a bus.

[0026] The communication unit 120 performs processing such as modulation and demodulation processing for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to an external source (for example, the server 20). The communication unit 120 performs reception processing on the signal received from an external source and outputs it to the control unit 190.

[0027] The input device 13 is a device for a user operating the terminal device 10 to input instructions or information. The input device 13 can be implemented, for example, by a touch-sensitive device 131 on which instructions are input by touching the operating surface. If the terminal device 10 is a PC, the input device 13 may be implemented by a reader, keyboard, mouse, etc. The input device 13 converts the instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.

[0028] The output device 14 is a device for presenting information to the user operating the terminal device 10. The output device 14 is implemented, for example, by a display 141. The display 141 displays data according to the control of the control unit 190. The display 141 is implemented, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0029] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of the audio signal. The audio processing unit 17 converts the signal received from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is implemented, for example, by an audio processing processor. The microphone 171 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 17. The speaker 172 converts the audio signal received from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0030] Camera 160 is a device that receives light using a photodetector and outputs it as a shooting signal.

[0031] The location information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10, which is equipped with a GPS module, is detected based on the received signals. The location information sensor 150 may also detect the current position of the terminal device 10 from the position of the wireless base station to which the terminal device 10 is connected.

[0032] The storage unit 180 is implemented by, for example, memory 15 and storage 16, and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181, object information 182, and environment information 183.

[0033] User information 181 includes, for example, information about the user using terminal device 10. Details will be described later.

[0034] Object information 182 contains information about the objects that will appear in the area where the column may appear. Details will be described later.

[0035] Environmental information 183 includes information representing the current environment. This information includes, for example, information on weather, atmospheric pressure, temperature, humidity, etc. Further details will be provided later.

[0036] The control unit 190 is realized when the processor 19 reads a program stored in the memory unit 180 and executes instructions contained in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, a shooting control unit 193, an image analysis unit 194, and a display control unit 195.

[0037] The operation reception unit 191 processes instructions or information input from the input device 13. Specifically, for example, the operation reception unit 191 receives instructions or information input from a touch-sensitive device 131 or the like.

[0038] Furthermore, the operation reception unit 191 receives images input from the camera 160. Specifically, for example, the operation reception unit 191 receives the image data captured by the camera 160.

[0039] Furthermore, the operation reception unit 191 receives audio information input from the microphone 171. Specifically, for example, the operation reception unit 191 receives audio data input from the microphone 171 and converted into digital data by the audio processing unit 17.

[0040] The transmitting / receiving unit 192 performs processing to enable the terminal device 10 to send and receive data with an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 transmits instructions input by the user or various information acquired to the server 20. The transmitting / receiving unit 192 also receives information provided by the server 20.

[0041] The shooting control unit 193 controls the shooting by the camera 160. For example, the shooting control unit 193 starts shooting with the camera 160 in response to instructions from the user.

[0042] The image analysis unit 194 controls the analysis of the captured data captured by the camera 160. For example, the image analysis unit 194 detects a predetermined region included in the captured data. Specifically, for example, the image analysis unit 194 detects a region included in the captured data for generating an object image using augmented reality.

[0043] Furthermore, for example, the image analysis unit 194 detects customers who are standing in line in the captured data. The image analysis unit 194 detects customers who are standing in line if they are located at a predetermined distance in a predetermined direction from the area for generating the object image and remain there for a predetermined time or longer. If there are marks on the floor or elsewhere to indicate the location of customers, the image analysis unit 194 may also detect customers who are standing near those marks as customers in line. The image analysis unit 194 may also detect customers at the end of the line.

[0044] The image analysis unit 194 may use the detection results of customers in a queue when detecting areas for generating object images using augmented reality. For example, the image analysis unit 194 may detect customers in a queue and then detect areas near the detected customers. Specifically, for example, the image analysis unit 194 may detect an area located at a predetermined distance behind the customer at the end of the queue in the direction in which the queue was formed. Alternatively, for example, the image analysis unit 194 may detect areas located between customers in a queue.

[0045] Any existing technology may be used by the image analysis unit 194 to detect areas or customers in a line. The image analysis unit 194 may also use a trained model to detect areas or customers in a line.

[0046] The display control unit 195 controls the output device 14 to display a predetermined image to the user. For example, the display control unit 195 generates an object image based on information detected from the captured data and displays it on the display 141. Specifically, for example, the display control unit 195 generates an object image corresponding to the captured area and displays the object image at the position corresponding to that area. The display control unit 195 may generate different object images based on the following information, for example. • Past purchase history (purchase log) • A specified date (birthday, anniversary) • Number of past visits (visit log) • Favorite character • Calendar (days of the week, public holidays, etc.) ·weather

[0047] The display control unit 195, for example, displays the captured area on the display 141 and superimposes the created object image onto the area within the captured area that is used to generate augmented reality object images. In the case of a device that allows the user to view the actual situation through the display, such as an HMD, the display control unit 195 displays the object image at a position corresponding to the area used to generate augmented reality object images.

[0048] Furthermore, the display control unit 195 generates effects according to predetermined information. The predetermined information includes, for example, the following: • Detection of new customers joining the queue • Detection of the movement of customers in a queue. • Detection of a customer leaving a line. - Interference with the object image displayed on display 141 (touch of the object image displayed on display 141 via the touch-sensitive device 131, audio input to the object image displayed on display 141) • Past product purchase history • A specified date (birthday, anniversary) • Number of past visits Favorite character • Calendar (days of the week, public holidays, etc.) ·weather

[0049] The configuration of the terminal device 10 is not limited to the above. The terminal device 10 may include a 3D measuring unit. The 3D measuring unit calculates the 3D coordinates of the reflection points by, for example, irradiating a predetermined scanning area with a laser at a predetermined period and analyzing the reflected light. The 3D measuring unit is implemented, for example, by LiDAR (Light Detection And Ranging). The 3D measuring unit is installed, for example, so as to be able to measure in the same direction as the camera 160. The 3D measuring unit transmits the calculated 3D coordinate data as point cloud data to the control unit 190.

[0050] <1.2 Functional Configuration of the Server> Figure 3 shows an example of the functional configuration of server 20. As shown in Figure 3, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0051] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0052] The memory unit 202 includes, for example, a customer information table 2021, an object information table 2022, a location information table 2023, an environment information table 2024, and a trained model 2025.

[0053] Customer Information Table 2021 is a table that stores information about registered customers.

[0054] The object information table 2022 is a table that stores information about objects that may appear in the area where columns may be located.

[0055] The location information table 2023 is a table that stores the customer's current location. The customer's current location is set based on a 3D position measured by a 3D measuring device installed in the facility, for example. The 3D measuring device calculates the 3D coordinates of the reflection points by, for example, irradiating a predetermined scanning area with a laser at a predetermined period and analyzing the reflected light. The 3D measuring device is implemented by, for example, LiDAR. The 3D measuring device is installed in a position that allows for an unobstructed view of the entire facility. The 3D measuring device transmits the calculated 3D coordinate data as point cloud data to the server 20. When a customer visits the facility, they operate the terminal device 10 to send a notification of their visit to the server 20, and the registered location and the user's ID are associated. The server 20 determines the customer's current location by referring to the point cloud data and tracking the user's movements.

[0056] Environmental Information Table 2024 is a table that stores information related to the environment.

[0057] The trained model 2025 is, for example, a model for detecting a predetermined region from a captured image. The trained model 2025 is stored in advance when the server 20 provides the service. The trained model 2025 is generated by having a machine learning model perform machine learning according to a model training program. The trained model 2025 is, for example, a parameterized composite function composed of multiple functions. A parameterized composite function is defined by a combination of multiple tunable functions and parameters. The trained model 2025 in this embodiment may be any parameterized composite function that satisfies the above requirements.

[0058] For example, if the trained model 2025 is generated using a feedforward multilayer network, the parameterized composite function is defined as a combination of, for example, linear relationships between layers using weight matrices, nonlinear (or linear) relationships using activation functions in each layer, and biases. The weight matrices and biases are called the parameters of the multilayer network. The parameterized composite function changes its form depending on how the parameters are chosen. In a multilayer network, by appropriately setting the constituent parameters, it is possible to define a function that can output desirable results from the output layer.

[0059] As the multilayer network according to this embodiment, for example, a deep neural network (DNN), which is a multilayer neural network targeted by deep learning, may be used. As the DNN, for example, a convolutional neural network (CNN) that targets images may be used.

[0060] The pre-trained model 2025 is trained, for example, using an image containing a region where a queue may occur as input data, and the customers queuing in the queue as ground truth output data. The pre-trained model 2025 may be retrained as needed to accommodate variations in the customers queuing.

[0061] The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes instructions contained in the program. By operating according to the program, the control unit 203 performs functions as a receive control module 2031, a transmit control module 2032, an image analysis module 2033, and a display control module 2034.

[0062] The receive control module 2031 controls the process by which the server 20 receives signals from external devices according to a communication protocol.

[0063] The transmission control module 2032 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.

[0064] The image analysis module 2033 controls the analysis of the captured data captured by the imaging device 41. For example, the image analysis module 2033 detects a predetermined region included in the captured data. Specifically, for example, the image analysis module 2033 detects a region included in the captured data for generating an object image using augmented reality.

[0065] Furthermore, for example, the image analysis module 2033 detects customers who are standing in line in the captured data. The image analysis module 2033 detects customers who remain for a predetermined time or longer at a predetermined distance in a predetermined direction from the area for generating an object image as customers who are standing in line. In addition, if there are markers on the floor or elsewhere to specify the position of customers, the image analysis module 2033 may also detect customers who remain near those markers as customers who are standing in line.

[0066] The image analysis module 2033 may utilize the detection results of customers in a queue when detecting areas for generating object images using augmented reality. For example, the image analysis module 2033 may detect customers in a queue and then detect areas near the detected customers. Specifically, for example, the image analysis module 2033 may detect an area located at a predetermined distance behind the customer at the end of the queue in the direction in which the queue was formed. Alternatively, for example, the image analysis module 2033 may detect areas located between customers in a queue.

[0067] Any existing technology may be used by the image analysis module 2033 to detect regions or customers in a line. The image analysis module 2033 may also use a pre-trained model to detect regions or other objects.

[0068] The display control module 2034 controls the display 30 to display a predetermined image to the customer. For example, the display control module 2034 generates an object image based on information detected from the captured data and displays it on the display 30. Specifically, for example, the display control module 2034 generates an object image corresponding to the captured area and displays the object image at a position corresponding to that area. For example, a different object image may be generated based on predetermined information. The predetermined information is, for example, the same as the information described in the display control unit 195. For example, the display control module 2034 displays the captured area on the display 30 and superimposes the created object image onto the area included in the captured area for generating an augmented reality object image.

[0069] Furthermore, the display control module 2034 generates visual effects corresponding to predetermined information. This predetermined information is, for example, the same as the information described in the display control unit 195.

[0070] <2 Data Structure> Figures 4 to 6 show the data structure of the information stored by the terminal device 10. Note that Figures 4 to 6 are examples and do not exclude data that is not shown.

[0071] Figure 4 shows the data structure of user information 181. User information 181 shown in Figure 4 has columns such as name, age, address, date of birth, email address, registration date, visit information, purchase information, favorite character, and generated object, with user ID as the key.

[0072] The User ID is an item that stores an identifier to uniquely identify a user. The User ID may be the ID set when the user becomes a member of the store. The Name is an item that stores the user's name. The Age is an item that stores the user's age. The Address is an item that stores the user's place of residence. The Date of Birth is an item that stores the date the user was born. The Email Address is an item that stores the user's email address. The Registration Date is an item that stores the date the user registered as a member.

[0073] Visit information is an item that stores a user's history of visiting a facility. If a user has visited a facility multiple times, the visit information includes the date and time of each visit. If there are multiple related facilities, the visit information may store the date and time of the visit in association with the name of the facility visited. Purchase information is an item that stores a user's history of purchased items. Purchase information includes, for example, the date and time of purchase, the product ID of the purchased item, and the name of the purchased item. Favorite characters is an item that stores the object ID of characters that the user has registered as favorites.

[0074] Figure 5 shows the data structure of object information 182. The object information 182 shown in Figure 5 is a table that has columns such as object name, occurrence conditions, object data, performance content, and requirements corresponding to the performance content, with object ID as the key.

[0075] The Object ID is an item that stores an identifier to uniquely identify an object. The Object Name is an item that stores the name of the object. The Occurrence Condition is an item that stores the conditions for generating an object image. Conditions for generating an object image include, for example, the following: • You have previously purchased the specified product. • Having previously purchased a specified type of product. • The number of times the product has been purchased exceeds the specified number. • The purchase price of the goods exceeds the specified amount. • The current date matches the user's birthday. • The current date must coincide with a designated day (e.g., Christmas, Marine Day, year-end, New Year's Day, Coming of Age Day, etc.) • The current day of the week is a designated day of the week (for example, Friday). • The number of past visits exceeds the specified limit. • The configured object (character) matches your favorite character. • The weather is a specified type of weather (e.g., snow). • Identification information contained in the marker

[0076] If the marker contains identification information, the item "Conditions for Occurrence" may also store the identification number.

[0077] Object data is an item that stores image data about an object. Object data may also store reference information (paths) to object data files located elsewhere. Image data about an object may include, for example, the following: • Image data of the initial state of a given character • Image data of the evolved state of a specified character • Image data of a designated character wearing a special outfit Image data containing multiple instances of the specified character.

[0078] The "Performance Details" field stores image data related to the performance actions performed by the object. The "Performance Details" field may also store reference information (paths) to image data files located elsewhere. Examples of "Performance Details" include: • An effect where an object performs a predetermined action (an effect celebrating a birthday or a specific day). - A performance where customers join the line behind newly added customers. • A performance that moves forward in accordance with the customers in line. - A performance where the character steps out of the line • An effect where objects evolve. • An effect in which an object wears a specific accessory.

[0079] In the example shown in Figure 5, the performance content includes the first performance, the second performance, etc. The requirements are items that store the requirements for the object to perform the performance. The requirements include, for example, the following: • Detection of new customers joining the queue • Detection of the movement of customers in a queue. • Detection of a customer leaving a line. • Touch of an object image displayed on display 141 via touch-sensitive device 131 • Designated voice input • You have previously purchased the specified product. • Having previously purchased a specified type of product. • The number of times the product has been purchased exceeds the specified number. • The purchase price of the goods exceeds the specified amount. • The current date matches the user's birthday. • The current date must coincide with a designated day (e.g., Christmas, Marine Day, year-end, New Year's Day, Coming of Age Day, etc.) • The current day of the week is a designated day of the week (for example, Friday). • The number of past visits exceeds the specified limit. • The configured object (character) matches your favorite character. • The weather is a specified type of weather (e.g., snow).

[0080] The example shown in Figure 5 includes the first requirement for carrying out the first performance, the second requirement for carrying out the second performance, and so on.

[0081] Figure 6 shows the data structure of environmental information 183. The environmental information 183 shown in Figure 6 is a table having columns for date, weather, atmospheric pressure, temperature, and humidity. Environmental information 183 may also include calendar information. Calendar information may include, for example, information about the day of the week and information about public holidays.

[0082] The date field stores the date on which environmental information was obtained. Weather, atmospheric pressure, temperature, and humidity are examples of environmental information, and the system stores information provided by a designated source.

[0083] <3 operations> This section describes the operation of the terminal device 10 and the server 20 when displaying object images to customers to add interest to the act of waiting in line. (Display of object image by terminal device 10)

[0084] The following explanation assumes, for example, that a customer visits a designated store, places the items they wish to purchase in their shopping basket, and waits at the checkout counter to pay for their items. It is also assumed that the terminal device 10 used by the customer is a smartphone, tablet, or other device with a display.

[0085] Figure 7 is a schematic diagram showing the area around the cash register counter in a store according to this embodiment. In Figure 7, the store floor is marked with markers 51 for customers waiting in line to pay at the cash register counter. Markers 51 can also be described as markers for forming a queue. Markers 51 may be physical, such as stickers or paint, or optical, with a predetermined shape projected onto them. Multiple markers 51 are placed at intervals such that droplets are less likely to scatter when people are having a normal conversation, for example, at intervals of about 2m. Markers 51 may be arranged in a straight line or in a curved line.

[0086] Markers 52 are placed between the marks 51. The markers 52 are positioned at a predetermined distance and in a predetermined direction relative to the marks 51. The markers 52 may be physical, such as a sticker or paint, or they may be optical, projecting a predetermined shape. Note that the marks 51 do not necessarily have to be placed if markers 52 are present.

[0087] Figure 8 is a flowchart illustrating an example of the operation when terminal device 10 generates object images within a column.

[0088] First, the user operates the terminal device 10 to take a picture of the area around the cash register counter.

[0089] In step S11, the control unit 190 controls the camera 160 via the shooting control unit 193 to capture video of an area where a queue may form, for example, the area around the cash register counter.

[0090] In step S12, the control unit 190 analyzes the captured image data using the image analysis unit 194. For example, the image analysis unit 194 detects markers 52, which are regions for generating augmented reality object images, within the captured data. Also, if customers are forming a line in front of the cash register, the image analysis unit 194 detects the customers in the line within the captured data.

[0091] In step S13, the control unit 190 causes the display control unit 195 to display an image on the display 141. For example, the display control unit 195 generates an object image based on information detected from the captured data and displays the generated object image on the display 141 along with the captured data.

[0092] (Specific example of object image display) Figure 9 is a flowchart illustrating a specific example of the operation of the display control unit 195 according to this embodiment when displaying an object image.

[0093] In step S21, the display control unit 195 determines whether a marker 52 located immediately after the customer at the end of the line, or a marker 52 located between the customers in the line, has been detected. If a marker 52 located immediately after the customer at the end of the line, or a marker 52 located between the customers in the line, has been detected (Yes in step S21), the display control unit 195 proceeds to step S22. If a marker 52 located immediately after the customer at the end of the line, or a marker 52 located between the customers in the line, has not been detected (No in step S21), the display control unit 195 proceeds to step S24.

[0094] In step S22, the display control unit 195 creates an object image for each marker 52 detected in step S21. Specifically, the display control unit 195 obtains information related to the item "occurrence conditions" of the object information 182 from the user information 181 and the environment information 183. Based on the obtained information, the display control unit 195 selects a record that satisfies the occurrence conditions. The display control unit 195 obtains image data based on the data stored in the item "object data" of the selected record. If there are multiple objects that satisfy the occurrence conditions, the display control unit 195 selects a record based on, for example, a pre-set priority. The display control unit 195 may select a different object for each detected marker 52, or it may select the same object for each detected marker 52.

[0095] In step S23, the display control unit 195 displays the object image on the display 141. Specifically, the display control unit 195 superimposes the object image based on the acquired image data onto the marker 52 detected in step S21. As a result, the object image is displayed on the marker 52 located immediately behind the customer at the end of the line and on the marker 52 located between the customers in the line.

[0096] Figure 10 is a schematic diagram showing an example of the display 141 according to this embodiment. In Figure 10, the image analysis unit 194 detects the customer 100, who is staying within the mark 51, as a customer in a queue. The image analysis unit 194 then detects the marker 52 located immediately behind the customer 100. The display control unit 195 superimposes the object image 53 onto the detected marker 52. In the example shown in Figure 10, a character standing in line with the customer and facing away is displayed as the object image 53. In other words, the object image is displayed behind the customer.

[0097] In step S24, the display control unit 195 determines whether or not an event has occurred that satisfies the requirements for the occurrence of a special effect. Specifically, for example, the display control unit 195 determines whether or not an event has occurred that satisfies the item "Requirements" in the object information 182. If an event has occurred that satisfies the requirements for the occurrence of a special effect, the display control unit 195 proceeds to step S25.

[0098] In step S25, the display control unit 195 generates an effect corresponding to the event. Specifically, for example, the display control unit 195 generates an effect stored in the item "Effect" that corresponds to the requirement stored in the item "Requirement".

[0099] Specifically, for example, in step S24, the display control unit 195 determines that an event satisfying the requirements for the performance to occur has been detected when a new customer has joined the queue. Then, in step S25, the display control unit 195 acquires image data as shown in step S22 and moves the object image based on the acquired image data toward the marker 52.

[0100] Figures 11 and 12 are schematic diagrams showing an example of the display 141 according to this embodiment. In Figures 11 and 12, the image analysis unit 194 detects a customer 100 that has moved into the mark 51. The image analysis unit 194 then detects a marker 52 located immediately after the customer 100. The display control unit 195 moves the object image 53 to the detected marker 52 so that it is added to the row. The display control unit 195 may also cause the object image 53, after it has finished moving, to perform an effect such as a greeting.

[0101] As shown in Figure 12, when a new object image is displayed in addition to the already displayed object image, the newly displayed object image may be the same character as the already displayed object image, or it may be a different character. Figure 12 shows an example where it is the same character.

[0102] Furthermore, for example, in step S24, the display control unit 195 determines that an event satisfying the requirements for generating an effect has occurred if the user touches the object image displayed on the display 141 via the touch-sensitive device 131. Then, in step S25, the display control unit 195 displays a predetermined effect for the touched object image.

[0103] Figures 13 and 14 are schematic diagrams showing an example of the display 141 according to this embodiment. In Figure 13, the displayed object image 53 is touched by the user. The hand in Figure 13 schematically represents the user's contact with the object image 53. As shown in Figure 14, the display control unit 195 causes the object image 53 to turn around and strike a pose. Note that the action in response to user interference is not limited to a predetermined pose, but may also be a predetermined action or a predetermined technique.

[0104] Furthermore, for example, in step S24, the display control unit 195 determines that an event has occurred that satisfies the requirements for the performance to occur if, while displaying an object image at the end of the line, a customer in the line moves forward to the marker 51 on the cash register side. Then, in step S25, the display control unit 195 moves the displayed object image toward the next marker 52 in the direction the customer moved.

[0105] Figure 15 is a schematic diagram showing an example of the display 141 according to this embodiment. For example, suppose that in the state shown in Figure 13, the first customer has finished paying, and as shown in Figure 15, customer 100 in the line has moved forward. The image analysis unit 194 detects that customer 100 has moved to the marker 51 one position further towards the cash register counter. The image analysis unit 194 then detects the marker 52 located immediately behind the moved customer 100. The display control unit 195 moves the object image 53 from the marker 52 on which the object image 53 was superimposed in Figure 13 to the marker 52 one position further towards the cash register counter.

[0106] Furthermore, for example, in step S24, the display control unit 195 determines that an event has occurred that satisfies the requirements for triggering a performance if the customer at the end of the line leaves the line while an object image is being displayed at the end of the line. Then, in step S25, the display control unit 195 moves the displayed object image so that it leaves the line.

[0107] Figure 16 is a schematic diagram showing an example of the display of the display 141 according to this embodiment. For example, suppose that in the state shown in Figure 13, the last customer 100 in the line leaves the line as shown in Figure 16. The image analysis unit 194 detects the customer 100 that has left the line and moved away from the marker 51. The display control unit 195 moves the object image 53 from the marker 52 on which the object image 53 was superimposed in Figure 13 in the direction away from the line.

[0108] (Display of object images by Server 20) Next, we will explain the case where the inside of the store is photographed by a camera 41 installed inside the store and displayed on the display 30. In the following explanation, we will assume a situation in which a customer who has come to a designated store places the items they wish to purchase into a shopping basket and waits at the cash register to pay for the items.

[0109] Figure 17 is a flowchart illustrating an example of how server 20 generates object images within a column.

[0110] First, the camera 41 installed inside the store takes video of the area around the cash register counter.

[0111] In step S31, the control unit 203 acquires the image data captured by the imaging device 41 using the receiving control module 2031.

[0112] In step S32, the control unit 203 analyzes the captured image data using the image analysis module 2033. For example, the image analysis module 2033 detects markers 52, which are regions in the captured data used to generate augmented reality object images. If customers are lined up in front of the cash register, the image analysis module 2033 also detects the customers in line in the captured data.

[0113] In step S33, the control unit 203 causes the display control module 2034 to display an image on the display 30. For example, the display control module 2034 generates an object image based on information detected from the captured data and displays the generated object image on the display 30 together with the captured data.

[0114] Figure 18 is a schematic diagram showing an example of the display 30 according to this embodiment. In Figure 18, the image analysis module 2033 detects customers who remain within the mark 51 as customers in a queue. The image analysis module 2033 then detects a marker 52 located immediately after the customer at the end of the queue and another marker 52 located between the customers in the queue. The display control module 2034 superimposes an object image 53 onto the detected markers 52. In the example shown in Figure 18, a character standing in line with the customers is displayed as the object image 53.

[0115] As described above, in the above embodiment, multiple second regions for generating augmented reality object images are arranged at predetermined intervals in a first region where a queue may be formed. The control unit 190 acquires captured image data of the first region. The control unit 190 detects the second region based on the acquired image data using the image analysis unit 194. The control unit 190 creates an object image corresponding to the detected second region using the display control unit 195. The display control unit 195 displays the created object image at a position corresponding to the second region. As a result, the terminal device 10 or server 20 according to this embodiment can generate augmented reality objects in the queue, making the waiting experience more enjoyable for customers waiting in line.

[0116] Therefore, according to this embodiment, the enjoyment of waiting in line can be enhanced.

[0117] Furthermore, in the above embodiment, the display control unit 195 displays an image based on the captured data and superimposes the object image onto a position corresponding to the second region in the image. This allows the user to easily view the object image generated within the row by holding a smartphone or tablet device over an area where a row may be formed.

[0118] Furthermore, in the above embodiment, the image analysis unit 194 detects people standing in a line and uses the person detection result to detect a second region. The display control unit 195 displays an object image at a position corresponding to the second region behind the people in the direction in which the line is formed. As a result, an object image appears behind the people standing in line. In other words, it appears as if people and objects are lined up alternately. It becomes possible to see a line that does not actually exist, and the user can enjoy the act of waiting in line more. Also, if there are multiple people in line, multiple objects will be generated, and the user can enjoy the act of waiting in line even more.

[0119] Furthermore, in the above embodiment, the image analysis unit 194 detects the person at the end of the line and uses the detection result of the person at the end to detect the second area. The display control unit 195 displays the object image behind the person at the end of the line. As a result, the object is placed at the end of the line, and the user lines up after the object. By making the object feel closer, the user can enjoy the act of lining up more.

[0120] Furthermore, in the above embodiment, the display control unit 195 creates object images based on user information. This makes it possible to generate different objects each time the user visits the store. As a result, the user's enjoyment of visiting the store increases, and it motivates them to visit again. User information includes, for example, visit logs, purchase logs, special days, or information about favorites.

[0121] Furthermore, in the above embodiment, the display control unit 195 creates object images based on environmental information. This makes it possible to generate different objects depending on the environment. As a result, the user's enjoyment of visiting the store increases, and it becomes a motivation for them to visit again. Environmental information includes, for example, calendar or weather information.

[0122] Furthermore, in the above embodiment, the display control unit 195 determines whether or not the requirements for generating an effect are met. The display control unit 195 causes the object image to perform an effect according to the requirements that have been met. As a result, the object performs an effect according to the requirements, allowing the user to enjoy the arranging motion more.

[0123] Furthermore, in the above embodiment, the display control unit 195 detects a person who has joined the queue. The display control unit 195 causes the object image to perform the animation of joining the queue. As a result, the number of objects increases when a new customer joins the queue, further enhancing the entertainment value.

[0124] Furthermore, in the above embodiment, the display control unit 195 detects people moving forward along the line. The display control unit 195 causes the object image to perform an animation of moving along the line. As a result, the object moves in a distinctive manner as the line moves forward, allowing the user to look forward to the line moving forward.

[0125] Furthermore, in the above embodiment, the display control unit 195 detects a person who has left the line. The display control unit 195 causes the object image to perform an animation of leaving the line. As a result, the object behaves distinctively when a customer leaves the line, allowing the user to look forward to the customer leaving the line.

[0126] Furthermore, in the above embodiment, the display control unit 195 detects interference with the object image. The display control unit 195 causes the object image to perform an action in response to the interference. As a result, even if there is no change in the queue, the object will move due to user interference, allowing the user to enjoy the act of waiting even when there is no change in the queue.

[0127] <Variation> In the above embodiment, when a predetermined condition is met, an object image corresponding to that condition is displayed, and when a predetermined requirement is met, the object image performs an effect corresponding to the requirement. At this time, it is not displayed what conditions have been met for the object image to be displayed, or what requirements have been met for the object image to perform the predetermined effect.

[0128] However, it may be specified what conditions must be met for the object image to be displayed. It may also be specified what requirements must be met for the object image to perform a specific effect. For example, a statement such as "Occurred because the 'specific action' was completed!" may be displayed in a manner that does not interfere with the display of the object image, such as a temporary pop-up. This makes it possible to understand the conditions for the occurrence of a characteristic object or the requirements for the occurrence of a characteristic effect, and allows the user to consciously reproduce the characteristic object or characteristic effect. Therefore, the user can adjust their actions to produce the object or effect they desire.

[0129] Furthermore, the above embodiment does not include any indication of the object to be displayed next. However, indications of the object to be displayed next may be included. For example, a statement such as "When you perform the 'predetermined action' next, the 'predetermined object or predetermined effect' will occur!" may be displayed in a manner that does not interfere with the display of the object image, for example, as a temporary pop-up. The expression may also be one that further conceals the occurrence. This will allow the user to understand what object or effect may occur on the next occasion, making their next visit more enjoyable.

[0130] Furthermore, in the above embodiment, the area for generating object images may be installed on the cash register counter. The marker, which is the area placed on the cash register counter, may include a predetermined code. The predetermined code is, for example, a code individually assigned to the cashier, and is a code for generating an object assigned to that cashier. As a result, a cashier-specific object will be generated at the front of the line, allowing users to look forward to seeing what their cashier object will be as they wait in line. In other words, the user's enjoyment of waiting in line will increase.

[0131] Furthermore, in the above embodiment, the case in which the image analysis unit 194 or the image analysis module 2033 detects customers in a queue was described as an example. However, the detection of customers in a queue is not limited to the image analysis unit 194 or the image analysis module 2033. The control unit 190 may function as a queue state recognition unit by operating according to a program. The control unit 203 may also function as a queue state recognition module by operating according to a program. The queue state recognition unit detects customers in a queue based on point cloud data measured by, for example, the 3D measurement unit. The queue state recognition module also detects customers in a queue based on point cloud data measured by, for example, the 3D measurement module.

[0132] Any existing technology may be used when the queue state recognition unit or queue state recognition module detects customers or others standing in a queue.

[0133] Furthermore, in the above embodiment, the example given was that another customer is waiting in line behind the customer who is paying, as shown in Figure 10. However, the character may be positioned immediately after the customer who is paying. That is, for example, as shown in Figure 19, an area for generating an augmented reality object image may be provided between the cash register and the position where the customer at the front of the line is waiting. This would result in an object image being generated between the customer paying and the customer waiting at the front of the line, for example, when a customer is paying at the cash register. Also, as shown in Figure 20, a situation may occur where only the object image is lined up behind the customer paying. This makes it easy for customers to recognize where the line begins.

[0134] Furthermore, for example, when a customer completes payment at the cash register and moves away from the register, the object image may guide the customer at the front of the line. Specifically, for example, when a customer completes payment at the cash register and moves away from the register, the image analysis unit 194 or image analysis module 2033 detects that the customer has left the cash register. The display control unit 195 or display control module 2034 guides the customer at the front of the line to the object image superimposed on the marker immediately in front of the cash register. Figure 21 is a schematic diagram showing an example of the display of the display 141 according to this embodiment.

[0135] <4. Basic Hardware Configuration of a Computer> Figure 22 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 includes at least a processor 91, main memory 92, auxiliary storage 93, and a communication interface 99. These are electrically connected to each other by a bus.

[0136] The processor 91 is hardware for executing the instruction set written in the program. The processor 91 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0137] Main memory 92 is used to temporarily store programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0138] Auxiliary storage device 93 is a storage device for storing data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc.

[0139] A communication IF99 is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards.

[0140] A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0141] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0142] <Basic Functional Configuration of Computer 90> The functional configuration of the computer realized by the basic hardware configuration of computer 90 shown in Figure 22 will be explained. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0143] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0144] The control unit is realized when the processor 91 reads various programs stored in the auxiliary storage device 93, loads them into the main memory device 92, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0145] The memory unit is implemented by a main memory 92 and an auxiliary memory 93. The memory unit stores data, various programs, and various databases. The processor 91 can also reserve memory areas corresponding to the memory unit in the main memory 92 or the auxiliary memory 93 according to the program. The control unit can also cause the processor 91 to perform operations such as adding, updating, and deleting data stored in the memory unit according to the various programs.

[0146] A database, specifically a relational database, is used to manage and link data sets called tables, which are structurally defined by rows and columns. In a database, tables are called tables, the columns of a table are called columns, and the rows of a table are called records. In a relational database, relationships can be established and linked between tables.

[0147] Typically, each table has a key column to uniquely identify records, but setting a key on a column is not mandatory. The control unit can instruct the processor 91 to add, delete, or update records in specific tables stored in the memory unit according to various programs.

[0148] The communication unit is implemented by the communication IF99. The communication unit implements the function of communicating with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 91 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0149] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0150] <Note> The details described in each of the above embodiments are noted below. (Note 1) A program for execution on a computer having a processor and memory, wherein the program causes the processor to execute the following steps: a step of acquiring photographic data taken of the first region, a step of detecting the second region based on the acquired photographic data, a step of creating an object image corresponding to the detected second region, and a step of displaying the created object image at a position corresponding to the second region. (Note 2) The program described in (Appendix 1) displays an image based on the captured data in the step of displaying an object image, and superimposes the object image onto the position corresponding to the second region in the image. (Note 3) The program described in (Appendix 1) or (Appendix 2), wherein in the detection step, customers lined up in a queue are detected, the second region is detected using the customer detection results, and in the step of displaying an object image, the object image is displayed for the customer at a position corresponding to the second region behind in the direction in which the queue is formed. (Note 4) The program described in Appendix 3, which, in the detection step, detects the customer at the end of the line, uses the detection result of the last customer to detect a second area, and in the step of displaying an object image, displays the object image behind the last customer. (Note 5) A program that, in the step of creating an object image, creates an object image based on user information, as described in any of (Appendix 1) to (Appendix 4). (Note 6) The program described in (Appendix 5) creates an object image based on information about visit logs, purchase logs, special days, or favorites included in the user information during the step of creating the object image. (Note 7) A program that, in the step of creating an object image, creates an object image based on environmental information, as described in any of (Appendix 1) to (Appendix 4). (Note 8) The program described in (Appendix 7) creates an object image based on calendar or weather information included in the environmental information during the step of creating the object image. (Note 9) A program as described in any of (Appendix 1) to (Appendix 8), which causes the processor to perform a step of determining whether or not the requirements for an effect to occur are met, and in the step of displaying an object image, causes the object image to perform an effect according to the requirements that have been met. (Note 10) The program described in Appendix 9, which, in the step of making a decision, detects a customer who has joined the queue, and in the step of displaying an object image, causes the object image to perform the animation of joining the queue. (Note 11) The program described in Appendix 9, which, in the step of making a judgment, detects customers moving forward along the line, and in the step of displaying an object image, causes the object image to perform an animation of moving along the line. (Note 12) The program described in Appendix 9, which, in the step of making a judgment, detects a customer who has left the line and, in the step of displaying an object image, causes the object image to perform an animation of leaving the line. (Note 13) The program described in (Appendix 9) includes a step to detect interference with an object image in the decision-making step, and a step to display the object image in which the object image is made to perform an action corresponding to the interference. (Note 14) A method to be performed on a computer having a processor and memory, wherein the processor has a first area where a queue of customers may form, and a plurality of second areas for generating augmented reality object images are arranged at predetermined intervals in that area, and the method includes the steps of: acquiring photographic data taken of the first area; detecting the second area based on the acquired photographic data; creating an object image corresponding to the detected second area; and displaying the created object image at a position corresponding to the second area. (Note 15) An information processing device comprising a control unit and a storage unit, wherein the control unit has multiple second areas for generating augmented reality object images arranged at predetermined intervals in a first area where a queue of customers may form, and the information processing device performs the steps of acquiring photographic data taken of the first area, detecting the second area based on the acquired photographic data, creating an object image corresponding to the detected second area, and displaying the created object image at a position corresponding to the second area. (Note 16) A system comprising a first area where queues of customers may form, and a second area within the first area, which is arranged at predetermined intervals to generate augmented reality object images. (Note 17) The system described in Appendix 16 includes a marker in the first area indicating the stopping position of customers, separate from the second area. (Note 18) The system described in (Appendix 17), wherein the marks and the second region are arranged alternately in the first region. (Note 19) A system according to any one of the (Appendix 16) to (Appendix 18) comprising: means for acquiring photographic data of a first region; means for detecting a second region based on the acquired photographic data; means for creating an object image corresponding to the detected second region; and means for displaying the created object image at a position corresponding to the second region. [Explanation of symbols]

[0151] 1... System 10…Terminal device 12…Communication IF 120... Communications Department 13…Input device 131…Touch-sensitive devices 14…Output device 141…Display 15…Memory 150…Location information sensor 16…Storage 160... Camera 17…Sound Processing Unit 171... Mike 172...Speaker 180...Storage section 19… Processor 190... Control Unit 20... Server

Claims

1. A program to be executed by a computer having a processor and memory, wherein the program is to be executed by the processor, In a first area where a queue of customers may form, multiple second areas for generating object images using augmented reality are arranged at predetermined intervals, and the process involves acquiring photographic data captured for the first area. The steps include detecting the second region based on the acquired imaging data, The steps include creating an object image corresponding to the detected second region, The steps include displaying the created object image at a position corresponding to the second region, and A program that executes the command.

2. The program according to claim 1, wherein in the step of displaying the object image, an image based on the captured data is displayed, and the object image is superimposed and displayed at a position corresponding to the second region in the image.

3. In the detection step described above, customers lined up in a queue are detected, and the second area is detected using the customer detection results. The program according to claim 1 or 2, wherein in the step of displaying the object image, the object image is displayed to the customer at a position corresponding to the second area behind in the direction in which the queue is formed.

4. In the detection step described above, the customer at the end of the line is detected, and the second area is detected using the detection result of the customer at the end of the line. The program according to claim 3, wherein, in the step of displaying the object image, the object image is displayed behind the last customer in line.

5. The program according to any one of claims 1 to 4, wherein in the step of creating the object image, the program creates the object image based on user information.

6. The program according to claim 5, wherein in the step of creating the object image, the program creates the object image based on information about visit logs, purchase logs, special days, or favorites included in the user information.

7. The program according to any one of claims 1 to 4, wherein in the step of creating the object image, the program creates the object image based on environmental information.

8. The program according to claim 7, wherein in the step of creating the object image, the program creates the object image based on calendar or weather information included in the environmental information.

9. The processor is instructed to perform a step of determining whether or not the requirements for the occurrence of the effect are met. The program according to any one of claims 1 to 8, wherein in the step of displaying the object image, the program causes the object image to perform an effect according to the requirements that have been met.

10. In the aforementioned decision-making step, the customer who has joined the queue is detected, The program according to claim 9, wherein, in the step of displaying the object image, the object image is made to perform an effect that adds to the row.

11. In the aforementioned decision-making step, the customer moving forward along the line is detected, The program according to claim 9, wherein, in the step of displaying the object image, the object image is made to perform an animation of moving along a row.

12. In the aforementioned decision-making step, a customer who has left the line is detected, The program according to claim 9, wherein, in the step of displaying the object image, the object image is made to perform an effect of moving out of the row.

13. In the aforementioned determination step, interference with the object image is detected, The program according to claim 9, wherein in the step of displaying the object image, the program causes the object image to perform an action in response to interference.

14. A method to be performed on a computer comprising a processor and memory, wherein the processor In a first area where a queue of customers may form, multiple second areas for generating object images using augmented reality are arranged at predetermined intervals, and the process involves acquiring photographic data captured for the first area. The steps include detecting the second region based on the acquired imaging data, The steps include creating an object image corresponding to the detected second region, The steps include displaying the created object image at a position corresponding to the second region, and How to do it.

15. An information processing apparatus comprising a control unit and a storage unit, wherein the control unit is In a first area where a queue of customers may form, multiple second areas for generating object images using augmented reality are arranged at predetermined intervals, and the process involves acquiring photographic data captured for the first area. The steps include detecting the second region based on the acquired imaging data, The steps include creating an object image corresponding to the detected second region, The steps include displaying the created object image at a position corresponding to the second region, and An information processing device that performs the following actions.

16. The first area where a queue of customers may form, A second region for generating augmented reality object images, which is arranged at predetermined intervals within the first region. A system that is equipped with [the following].

17. The system according to claim 16, wherein a mark indicating the position where a customer stops is arranged separately in the first area from the second area.

18. The system according to claim 17, wherein the marks and the second region are arranged alternately in the first region.

19. Means for acquiring imaging data captured for the first region, A means for detecting the second region based on the acquired imaging data, Means for creating an object image corresponding to the detected second region, Means for displaying the created object image at a position corresponding to the second region The system according to any one of claims 16 to 18, comprising the above.

Images