Program, method, information processing device, and system

Abstract

To increase visitors' interest when the visitors wait in line.SOLUTION: A program is to be executed by a computer comprising a processor and a memory. The program causes the processor to execute the steps of: acquiring photographic data obtained by photographing a first area where a line may be formed by visitors and a plurality of second areas for generating object images by augmented reality is arranged at every predetermined interval; detecting the second areas based on the acquired photographic data; creating the object images corresponding to the detected second areas; and displaying the created object images at positions corresponding to the second areas.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] In recent years, technologies related to AR (Augmented Reality) and MR (Mixed Reality) have been developing (see Patent Document 1). AR and MR extend the real world seen by a user by adding information to the real space that the user is viewing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2016-506565 Summary of the Invention [Problem to be solved by the invention]

[0004] In places where people tend to gather, measures to avoid crowding are being implemented everywhere. For example, in queues to check out, signs indicating the location of customers are placed on the floor of the waiting area to ensure distance between customers in the queue. However, waiting in line while maintaining a distance is not very interesting.

[0005] An object of the present disclosure is to make waiting in line more interesting. [Means for solving the problem]

[0006] A program to be executed by a computer having a processor and a memory. The program causes the processor to execute the following steps: acquiring image data of a first area where customers may queue, where a plurality of second areas for generating augmented reality object images are arranged at predetermined intervals; detecting the second area based on the acquired image data; creating an object image corresponding to the detected second area; and displaying the created object image at a position corresponding to the second area. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to increase the enjoyment of waiting in line. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the overall configuration of a system 1. FIG. [Figure 2] 2 is a block diagram illustrating an example of the configuration of a terminal device 10 shown in FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of the functional configuration of a server 20. [Figure 4] FIG. 10 is a diagram showing the data structure of user information 181. [Figure 5] FIG. 10 is a diagram showing the data structure of object information 182. [Figure 6] FIG. 10 is a diagram showing the data structure of environment information 183. [Figure 7] FIG. 2 is a schematic diagram illustrating an area around a checkout counter in a store according to the present embodiment. [Figure 8] 10 is a flowchart illustrating an example of an operation performed by the terminal device 10 when generating an object image in a column. [Figure 9] 10 is a flowchart showing a specific example of the operation of the display control unit 195 according to the present embodiment when displaying an object image. [Figure 10] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 11]3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 12] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 13] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 14] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 15] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 16] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 17] 10 is a flowchart showing an example of the operation of the server 20 when generating an object image in a sequence. [Figure 18] 3A and 3B are schematic diagrams illustrating an example of a display on a display 30 according to the present embodiment. [Figure 19] 10A and 10B are schematic diagrams illustrating other examples of areas around the checkout counters in a store according to the present embodiment. [Figure 20] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 21] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 22] FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of the components are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0010] <Summary> The system according to this embodiment is implemented, for example, in a facility where customers queue up. The system recognizes a predetermined area placed between the customers in line and generates an object image so that the customers can understand that an object is present in the recognized area. This allows the customers to feel as if they are queuing with the object, making the time spent waiting in line, which would otherwise be boring, more enjoyable.

[0011] <1 Overall system configuration> Fig. 1 is a block diagram showing an example of the overall configuration of a system 1. The system 1 shown in Fig. 1 includes, for example, a terminal device 10, a server 20, a display 30, and an image capturing device 41. The terminal device 10, the server 20, the display 30, and the image capturing device 41 are communicatively connected via, for example, a network 80.

[0012] 1 shows an example in which the system 1 includes one terminal device 10, but the number of terminal devices 10 included in the system 1 is not limited to one. The number of terminal devices 10 included in the system 1 may be one or more.

[0013] 1 shows an example in which the system 1 includes one server 20, but the number of servers 20 included in the system 1 is not limited to one. The server 20 may be composed of multiple servers depending on the functions it has. Furthermore, the server 20 may be, for example, a collection of multiple devices. The way in which the multiple functions required to realize the server 20 according to this embodiment are allocated to one or multiple pieces of hardware can be determined appropriately in consideration of the processing capacity of each piece of hardware and / or the specifications required for the server 20.

[0014] 1 shows an example in which the system 1 includes one display 30 and one image capturing device 41, but the number of displays 30 and image capturing devices 41 included in the system 1 is not limited to one. The number of displays 30 and image capturing devices 41 included in the system 1 may be one or more.

[0015] 1 is, for example, an information processing device operated by a customer waiting in line. The terminal device 10 is realized by, for example, a mobile terminal such as a smartphone or a tablet. The terminal device 10 may also be a wearable terminal such as an HMD (Head Mount Display).

[0016] The terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19. The input device 13 is a device (e.g., a touch panel, a touch pad, etc.) for receiving input operations from a user. The output device 14 is a device (e.g., a display, a speaker, etc.) for presenting information to a user.

[0017] The server 20 is, for example, an information processing device that manages information about customers and the like.

[0018] The server 20 is realized by, for example, a computer connected to a network 80. As shown in Fig. 1, the server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29. The input / output IF 23 functions as an input device for receiving input operations from a user and as an interface with an output device for presenting information to the user.

[0019] Display 30 displays data according to the control of server 20. Display 30 is installed, for example, in a facility where a queue is formed, and displays an image of the facility's situation. Display 30 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0020] The image capturing device 41 is a device that receives light using a light receiving element and outputs the light as image capturing data. The image capturing device 41 is installed, for example, so that it can capture an image of a predetermined position within a facility at a predetermined angle of view. Specifically, the image capturing device 41 is installed, for example, so that it can capture an image of a line of people waiting for a predetermined process within the facility. The image capturing device 41 outputs the image capturing data to the server 20.

[0021] Within the facility, in an area where a queue may occur, a plurality of areas for generating augmented reality object images are provided at predetermined intervals. Examples of the areas for generating augmented reality object images are as follows: Areas with designated markers (floors, walls, designated objects, etc.) - Objects with preset shapes or colors

[0022] The marker may be a code in which identification information is written. For example, the marker may be realized by a barcode in which identification information is written, a QR code (registered trademark), or the like. Furthermore, the information written in the marker may be address information for reading information about the object from the server 20. The marker is not limited to being attached in one location, and multiple markers may be attached in one area.

[0023] In the area where a queue may be generated, marks for specifying the positions of customers may be provided separately from the area for generating object images. For example, marks for forming queues at predetermined intervals may be provided on the floor of the area where a queue may be generated. When marks are provided on the floor, the areas for generating object images are arranged between the marks. In other words, the marks and the areas are arranged alternately.

[0024] Each information processing device is configured by a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the terminal device 10 and the server 20, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer will be omitted.

[0025] <1.1 Terminal device configuration> Fig. 2 is a block diagram showing an example configuration of the terminal device 10 shown in Fig. 1. As shown in Fig. 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 160, a position information sensor 150, a storage unit 180, and a control unit 190. The blocks included in the terminal device 10 are electrically connected by, for example, a bus or the like.

[0026] The communication unit 120 performs processing such as modulation and demodulation for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to the outside (for example, the server 20). The communication unit 120 performs reception processing on the signal received from the outside and outputs it to the control unit 190.

[0027] The input device 13 is a device for inputting instructions or information by a user operating the terminal device 10. The input device 13 is realized, for example, by a touch-sensitive device 131 or the like, which inputs instructions by touching an operation surface. When the terminal device 10 is a PC or the like, the input device 13 may be realized by a reader, keyboard, mouse, or the like. The input device 13 converts instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may include, for example, a receiving port that receives electrical signals input from an external input device.

[0028] The output device 14 is a device for presenting information to a user operating the terminal device 10. The output device 14 is realized, for example, by a display 141 or the like. The display 141 displays data according to the control of the control unit 190. The display 141 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display or the like.

[0029] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of an audio signal. The audio processing unit 17 converts a signal provided from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is realized, for example, by a processor for audio processing. The microphone 171 receives audio input and provides an audio signal corresponding to the audio input to the audio processing unit 17. The speaker 172 converts the audio signal provided from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0030] The camera 160 is a device that receives light with a light receiving element and outputs the light as an image capturing signal.

[0031] The position information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. In the satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10 equipped with the GPS module is detected based on the received signals. The position information sensor 150 may detect the current position of the terminal device 10 from the position of the wireless base station to which the terminal device 10 is connected.

[0032] The storage unit 180 is realized by, for example, the memory 15, the storage 16, etc., and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181, object information 182, and environment information 183.

[0033] The user information 181 includes, for example, information about the user who uses the terminal device 10. Details will be described later.

[0034] The object information 182 includes information about objects to be generated in the area where columns can occur, as will be described in detail later.

[0035] The environmental information 183 includes information that indicates the current environment. The information that indicates the current environment includes, for example, information about the weather, atmospheric pressure, temperature, humidity, etc. Details will be described later.

[0036] The control unit 190 is realized by the processor 19 reading a program stored in the storage unit 180 and executing instructions included in the program. The control unit 190 controls the operation of the terminal device 10. By operating in accordance with the program, the control unit 190 fulfills the functions of an operation reception unit 191, a transmission / reception unit 192, an imaging control unit 193, an image analysis unit 194, and a display control unit 195.

[0037] The operation reception unit 191 performs processing for receiving instructions or information input from the input device 13. Specifically, for example, the operation reception unit 191 receives instructions or information input from the touch-sensitive device 131 or the like.

[0038] Furthermore, the operation receiving unit 191 receives an image input from the camera 160. Specifically, for example, the operation receiving unit 191 receives image data captured by the camera 160.

[0039] Furthermore, the operation reception unit 191 receives audio information input from the microphone 171. Specifically, for example, the operation reception unit 191 receives audio data that is input from the microphone 171 and converted into digital data by the audio processing unit 17.

[0040] The transmitting / receiving unit 192 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 transmits instructions input by a user or various pieces of acquired information to the server 20. In addition, the transmitting / receiving unit 192 receives information provided by the server 20.

[0041] The photographing control unit 193 controls photographing by the camera 160. For example, the photographing control unit 193 starts photographing by the camera 160 in response to an instruction from the user.

[0042] The image analysis unit 194 controls the analysis of the photographed data captured by the camera 160. For example, the image analysis unit 194 detects a predetermined area included in the photographed data. Specifically, for example, the image analysis unit 194 detects an area included in the photographed data for generating an object image using augmented reality.

[0043] Also, for example, the image analysis unit 194 detects customers who are waiting in line, which are included in the captured data. For example, the image analysis unit 194 detects customers who stay at a position a predetermined distance in a predetermined direction from the area for generating the object image for a predetermined period of time or more as customers who are waiting in line. Also, if a mark for specifying the position of the customers is attached to the floor or the like, the image analysis unit 194 may detect customers who stay near the mark as customers who are waiting in line. The image analysis unit 194 may also detect customers who are waiting at the end of the line.

[0044] The image analysis unit 194 may use the detection results of customers waiting in line when detecting an area for generating an object image using augmented reality. For example, the image analysis unit 194 may detect customers waiting in line and detect an area near the detected customers. Specifically, for example, the image analysis unit 194 may detect an area located a predetermined distance behind the customer at the end of the line in the direction of line formation. Furthermore, for example, the image analysis unit 194 may detect an area located between customers waiting in line.

[0045] The image analysis unit 194 may use any existing technology as a method for detecting an area, customers lining up, etc. The image analysis unit 194 may detect an area, customers lining up, etc. by using a trained model.

[0046] The display control unit 195 controls the output device 14 to display a predetermined image to the user. For example, the display control unit 195 generates an object image based on information detected from the shooting data and displays it on the display 141. Specifically, for example, the display control unit 195 generates an object image corresponding to the photographed area and displays the object image at a position corresponding to the area. The display control unit 195 may generate different object images based on the following information, for example: - Past product purchase history (purchase log) -Specified dates (birthdays, anniversaries) · Number of past visits (visit log) Favorite Character Calendar (days of the week, holidays, etc.) ·weather

[0047] For example, the display control unit 195 displays the photographed area on the display 141 and superimposes the created object image on an area included in the photographed area for generating an object image by augmented reality. In addition, in the case of a device such as an HMD in which the actual situation is viewed through the display, the display control unit 195 displays the object image at a position corresponding to the area for generating an object image by augmented reality.

[0048] Furthermore, the display control unit 195 generates a performance according to predetermined information. The predetermined information is, for example, the following. -Detecting new customers joining the queue -Detecting the movement of customers in line moving forward -Detecting customers leaving the queue Interference with the object image displayed on the display 141 (contact with the object image displayed on the display 141 via the touch-sensitive device 131, input of sound to the object image displayed on the display 141) Past product purchase history -Specified dates (birthdays, anniversaries) Number of past visits Favorite Character Calendar (days of the week, holidays, etc.) ·weather

[0049] The configuration of the terminal device 10 is not limited to the above. The terminal device 10 may also include a three-dimensional measurement unit. The three-dimensional measurement unit calculates the three-dimensional coordinates of the reflection point, for example, by irradiating a laser onto a predetermined scanning area at a predetermined cycle and analyzing the reflected light. The three-dimensional measurement unit is realized, for example, by LiDAR (Light Detection and Ranging). The three-dimensional measurement unit is installed, for example, so as to be able to measure the same direction as the camera 160. The three-dimensional measurement unit transmits data related to the calculated three-dimensional coordinates to the control unit 190 as point cloud data.

[0050] <1.2 Functional configuration of the server> 3 is a diagram showing an example of the functional configuration of the server 20. As shown in FIG. 3, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0051] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0052] The storage unit 202 includes, for example, a customer information table 2021, an object information table 2022, a location information table 2023, an environment information table 2024, a trained model 2025, and the like.

[0053] The customer information table 2021 is a table that stores information about registered customers.

[0054] The object information table 2022 is a table that stores information about objects that are generated in areas where columns can occur.

[0055] The position information table 2023 is a table that stores the current location of a customer. The current location of a customer is set based on, for example, a three-dimensional position measured by a three-dimensional measuring device installed in the facility. The three-dimensional measuring device, for example, irradiates a predetermined scanning area with a laser at a predetermined cycle and analyzes the reflected light to calculate the three-dimensional coordinates of the reflection point. The three-dimensional measuring device is realized, for example, by LiDAR. The three-dimensional measuring device is installed, for example, in a position that allows an unobstructed view of the entire facility. The three-dimensional measuring device transmits data related to the calculated three-dimensional coordinates to the server 20 as point cloud data. When a customer visits the facility and operates the terminal device 10 to transmit their visit to the server 20, the location at the time of registration is associated with the user's ID. The server 20 refers to the point cloud data and tracks the user's movement to determine the customer's current location.

[0056] The environment information table 2024 is a table that stores information related to the environment.

[0057] The trained model 2025 is, for example, a model for detecting a predetermined area from a captured image. The trained model 2025 is stored in advance when the server 20 provides a service. The trained model 2025 is generated by having a machine learning model perform machine learning in accordance with a model learning program. The trained model 2025 is, for example, a parameterized composite function in which multiple functions are combined. The parameterized composite function is defined by a combination of multiple adjustable functions and parameters. The trained model 2025 according to this embodiment may be any parameterized composite function that meets the above requirements.

[0058] For example, when the trained model 2025 is generated using a forward propagation type multilayer network, the parameterized composite function is defined as a combination of, for example, a linear relationship between each layer using a weight matrix, a nonlinear relationship (or a linear relationship) using an activation function in each layer, and a bias. The weight matrix and bias are called parameters of the multilayer network. The form of the parameterized composite function as a function changes depending on how the parameters are selected. In a multilayer network, by appropriately setting the constituent parameters, it is possible to define a function that can output a desirable result from the output layer.

[0059] The multi-layered network according to the present embodiment may be, for example, a deep neural network (DNN), which is a multi-layered neural network that is the subject of deep learning. As the DNN, for example, a convolution neural network (CNN) that targets images may be used.

[0060] The trained model 2025 is trained, for example, using an image including an area where a queue may occur as input data and customers lining up in a queue that has occurred as correct output data. The trained model 2025 may be retrained as needed in accordance with variations in customers lining up in a queue.

[0061] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. The control unit 203 operates in accordance with the program to fulfill the functions of a reception control module 2031, a transmission control module 2032, an image analysis module 2033, and a display control module 2034.

[0062] The reception control module 2031 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol.

[0063] The transmission control module 2032 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0064] The image analysis module 2033 controls the analysis of the image data captured by the image capturing device 41. For example, the image analysis module 2033 detects a predetermined area included in the image data. Specifically, for example, the image analysis module 2033 detects an area included in the image data for generating an object image using augmented reality.

[0065] Furthermore, for example, the image analysis module 2033 detects customers who are waiting in line, which are included in the captured data. For example, the image analysis module 2033 detects customers who stay at a position a predetermined distance in a predetermined direction from the area for generating the object image for a predetermined period of time or more as customers who are waiting in line. Furthermore, if a mark specifying the position of customers is attached to the floor or the like, the image analysis module 2033 may detect customers who stay near the mark as customers who are waiting in line.

[0066] The image analysis module 2033 may use the detection results of customers waiting in line when detecting an area for generating an object image using augmented reality. For example, the image analysis module 2033 may detect customers waiting in line and detect an area near the detected customers. Specifically, for example, the image analysis module 2033 may detect an area located a predetermined distance behind the customer at the end of the line in the direction of line formation. Furthermore, for example, the image analysis module 2033 may detect an area located between customers waiting in line.

[0067] The image analysis module 2033 may use any existing technology to detect an area, customers waiting in line, etc. The image analysis module 2033 may detect an area, etc., using a trained model.

[0068] The display control module 2034 controls the display 30 to display a predetermined image to the customer. For example, the display control module 2034 generates an object image based on information detected from the shooting data and displays it on the display 30. Specifically, for example, the display control module 2034 generates an object image corresponding to the photographed area and displays the object image at a position corresponding to the area. The display control module 2034 may generate a different object image based on predetermined information. The predetermined information is, for example, the same as the information described in the display control unit 195. For example, the display control module 2034 displays the photographed area on the display 30 and superimposes the created object image on an area included in the photographed area for generating an augmented reality object image.

[0069] Furthermore, the display control module 2034 generates a performance according to predetermined information. The predetermined information is, for example, the same as the information described in the display control unit 195.

[0070] <2 Data Structure> 4 to 6 are diagrams showing the data structure of information stored in the terminal device 10. Note that FIGS. 4 to 6 are merely examples, and do not exclude data not shown.

[0071] Fig. 4 is a diagram showing the data structure of user information 181. The user information 181 shown in Fig. 4 has columns such as name, age, address, date of birth, email address, registration date, store visit information, purchase information, favorite character, and generated object, with the user ID as a key.

[0072] The user ID is an item that stores an identifier for uniquely identifying a user. The user ID may be an ID that was set when the user became a member of a store. The name is an item that stores the user's name. The age is an item that stores the user's age. The address is an item that stores the place where the user lives. The date of birth is an item that stores the date the user was born. The email address is an item that stores the user's email address. The registration date is an item that stores the date the user registered as a member.

[0073] The store visit information is an item that stores the history of a user's visits to a facility. If the user has visited the facility multiple times, the store visit information includes the date and time of each visit. If there are multiple related facilities, the store visit information may store the date and time of the visit in association with the name of the facility visited. The purchase information is an item that stores the history of products purchased by the user. The purchase information includes, for example, the purchase date and time, the product ID of the purchased product, the product name of the purchased product, etc. The favorite character is an item that stores the object ID of a character that the user has registered as a favorite.

[0074] Fig. 5 is a diagram showing the data structure of object information 182. Object information 182 shown in Fig. 5 is a table having columns such as object name, occurrence condition, object data, effect content, and requirements corresponding to the effect content, with object ID as a key.

[0075] The object ID is an item that stores an identifier for uniquely identifying an object. The object name is an item that stores the name of the object. The generation condition is an item that stores the condition for generating an object image. The condition for generating an object image includes, for example, the following: - You have previously purchased the specified product - You have previously purchased the specified type of product - The number of purchases of the product has exceeded the specified number of times. The purchase amount of the product exceeds a specified amount. The current date must match the user's birthday. The current date must match a specific date (e.g., Christmas, Marine Day, New Year's Day, Coming of Age Day, etc.). The current day is a specific day (e.g., Friday). - The number of past visits exceeds the specified number. The set object (character) matches your favorite character. The weather is the specified weather (e.g., snow). -Identifying information contained in the marker

[0076] If the marker includes identification information, the item "occurrence condition" may store an identification number.

[0077] Object data is an item that stores image data about an object. Object data may also store reference information (path) to an object data file located elsewhere. Image data about an object includes, for example: - Image data of the initial state of a specified character Image data of the evolved state of a specific character Image data of a specific character wearing special clothing Image data with multiple specified characters

[0078] The performance content is an item that stores image data about the performance that the object will perform. The performance content may also store reference information (path) to an image data file located in another location. The performance content includes, for example, the following: - A performance in which an object performs a specific action (celebrating a birthday or a specific day) -Standing behind new customers in line - A performance that moves forward in line with customers who have progressed in line · Stepping out of line -Evolving objects - The object will wear a specific accessory.

[0079] 5, the effect content includes a first effect, a second effect, etc. The requirement is an item for storing requirements for making the object perform the effect. The requirement includes, for example, the following: -Detecting new customers joining the queue -Detecting the movement of customers in line moving forward -Detecting customers leaving the queue Touching an object image displayed on a display 141 via a touch-sensitive device 131 Predefined voice input - You have previously purchased the specified product - You have previously purchased the specified type of product - The number of purchases of the product has exceeded the specified number of times. The purchase amount of the product exceeds a specified amount. The current date must match the user's birthday. The current date must match a specific date (e.g., Christmas, Marine Day, New Year's Day, Coming of Age Day, etc.). The current day is a specific day (e.g., Friday). - The number of past visits exceeds the specified number. The set object (character) matches your favorite character. The weather is the specified weather (e.g., snow).

[0080] The example shown in FIG. 5 includes a first requirement for implementing a first performance, a second requirement for implementing a second performance, and so on.

[0081] Fig. 6 is a diagram showing the data structure of environmental information 183. The environmental information 183 shown in Fig. 6 is a table having columns for date, weather, atmospheric pressure, temperature, and humidity. The environmental information 183 may also include information related to the calendar. The information related to the calendar includes, for example, information related to days of the week and information related to public holidays.

[0082] The date is an item for storing the date on which the environmental information was acquired. Weather, atmospheric pressure, temperature, and humidity are examples of environmental information, and information provided by a predetermined information source is stored.

[0083] <3 operations> The operation of the terminal device 10 and the server 20 when displaying object images to customers to make the act of queuing more interesting will be described. (Display of object image by terminal device 10)

[0084] In the following description, it is assumed that a customer visits a store, places the items they want to purchase in a shopping cart, and waits at the cash register to pay for the purchase. It is also assumed that the terminal device 10 used by the customer is a device with a display, such as a smartphone or tablet.

[0085] FIG. 7 is a schematic diagram showing the area around the cash register counter in a store according to this embodiment. In FIG. 7, markings 51 are provided on the floor of the store to help customers waiting for their turn at the cash register counter. Alternatively, markings 51 may be used to form a queue. Markings 51 may be physical marks such as stickers or paint, or optical marks with a predetermined shape projected onto them. Markings 51 are provided at intervals that prevent droplets from spreading during normal conversation, for example, at intervals of about 2 meters. Markings 51 may be arranged in a straight line or a curved line.

[0086] Markers 52 are provided between the marks 51. The markers 52 are arranged at a predetermined distance and in a predetermined direction relative to the marks 51. The markers 52 may be physical, such as stickers or paint, or may be optical, with a predetermined shape projected onto them. Note that as long as the markers 52 are provided, the marks 51 do not necessarily have to be provided.

[0087] FIG. 8 is a flowchart showing an example of the operation of the terminal device 10 when generating an object image in a column.

[0088] First, the user operates the terminal device 10 to capture an image of the area around the checkout counter.

[0089] In step S11, the control unit 190 controls the camera 160 via the photography control unit 193 to take a video of an area where a queue may form, for example, an area around a checkout counter.

[0090] In step S12, the control unit 190 analyzes the captured image data using the image analysis unit 194. For example, the image analysis unit 194 detects a marker 52, which is included in the captured image data and is an area for generating an object image using augmented reality. Furthermore, if customers are queuing up in front of the cash register counter, the image analysis unit 194 detects the customers queuing up, which is included in the captured image data.

[0091] In step S13, the control unit 190 causes the display control unit 195 to display an image on the display 141. For example, the display control unit 195 generates an object image based on information detected from the shooting data, and causes the display 141 to display the generated object image together with the shooting data.

[0092] (Example of object image display) FIG. 9 is a flowchart showing a specific example of the operation of the display control unit 195 according to this embodiment when displaying an object image.

[0093] In step S21, the display control unit 195 determines whether a marker 52 located immediately after the customer at the end of the line or a marker 52 located between customers in the line has been detected. If a marker 52 located immediately after the customer at the end of the line or a marker 52 located between customers in the line has been detected (Yes in step S21), the display control unit 195 proceeds to step S22. If a marker 52 located immediately after the customer at the end of the line or a marker 52 located between customers in the line has not been detected (No in step S21), the display control unit 195 proceeds to step S24.

[0094] In step S22, the display control unit 195 creates an object image for each marker 52 detected in step S21. Specifically, the display control unit 195 acquires, for example, information related to the "occurrence condition" item of the object information 182 from the user information 181 and the environmental information 183. The display control unit 195 selects a record that satisfies the occurrence condition based on the acquired information. The display control unit 195 acquires image data based on data stored in the "object data" item of the selected record. If there are multiple objects that satisfy the occurrence condition, the display control unit 195 selects a record based on, for example, a predetermined priority. The display control unit 195 may select different objects or the same object for each detected marker 52.

[0095] In step S23, the display control unit 195 displays the object image on the display 141. Specifically, the display control unit 195 superimposes the object image based on the acquired image data on the marker 52 detected in step S21. As a result, the object image is displayed on the marker 52 located immediately behind the customer at the end of the line and on the markers 52 located between the customers in the line.

[0096] Fig. 10 is a schematic diagram showing a display example of the display 141 according to this embodiment. In Fig. 10, the image analysis unit 194 detects a customer 100 who remains within the mark 51 as a customer standing in line. Then, the image analysis unit 194 detects a marker 52 located immediately behind the customer 100. The display control unit 195 superimposes an object image 53 on the detected marker 52. In the example shown in Fig. 10, a character standing in line with the customer and facing away is displayed as the object image 53. In other words, the object image is displayed behind the customer.

[0097] In step S24, the display control unit 195 determines whether or not an event has occurred that satisfies the requirements for generating a performance. Specifically, for example, the display control unit 195 determines whether or not an event has occurred that satisfies the item "requirements" of the object information 182. If an event has occurred that satisfies the requirements for generating a performance, the display control unit 195 proceeds to step S25.

[0098] In step S25, the display control unit 195 generates a performance according to the event. Specifically, for example, the display control unit 195 generates a performance stored in the item: "performance" corresponding to the requirement stored in the item: "requirement."

[0099] Specifically, for example, in step S24, when a new customer is detected lining up, the display control unit 195 determines that an event that satisfies the requirements for generating a performance has occurred. Then, in step S25, the display control unit 195 acquires image data as shown in step S22, and moves an object image based on the acquired image data toward the marker 52.

[0100] 11 and 12 are schematic diagrams showing display examples of the display 141 according to this embodiment. In FIGS. 11 and 12, the image analysis unit 194 detects a customer 100 who has moved into a mark 51. Then, the image analysis unit 194 detects a marker 52 located immediately behind the customer 100. The display control unit 195 moves the object image 53 to the detected marker 52 so that the object image 53 joins the line. The display control unit 195 may cause the object image 53, which has completed its movement, to perform a greeting or other effect.

[0101] As shown in Fig. 12, when a new object image is displayed in addition to an object image that is already displayed, the newly displayed object image may be the same character as the object image that is already displayed, or may be a different character. Fig. 12 shows an example in which the character is the same.

[0102] Also, for example, in step S24, the display control unit 195 determines that an event that satisfies the requirements for generating a performance has occurred when the object image displayed on the display 141 is touched by the user via the touch-sensitive device 131. Then, in step S25, the display control unit 195 displays a predetermined performance for the touched object image.

[0103] 13 and 14 are schematic diagrams showing display examples of the display 141 according to this embodiment. In FIG. 13, the displayed object image 53 is poked by the user. The hand in FIG. 13 schematically represents the user's contact with the object image 53. As shown in FIG. 14, the display control unit 195 produces an effect in which the object image 53 turns around and strikes a pose. Note that the action in response to the user's interference is not limited to a predetermined pose, and a predetermined action or a predetermined technique may be performed.

[0104] Also, for example, in step S24, when an object image is displayed at the end of the line and a customer in line moves forward toward the mark 51 on the cashier counter side, the display control unit 195 determines that an event that satisfies the requirements for generating a performance has occurred. Then, in step S25, the display control unit 195 moves the displayed object image toward the next marker 52 in the direction the customer moved.

[0105] Fig. 15 is a schematic diagram showing an example of a display on the display 141 according to this embodiment. For example, suppose that the first customer in the state shown in Fig. 13 has finished the transaction, and customer 100 in line moves forward as shown in Fig. 15. The image analysis unit 194 detects customer 100 who has moved to marker 51, which is one step closer to the checkout counter. The image analysis unit 194 then detects marker 52, which is located immediately behind the moved customer 100. The display control unit 195 moves the object image 53 from the marker 52 on which the object image 53 was superimposed in Fig. 13, to the marker 52 that is one step closer to the checkout counter.

[0106] Also, for example, in step S24, when an object image is displayed at the end of a line and the customer at the end of the line leaves the line, the display control unit 195 determines that an event that satisfies the requirements for generating an effect has occurred. Then, in step S25, the display control unit 195 moves the displayed object image so that it leaves the line.

[0107] Fig. 16 is a schematic diagram showing a display example of the display 141 according to this embodiment. For example, suppose that the customer 100 at the end of the line in the state shown in Fig. 13 leaves the line as shown in Fig. 16. The image analysis unit 194 detects the customer 100 who has left the line and moved away from the mark 51. The display control unit 195 moves the object image 53 from the marker 52 on which the object image 53 was superimposed in Fig. 13 in a direction away from the line.

[0108] (Display of object image by server 20) Next, a case will be described in which the inside of a store is photographed by a photographing device 41 installed in the store and displayed on the display 30. In the following explanation, it is assumed that a customer visits a specific store, places the items they want to purchase in a shopping cart, and waits at the cash register to pay for the purchase.

[0109] FIG. 17 is a flowchart showing an example of the operation of the server 20 when generating an object image in a sequence.

[0110] First, the camera device 41 installed in the store captures a video of the area around the cash register counter.

[0111] In step S31, the control unit 203 causes the reception control module 2031 to acquire the photographic data photographed by the photographing device 41.

[0112] In step S32, the control unit 203 analyzes the captured image data using the image analysis module 2033. For example, the image analysis module 2033 detects a marker 52, which is included in the captured image data and is an area for generating an object image using augmented reality. Furthermore, if customers are queuing up in front of the cash register counter, the image analysis module 2033 detects the customers waiting in line, which is included in the captured image data.

[0113] In step S33, the control unit 203 causes the display control module 2034 to display an image on the display 30. For example, the display control module 2034 generates an object image based on information detected from the shooting data, and displays the generated object image on the display 30 together with the shooting data.

[0114] 18 is a schematic diagram showing a display example of the display 30 according to this embodiment. In FIG. 18, the image analysis module 2033 detects customers who remain within the mark 51 as customers waiting in line. The image analysis module 2033 then detects a marker 52 located immediately behind the customer at the end of the line and a marker 52 located between customers waiting in line. The display control module 2034 superimposes an object image 53 on the detected marker 52. In the example shown in FIG. 18, characters waiting in line with the customers are displayed as object images 53.

[0115] As described above, in the above embodiment, a plurality of second areas for generating augmented reality object images are arranged at predetermined intervals in a first area where a queue may be formed. The control unit 190 acquires image data captured of the first area. The control unit 190 detects the second area based on the acquired image data using the image analysis unit 194. The control unit 190 creates an object image corresponding to the detected second area using the display control unit 195. The display control unit 195 displays the created object image at a position corresponding to the second area. This makes it possible for the terminal device 10 or the server 20 according to this embodiment to generate augmented reality objects in the queue, making it possible for customers waiting in line to enjoy the act of waiting.

[0116] Therefore, according to this embodiment, it is possible to increase the interest of waiting in line.

[0117] In the above embodiment, the display control unit 195 displays an image based on the shooting data and displays an object image superimposed on a position in the image corresponding to the second region. This allows the user to easily view the object image generated in the line by holding a smartphone, tablet terminal, or the like over an area where a line may be formed.

[0118] Furthermore, in the above embodiment, the image analysis unit 194 detects people standing in a line and detects the second region using the person detection result. The display control unit 195 displays an object image at a position corresponding to the second region behind the person in the direction of line formation. As a result, an object image appears behind the people standing in a line. In other words, people and objects appear to be lined up alternately. This makes it possible to see a line that does not actually exist, allowing the user to enjoy the action of lining up even more. Furthermore, when multiple people are lined up, multiple objects appear, allowing the user to enjoy the action of lining up even more.

[0119] In the above embodiment, the image analysis unit 194 detects the person at the end of the line, and detects the second area using the detection result of the last person. The display control unit 195 displays the object image behind the last person. This makes the object appear to be at the end of the line, and the user will line up next to the object. Feeling the object closer to the user makes lining up more enjoyable.

[0120] In the above embodiment, the display control unit 195 creates an object image based on user information. This makes it possible to generate a different object each time the user visits the store. This increases the user's enjoyment of visiting the store and motivates the user to visit again. The user information includes, for example, a store visit log, a purchase log, special days, or information about favorites.

[0121] In the above embodiment, the display control unit 195 creates object images based on environmental information. This makes it possible to generate different objects depending on the environment. This increases the user's enjoyment of visiting the store, motivating them to visit again. The environmental information includes, for example, information about the calendar or weather.

[0122] In the above embodiment, the display control unit 195 determines whether or not the requirements for generating an effect are met. The display control unit 195 causes the object image to perform an effect according to the met requirements. This allows the object to perform an effect according to the requirements, allowing the user to enjoy the lining up action even more.

[0123] In the above embodiment, the display control unit 195 detects a person who has joined the queue. The display control unit 195 causes the object image to perform an effect of joining the queue. As a result, when a new customer joins the queue, the number of objects increases, further increasing the interest.

[0124] In the above embodiment, the display control unit 195 detects a person moving forward in a line. The display control unit 195 causes the object image to perform an effect of moving forward in a line. As a result, the object behaves in a distinctive manner as the line moves forward, allowing the user to look forward to the line moving forward.

[0125] In the above embodiment, the display control unit 195 detects a person who has left the queue. The display control unit 195 causes the object image to perform an effect of leaving the queue. As a result, when a customer leaves the queue, the object behaves in a distinctive manner, so that the user can look forward to the action of the customer leaving the queue.

[0126] In the above embodiment, the display control unit 195 detects interference with the object image. The display control unit 195 causes the object image to perform an action in response to the interference. This allows the object to perform an action in response to the user's interference even if there is no change in the queue, so that the user can enjoy the act of waiting even if there is no change in the queue.

[0127] <Modification> In the above embodiment, when a predetermined condition is met, an object image according to the condition is displayed, and when a predetermined requirement is met, the object image performs an effect according to the requirement. At this time, it is not displayed what condition is met to display the object image, or what requirement is met to cause the object image to perform the predetermined effect.

[0128] However, it may also be clearly stated what conditions must be met for the object image to be displayed. It may also be clearly stated what requirements must be met for the object image to produce a predetermined effect. For example, a statement such as "This occurred because the 'predetermined action' was completed!" may be displayed in a manner that does not interfere with the display of the object image, for example, as a temporary pop-up. This allows the user to understand the conditions for the appearance of a characteristic object or the requirements for the appearance of a characteristic effect, and enables the user to consciously reproduce a characteristic object or a characteristic effect. This allows the user to adjust their actions to produce the object or effect they desire.

[0129] Furthermore, in the above embodiment, no hint about the next object to be displayed is given. However, hints about the next object to be displayed may be given. For example, a statement such as "Next time you perform a 'predetermined action', a 'predetermined object or a predetermined effect' will occur!" may be displayed in a manner that does not interfere with the display of the object image, for example, as a temporary pop-up. Alternatively, a more concealed expression may be used. This allows the user to know what object or effect may occur next, making the user more excited about their next visit.

[0130] Furthermore, in the above embodiment, the area for generating the object image may be installed at the checkout counter. The marker as the area placed at the checkout counter may include a predetermined code. The predetermined code may be, for example, a code individually assigned to the store clerk who will be handling the transaction, and is a code for generating the object assigned to the store clerk. As a result, an object unique to the store clerk will be generated at the head of the line, allowing users to line up in anticipation of what the store clerk's object will be. In other words, the user's enjoyment of waiting in line will increase.

[0131] In the above embodiment, the image analysis unit 194 or the image analysis module 2033 detects customers queuing up. However, the detection of customers queuing up is not limited to the image analysis unit 194 or the image analysis module 2033. The control unit 190 may function as a queue state recognition unit by operating according to a program. Furthermore, the control unit 203 may function as a queue state recognition module by operating according to a program. The queue state recognition unit detects customers queuing up, for example, based on point cloud data measured by a three-dimensional measurement unit. Furthermore, the queue state recognition module detects customers queuing up, for example, based on point cloud data measured by a three-dimensional measurement module.

[0132] The queue state recognition unit or queue state recognition module may use any existing technology to detect customers or the like waiting in line.

[0133] In the above embodiment, an example has been described in which another customer is lined up behind a customer currently making a purchase, as shown in FIG. 10 . However, a character may also be lined up immediately behind the customer currently making a purchase. That is, for example, as shown in FIG. 19 , an area for generating an object image using augmented reality may be provided between the cash register counter and the position where the first customer in line is waiting. As a result, for example, if there is a customer currently making a purchase at the cash register counter, an object image will be generated between the customer currently making a purchase and the customer waiting at the front of the line. Furthermore, as shown in FIG. 20 , a situation may arise in which only the object image is lined up behind the customer currently making a purchase. This allows customers to easily recognize where the line begins.

[0134] Furthermore, for example, when a customer completes a transaction at a checkout counter and leaves the counter, the object image may guide the customer at the front of the line. Specifically, for example, when a customer completes a transaction at a checkout counter and leaves the counter, the image analysis unit 194 or the image analysis module 2033 detects that the customer has left the checkout counter. The display control unit 195 or the display control module 2034 causes the object image superimposed on the marker immediately in front of the checkout counter to guide the customer at the front of the line. FIG. 21 is a schematic diagram illustrating an example of a display on the display 141 according to this embodiment.

[0135] <4 Basic computer hardware configuration> 22 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 91, a main memory device 92, an auxiliary memory device 93, and a communication IF (interface) 99. These are electrically connected to each other by a bus.

[0136] The processor 91 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0137] The main storage device 92 is used to temporarily store programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0138] The auxiliary storage device 93 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0139] The communication IF 99 is an interface for inputting and outputting signals for communicating with other computers via a network using wired or wireless communication standards.

[0140] A network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, networks include 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, networks also include those that are directly connected using a USB (Universal Serial Bus) cable, etc.

[0141] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration across multiple computers 90 and interconnecting them via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0142] <Basic functional configuration of computer 90> A description will be given of the functional configuration of a computer realized by the basic hardware configuration of a computer 90 shown in Fig. 22. The computer includes at least the functional units of a control unit, a storage unit, and a communication unit.

[0143] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0144] The control unit is realized by the processor 91 reading various programs stored in the auxiliary storage device 93, expanding them in the main storage device 92, and executing processing in accordance with the programs. The control unit can realize functional units that perform various types of information processing depending on the type of program. In this way, the computer is realized as an information processing device that processes information.

[0145] The storage unit is realized by a main storage device 92 and an auxiliary storage device 93. The storage unit stores data, various programs, and various databases. Furthermore, the processor 91 can allocate a storage area corresponding to the storage unit in the main storage device 92 or the auxiliary storage device 93 in accordance with the programs. Furthermore, the control unit can cause the processor 91 to execute processes for adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0146] A database refers to a relational database, which manages data sets called tables, which are structured by rows and columns, by relating them to each other. In a database, a table is called a table, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables can be set and associated.

[0147] Usually, each table has a column set as a key for uniquely identifying a record, but setting a key to a column is not essential. The control unit can cause the processor 91 to add, delete, or update records in a specific table stored in the storage unit according to various programs.

[0148] The communication unit is realized by the communication IF 99. The communication unit realizes the function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 91 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0149] Although several embodiments of the present disclosure have been described above, these embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are intended to be included in the scope of the inventions and their equivalents as defined in the claims, as well as in the scope and spirit of the inventions.

[0150] <Additional Notes> The matters described in the above embodiments will be supplemented below. (Appendix 1) A program to be executed by a computer having a processor and a memory, the program causing the processor to execute the following steps: acquiring photographic data of a first area where a queue of customers may form, and a plurality of second areas for generating augmented reality object images arranged at predetermined intervals in the first area; detecting the second area based on the acquired photographic data; creating an object image corresponding to the detected second area; and displaying the created object image at a position corresponding to the second area. (Appendix 2) The program according to claim 1, wherein in the step of displaying the object image, an image based on the shooting data is displayed, and the object image is superimposed and displayed at a position in the image corresponding to the second region. (Appendix 3) A program described in (Appendix 1) or (Appendix 2), wherein in the detecting step, customers standing in line are detected, a second area is detected using the customer detection results, and in the displaying step, the object image is displayed at a position corresponding to the second area behind the customer in the direction of queue formation. (Appendix 4) A program described in Appendix 3, in which, in the detection step, a customer at the end of a line is detected, a second area is detected using the detection result of the last customer, and in the object image display step, the object image is displayed behind the last customer. (Appendix 5) The program according to any one of (Supplementary Note 1) to (Supplementary Note 4), wherein in the step of creating an object image, the object image is created based on user information. (Appendix 6) The program according to claim 5, wherein in the step of creating an object image, the object image is created based on information about a store visit log, a purchase log, a special day, or favorites included in the user information. (Appendix 7) The program according to any one of (Supplementary Note 1) to (Supplementary Note 4), wherein in the step of creating an object image, the object image is created based on environmental information. (Appendix 8) The program according to claim 7, wherein in the step of creating an object image, the object image is created based on information about a calendar or weather included in the environmental information. (Appendix 9) A program described in any of (Appendix 1) to (Appendix 8), which causes a processor to execute a step of determining whether the requirements for generating an effect are met, and in a step of displaying an object image, causes the object image to perform an effect according to the met requirements. (Appendix 10) The program according to claim 9, wherein the determining step detects a customer who has joined the queue, and the object image is made to perform an effect of the customer joining the queue in the displaying step. (Appendix 11) The program according to claim 9, wherein in the determining step, a customer moving forward in line is detected, and in the displaying step, the object image is made to perform an effect of moving forward in line. (Appendix 12) The program according to claim 9, wherein the determining step detects a customer who has left the queue, and the object image is made to perform an effect of the customer leaving the queue in the displaying step of the object image. (Appendix 13) The program according to claim 9, wherein in the determining step, interference with the object image is detected, and in the displaying step, the object image is made to perform an action according to the interference. (Appendix 14) A method executed by a computer having a processor and a memory, wherein the processor executes the steps of: acquiring photographic data of a first area where a queue of customers may form, and a plurality of second areas for generating augmented reality object images arranged at predetermined intervals in the first area; detecting the second area based on the acquired photographic data; creating an object image corresponding to the detected second area; and displaying the created object image at a position corresponding to the second area. (Appendix 15) An information processing device comprising a control unit and a memory unit, wherein the control unit has a first area where customers may form a queue, and a plurality of second areas for generating object images using augmented reality arranged at predetermined intervals, and the information processing device executes the steps of acquiring photographic data taken of the first area, detecting the second area based on the acquired photographic data, creating an object image corresponding to the detected second area, and displaying the created object image at a position corresponding to the second area. (Appendix 16) A system comprising a first area where customers can form a queue, and a second area for generating augmented reality object images, which are arranged at predetermined intervals within the first area. (Appendix 17) A system as described in (Appendix 16), wherein a mark indicating the location where the customer should stop is placed in the first area separately from the second area. (Appendix 18) The system described in (Appendix 17), wherein the markings and the second area are arranged alternately in the first area. (Appendix 19) A system described in any one of (Appendix 16) to (Appendix 18) comprising: means for acquiring photographic data of a first area; means for detecting a second area based on the acquired photographic data; means for creating an object image corresponding to the detected second area; and means for displaying the created object image at a position corresponding to the second area. [Explanation of symbols]

[0151] 1. System 10...Terminal device 12...Communication IF 120…Communications Department 13...Input device 131...Touch-sensitive devices 14...Output device 141...Display 15...Memory 150...Location information sensor 16…Storage 160...camera 17...Audio processing unit 171...Mike 172...Speaker 180...Storage section 19...Processor 190...Control unit 20...Server

Claims

1. A program to be executed by a computer having a processor and a memory, the program causing the processor to: acquiring photographic data of a first area of ​​the store where customers may form a queue and a second area where customers may make a payment; displaying a character image in the first area behind a customer who is making a transaction in the second area; detecting the movement of customers in the second area; a step of displaying the character image, which moves from the first area to the second area in accordance with the detected movement of the customer in the second area, in an area between the first area and the second area; A program that executes the following.

2. In the detecting step, it is detected that customers have left the second area; The program of claim 1, wherein, in the display step, if there are no customers in the second area, the character image is displayed to guide customers in the first area to the second area.

3. A method implemented on a computer having a processor and a memory, wherein the processor executes all of the steps performed in the invention according to claim 1 or claim 2.

4. 10. An information processing device comprising a processor and a memory, wherein the processor executes all of the steps executed in the invention according to claim 1 or 2.

5. A system comprising means for executing all of the steps performed in the invention according to claim 1 or claim 2.

Images