Program, method, information processing apparatus, and system

Abstract

To increase visitors' interest when the visitors wait in line.SOLUTION: A program is to be executed by a computer comprising a processor and a memory. The program causes the processor to execute the steps of: acquiring photographic data obtained by photographing a first area where a line may be formed by visitors and a plurality of second areas for generating object images by augmented reality is arranged at every predetermined interval; detecting the second areas based on the acquired photographic data; creating the object images corresponding to the detected second areas; and displaying the created object images at positions corresponding to the second areas.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] In recent years, technologies related to AR (Augmented Reality) and MR (Mixed Reality) have been developing (see Patent Document 1). AR and MR extend the real world seen by a user by adding information to the real space that the user is viewing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2016-506565 Summary of the Invention [Problem to be solved by the invention]

[0004] In places where people tend to gather in large numbers, measures to avoid crowding are being implemented everywhere. For example, in lines waiting to pay, signs indicating the location of customers are installed on the floor of the waiting area to ensure distance between customers in the line. However, waiting in line while maintaining a distance is not very interesting.

[0005] An object of the present disclosure is to make waiting in line more interesting. [Means for solving the problem]

[0006] A program to be executed by a computer having a processor and a memory. The program causes the processor to execute the steps of: acquiring image data of a first area where a queue of customers may be formed, a plurality of second areas for generating an object image using augmented reality are arranged at predetermined intervals in a first area, detecting the second area based on the acquired image data, creating an object image corresponding to the detected second area, and displaying the created object image at a position corresponding to the second area. Effect of the Invention

[0007] According to the present disclosure, it is possible to increase the enjoyment of waiting in line. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the overall configuration of a system 1. FIG. [Diagram 2] 2 is a block diagram illustrating an example of the configuration of a terminal device 10 shown in FIG. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a server 20. [Figure 4] FIG. 13 is a diagram showing the data structure of user information 181. [Diagram 5] FIG. 13 is a diagram showing the data structure of object information 182. [Figure 6] FIG. 13 is a diagram showing the data structure of environment information 183. [Figure 7] 1 is a schematic diagram showing an area around a checkout counter in a store according to an embodiment of the present invention. FIG. [Figure 8] 11 is a flowchart showing an example of an operation performed by the terminal device 10 when generating an object image in a column. [Figure 9] 10 is a flowchart showing a specific example of an operation performed by the display control unit 195 according to the present embodiment when displaying an object image. [Figure 10] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 11]3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 12] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 13] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 14] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 15] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 16] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 17] 11 is a flowchart showing an example of an operation performed by the server 20 when generating an object image in a sequence. [Figure 18] 2 is a schematic diagram showing a display example of a display 30 according to the present embodiment. FIG. [Figure 19] 13 is a schematic diagram showing another example of the area around the checkout counter in the store according to the embodiment. FIG. [Figure 20] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 21] 3 is a schematic diagram showing a display example of a display 141 according to the present embodiment. FIG. [Figure 22] FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[0010] <Summary> The system according to this embodiment is constructed, for example, in a facility where customers queue up. The system recognizes a predetermined area arranged between customers in a queue, and generates an object image so that the customers can understand that an object exists in the recognized area. This allows the customers to feel as if they are queuing together with the object, and makes the time spent queuing up, which should be boring, more enjoyable.

[0011] <1 Overall system configuration> Fig. 1 is a block diagram showing an example of the overall configuration of a system 1. The system 1 shown in Fig. 1 includes, for example, a terminal device 10, a server 20, a display 30, and an image capturing device 41. The terminal device 10, the server 20, the display 30, and the image capturing device 41 are communicatively connected via, for example, a network 80.

[0012] 1 shows an example in which the system 1 includes one terminal device 10, the number of terminal devices 10 included in the system 1 is not limited to one. The number of terminal devices 10 included in the system 1 may be one or more.

[0013] 1 shows an example in which the system 1 includes one server 20, but the number of servers 20 included in the system 1 is not limited to one. The server 20 may be composed of multiple servers depending on the functions it has. Also, the server 20 may be, for example, a collection of multiple devices that constitutes one server. The method of allocating multiple functions required to realize the server 20 according to this embodiment to one or multiple pieces of hardware can be appropriately determined in consideration of the processing capacity of each piece of hardware and / or the specifications required for the server 20.

[0014] 1 shows an example in which the system 1 includes one display 30 and one image capturing device 41, but the number of displays 30 and image capturing devices 41 included in the system 1 is not limited to one. The number of displays 30 and image capturing devices 41 included in the system 1 may be one or more.

[0015] 1 is, for example, an information processing device operated by a customer waiting in line. The terminal device 10 is realized by, for example, a mobile terminal such as a smartphone or a tablet. The terminal device 10 may be a wearable terminal such as an HMD (Head Mount Display).

[0016] The terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19. The input device 13 is a device (e.g., a touch panel, a touch pad, etc.) for receiving an input operation from a user. The output device 14 is a device (a display, a speaker, etc.) for presenting information to a user.

[0017] The server 20 is, for example, an information processing device that manages information about customers.

[0018] The server 20 is realized by, for example, a computer connected to a network 80. As shown in Fig. 1, the server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29. The input / output IF 23 functions as an interface with an input device for receiving an input operation from a user and an output device for presenting information to the user.

[0019] Display 30 displays data according to the control of server 20. Display 30 is installed, for example, in a facility where a queue is formed, and displays an image of the facility's situation. Display 30 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0020] The photographing device 41 is a device for receiving light by a light receiving element and outputting the light as photographing data. The photographing device 41 is installed so as to be able to photograph a predetermined position in a facility with a predetermined angle of view, for example. Specifically, the photographing device 41 is installed so as to be able to photograph, for example, a line of people waiting for a predetermined process in the facility. The photographing device 41 outputs the photographing data to the server 20.

[0021] In an area where a queue may occur within a facility, a plurality of areas for generating object images using augmented reality are provided at predetermined intervals. Examples of the areas for generating object images using augmented reality are as follows: Areas with designated markers (floors, walls, designated objects, etc.) - Objects with preset shapes, colors, etc.

[0022] The marker may be a code in which identification information is written. For example, the marker may be realized by a barcode in which identification information is written, a QR code (registered trademark), or the like. Furthermore, the information written in the marker may be address information for reading information about the object from the server 20. The marker is not limited to being attached in one place, and multiple markers may be attached in one area.

[0023] In the area where a queue may occur, a mark for specifying the position of a customer may be provided separately from the area for generating an object image. For example, marks for forming queues at a predetermined interval are provided on the floor of the area where a queue may occur. When marks are provided on the floor, the area for generating an object image is disposed between the marks. In other words, the marks and the areas are disposed alternately.

[0024] Each information processing device is configured by a computer equipped with a calculation device and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the terminal device 10 and the server 20, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0025] <1.1 Terminal device configuration> Fig. 2 is a block diagram showing an example of the configuration of the terminal device 10 shown in Fig. 1. As shown in Fig. 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 160, a position information sensor 150, a storage unit 180, and a control unit 190. The blocks included in the terminal device 10 are electrically connected to each other, for example, by a bus or the like.

[0026] The communication unit 120 performs processing such as modulation and demodulation processing for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on a signal generated by the control unit 190 and transmits the signal to the outside (for example, the server 20). The communication unit 120 performs reception processing on a signal received from the outside and outputs the signal to the control unit 190.

[0027] The input device 13 is a device for inputting instructions or information by a user who operates the terminal device 10. The input device 13 is realized, for example, by a touch-sensitive device 131 or the like in which an instruction is input by touching an operation surface. In the case where the terminal device 10 is a PC or the like, the input device 13 may be realized by a reader, a keyboard, a mouse, or the like. The input device 13 converts an instruction input by a user into an electrical signal, and outputs the electrical signal to the control unit 190. Note that the input device 13 may include, for example, a receiving port that receives an electrical signal input from an external input device.

[0028] The output device 14 is a device for presenting information to a user who operates the terminal device 10. The output device 14 is realized, for example, by a display 141 or the like. The display 141 displays data according to the control of the control unit 190. The display 141 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display or the like.

[0029] The audio processing unit 17 performs, for example, digital-analog conversion processing of an audio signal. The audio processing unit 17 converts a signal provided from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is realized, for example, by a processor for audio processing. The microphone 171 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 17. The speaker 172 converts the audio signal provided from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0030] The camera 160 is a device for receiving light with a light receiving element and outputting the received light as an image capturing signal.

[0031] The position information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. In the satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10 equipped with the GPS module is detected based on the received signals. The position information sensor 150 may detect the current position of the terminal device 10 from the position of the wireless base station to which the terminal device 10 is connected.

[0032] The storage unit 180 is realized by, for example, the memory 15 and the storage 16, and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181, object information 182, and environmental information 183.

[0033] The user information 181 includes, for example, information about the user who uses the terminal device 10. Details will be described later.

[0034] The object information 182 includes information about objects to be generated in areas where columns can occur, as will be described in detail later.

[0035] The environmental information 183 includes information that represents the current environment. The information that represents the current environment includes, for example, information about the weather, atmospheric pressure, temperature, humidity, etc. Details will be described later.

[0036] The control unit 190 is realized by the processor 19 reading a program stored in the storage unit 180 and executing instructions included in the program. The control unit 190 controls the operation of the terminal device 10. The control unit 190 operates according to the program to fulfill the functions of an operation reception unit 191, a transmission / reception unit 192, an imaging control unit 193, an image analysis unit 194, and a display control unit 195.

[0037] The operation reception unit 191 performs processing for receiving instructions or information input from the input device 13. Specifically, for example, the operation reception unit 191 receives instructions or information input from the touch-sensitive device 131 or the like.

[0038] Furthermore, operation acceptance unit 191 accepts an image input from camera 160. Specifically, operation acceptance unit 191 receives image data captured by camera 160, for example.

[0039] Furthermore, the operation reception unit 191 receives audio information input from the microphone 171. Specifically, for example, the operation reception unit 191 receives audio data that is input from the microphone 171 and converted into digital data by the audio processing unit 17.

[0040] The transmitting / receiving unit 192 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 transmits instructions input by a user or various pieces of acquired information to the server 20. In addition, the transmitting / receiving unit 192 receives information provided by the server 20.

[0041] The photographing control unit 193 controls photographing by the camera 160. For example, the photographing control unit 193 starts photographing by the camera 160 in response to an instruction from a user.

[0042] The image analysis unit 194 controls the analysis of the photographed data captured by the camera 160. For example, the image analysis unit 194 detects a predetermined area included in the photographed data. Specifically, for example, the image analysis unit 194 detects an area included in the photographed data for generating an object image by augmented reality.

[0043] Also, for example, the image analysis unit 194 detects customers who are lined up and are included in the photographed data. For example, the image analysis unit 194 detects customers who stay at a position at a predetermined distance in a predetermined direction from the area for generating the object image for a predetermined time or more as customers who are lined up. Also, in a case where a mark for specifying the position of the customer is attached to the floor or the like, the image analysis unit 194 may detect customers who stay near the mark as customers who are lined up. The image analysis unit 194 may detect customers who are lined up at the end of the line.

[0044] The image analysis unit 194 may use the detection result of customers waiting in line when detecting an area for generating an object image by augmented reality. For example, the image analysis unit 194 may detect customers waiting in line and detect an area near the detected customers. Specifically, for example, the image analysis unit 194 may detect an area located a predetermined distance behind the customer at the end of the line in the direction in which the line is formed. Also, for example, the image analysis unit 194 may detect an area located between customers waiting in line.

[0045] The image analysis unit 194 may use any existing technology as a method for detecting an area, customers waiting in line, etc. The image analysis unit 194 may detect an area, customers waiting in line, etc. by using a trained model.

[0046] The display control unit 195 controls the output device 14 to display a predetermined image for the user. For example, the display control unit 195 generates an object image based on information detected from the shooting data, and displays it on the display 141. Specifically, for example, the display control unit 195 generates an object image corresponding to the shot area, and displays the object image at a position corresponding to the area. The display control unit 195 may generate different object images based on the following information, for example: -Past product purchase history (purchase log) -Specified dates (birthdays, anniversaries) · Number of past visits (visit log) Favorite Character Calendar (days of the week, holidays, etc.) ·weather

[0047] For example, the display control unit 195 displays the photographed area on the display 141 and superimposes the created object image on an area for generating an object image by augmented reality, which is included in the photographed area. In addition, in the case of a device such as an HMD in which an actual situation is viewed through a display, the display control unit 195 displays the object image at a position corresponding to the area for generating an object image by augmented reality.

[0048] Furthermore, the display control unit 195 generates a representation according to predetermined information. The predetermined information is, for example, the following. -Detection of new customers joining the queue -Detecting the movement of customers in line moving forward -Detecting when customers leave the queue Interference with the object image displayed on the display 141 (contact with the object image displayed on the display 141 via the touch-sensitive device 131, input of voice to the object image displayed on the display 141) -Past product purchase history -Specified dates (birthdays, anniversaries) Number of past visits Favorite Character Calendar (days of the week, holidays, etc.) ·weather

[0049] The configuration of the terminal device 10 is not limited to the above. The terminal device 10 may include a three-dimensional measurement unit. The three-dimensional measurement unit calculates the three-dimensional coordinates of the reflection point by, for example, irradiating a laser to a predetermined scanning area at a predetermined period and analyzing the reflected light. The three-dimensional measurement unit is realized by, for example, LiDAR (Light Detection And Ranging). The three-dimensional measurement unit is installed so as to be able to measure the same direction as the camera 160, for example. The three-dimensional measurement unit transmits data related to the calculated three-dimensional coordinates to the control unit 190 as point cloud data.

[0050] <1.2 Functional configuration of the server> 3 is a diagram showing an example of a functional configuration of the server 20. As shown in FIG. 3, the server 20 fulfills the functions of a communication unit 201, a storage unit 202, and a control unit 203.

[0051] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0052] The memory unit 202 includes, for example, a customer information table 2021, an object information table 2022, a position information table 2023, an environmental information table 2024, and a trained model 2025.

[0053] The customer information table 2021 is a table that stores information about registered customers.

[0054] The object information table 2022 is a table that stores information about objects to be generated in areas where columns can occur.

[0055] The position information table 2023 is a table that stores the current position of a customer. The current position of a customer is set based on a three-dimensional position measured by a three-dimensional measuring device installed in the facility, for example. The three-dimensional measuring device calculates the three-dimensional coordinates of the reflection point by, for example, irradiating a laser to a predetermined scanning area at a predetermined period and analyzing the reflected light. The three-dimensional measuring device is realized by, for example, LiDAR. The three-dimensional measuring device is installed, for example, at a position where the entire facility can be viewed without obstructions. The three-dimensional measuring device transmits data on the calculated three-dimensional coordinates to the server 20 as point cloud data. When a customer visits the facility and transmits the fact of his / her visit to the server 20 by operating the terminal device 10, the position at the time of registration is associated with the user's ID. The server 20 refers to the point cloud data and tracks the user's movement to grasp the current position of the customer.

[0056] The environment information table 2024 is a table that stores information related to the environment.

[0057] The trained model 2025 is, for example, a model for detecting a predetermined area from a captured image. The trained model 2025 is stored in advance when the server 20 provides a service. The trained model 2025 is generated by making a machine learning model perform machine learning according to a model learning program. The trained model 2025 is, for example, a parameterized composite function in which a plurality of functions are combined. The parameterized composite function is defined by a combination of a plurality of adjustable functions and parameters. The trained model 2025 according to this embodiment may be any parameterized composite function that satisfies the above requirements.

[0058] For example, when the trained model 2025 is generated using a forward propagation type multi-layer network, the parameterized composite function is defined as a combination of, for example, a linear relationship between each layer using a weight matrix, a nonlinear relationship (or a linear relationship) using an activation function in each layer, and a bias. The weight matrix and the bias are called parameters of the multi-layer network. The parameterized composite function changes its form as a function depending on how the parameters are selected. In the multi-layer network, by appropriately setting the constituent parameters, it is possible to define a function that can output a desirable result from the output layer.

[0059] As the multi-layered network according to the present embodiment, for example, a deep neural network (DNN) that is a multi-layered neural network that is the subject of deep learning may be used. As the DNN, for example, a convolution neural network (CNN) that targets images may be used.

[0060] The trained model 2025 is trained, for example, using an image including an area where a queue may occur as input data and using customers who line up in the queue as correct output data. The trained model 2025 may be retrained at any time according to the variation of customers who line up in the queue.

[0061] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. The control unit 203 performs functions as a reception control module 2031, a transmission control module 2032, an image analysis module 2033, and a display control module 2034 by operating according to the program.

[0062] The reception control module 2031 controls the process in which the server 20 receives a signal from an external device in accordance with a communication protocol.

[0063] The transmission control module 2032 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0064] The image analysis module 2033 controls the analysis of the photographed data photographed by the photographing device 41. For example, the image analysis module 2033 detects a predetermined area included in the photographed data. Specifically, for example, the image analysis module 2033 detects an area included in the photographed data for generating an object image by augmented reality.

[0065] Also, for example, the image analysis module 2033 detects customers who are lined up and are included in the captured data. For example, the image analysis module 2033 detects customers who stay at a position at a predetermined distance in a predetermined direction from the area for generating the object image for a predetermined time or more as customers who are lined up. Also, if a mark for specifying the position of the customer is attached to the floor or the like, the image analysis module 2033 may detect customers who stay near the mark as customers who are lined up.

[0066] The image analysis module 2033 may use the detection result of the customers waiting in line when detecting the area for generating an object image by augmented reality. For example, the image analysis module 2033 may detect the customers waiting in line and detect the area near the detected customers. Specifically, for example, the image analysis module 2033 may detect the area located a predetermined distance behind the customers waiting at the end of the line in the formation direction of the line. Also, for example, the image analysis module 2033 may detect the area located between the customers waiting in line.

[0067] The image analysis module 2033 may use any existing technology as a method for detecting an area or customers waiting in line. The image analysis module 2033 may detect an area using a trained model.

[0068] The display control module 2034 controls the display 30 to display a predetermined image to the customer. For example, the display control module 2034 generates an object image based on information detected from the shooting data, and displays it on the display 30. Specifically, for example, the display control module 2034 generates an object image corresponding to the photographed area, and displays the object image at a position corresponding to the area. may generate a different object image based on, for example, predetermined information. The predetermined information is, for example, the same as the information described in the display control unit 195. For example, the display control module 2034 displays the photographed area on the display 30, and superimposes the created object image on an area for generating an object image by augmented reality, which is included in the photographed area.

[0069] Furthermore, the display control module 2034 generates a representation according to predetermined information. The predetermined information is, for example, the same as the information described in the display control unit 195.

[0070] <2 Data Structure> 4 to 6 are diagrams showing data structures of information stored in the terminal device 10. Note that, Figs. 4 to 6 are merely examples, and do not exclude data that is not shown.

[0071] Fig. 4 is a diagram showing the data structure of user information 181. The user information 181 shown in Fig. 4 has columns such as name, age, address, date of birth, e-mail address, date of registration, store visit information, purchase information, favorite character, generated object, etc., with a user ID as a key.

[0072] The user ID is an item that stores an identifier for uniquely identifying a user. The user ID may be an ID that was set when the user became a member of a store. The name is an item that stores the name of the user. The age is an item that stores the age of the user. The address is an item that stores the place where the user lives. The date of birth is an item that stores the date the user was born. The email address is an item that stores the email address of the user. The registration date is an item that stores the date the user registered as a member.

[0073] The store visit information is an item that stores the history of a user's visits to a facility. If the user has visited the facility multiple times, the store visit information includes the date and time of each visit. If there are multiple related facilities, the store visit information may store the date and time of the visit in association with the name of the facility visited. The purchase information is an item that stores the history of products purchased by the user. The purchase information includes, for example, the purchase date and time, the product ID of the purchased product, the product name of the purchased product, etc. The favorite character is an item that stores the object ID of a character that the user has registered as a favorite.

[0074] Fig. 5 is a diagram showing the data structure of object information 182. Object information 182 shown in Fig. 5 is a table having columns such as object name, generation condition, object data, performance content, and requirements corresponding to the performance content, with object ID as a key.

[0075] The object ID is an item that stores an identifier for uniquely identifying an object. The object name is an item that stores the name of an object. The generation condition is an item that stores the condition for generating an object image. The condition for generating an object image includes, for example, the following: - You have previously purchased the specified product - You have previously purchased the specified type of product - The number of purchases of the product has exceeded the specified number of times. - The purchase amount of the product exceeds a certain amount. -The current date must match the user's birthday. The current date matches a specific date (e.g. Christmas, Marine Day, New Year's Day, Coming of Age Day, etc.) The current day is a specific day (e.g., Friday). - The number of past visits has exceeded the specified number. The set object (character) matches your favorite character - The weather is as specified (e.g., snow). -Identification information contained in the marker

[0076] If the marker includes identification information, the identification number may be stored in the item: "occurrence condition."

[0077] Object data is an item that stores image data about an object. Object data may also store reference information (paths) to object data files located elsewhere. Image data about an object may include, for example: -Image data of the initial state of a specified character Image data of the evolved state of a specific character Image data of a specific character wearing special clothing Image data containing multiple specified characters

[0078] The performance content is an item that stores image data about the performance that the object performs. The performance content may store reference information (path) to an image data file located in another location. The performance content includes, for example, the following: -Performances where objects perform specific actions (birthdays, celebrations of specific dates, etc.) -Standing behind new customers in line -The stage moves forward in tandem with the customers who have advanced in line. · Stepping out of line -Evolving objects -The object will wear a designated accessory.

[0079] In the example shown in Fig. 5, the content of the effect includes a first effect, a second effect, etc. The requirement is an item for storing a requirement for making the object perform the effect. The requirement includes, for example, the following. -Detection of new customers joining the queue -Detecting the movement of customers in line moving forward -Detecting when customers leave the queue Touching an object image displayed on a display 141 via a touch sensitive device 131 -Prescribed voice input - You have previously purchased the specified product - You have previously purchased the specified type of product - The number of purchases of the product has exceeded the specified number of times. - The purchase amount of the product exceeds a certain amount. -The current date must match the user's birthday. The current date matches a specific date (e.g. Christmas, Marine Day, New Year's Day, Coming of Age Day, etc.) The current day is a specific day (e.g., Friday). - The number of past visits has exceeded the specified number. The set object (character) matches your favorite character - The weather is as specified (e.g., snow).

[0080] The example shown in FIG. 5 includes a first requirement for implementing a first performance, a second requirement for implementing a second performance, and so on.

[0081] Fig. 6 is a diagram showing the data structure of environmental information 183. Environmental information 183 shown in Fig. 6 is a table having columns for date, weather, atmospheric pressure, temperature, and humidity. Environmental information 183 may include information related to the calendar. The information related to the calendar includes, for example, information related to days of the week and information related to holidays.

[0082] The date is an item for storing the date on which the environmental information was acquired. The weather, atmospheric pressure, temperature, and humidity are examples of the environmental information, and the information provided by a predetermined information source is stored.

[0083] <3 operations> The operation of the terminal device 10 and the server 20 when displaying object images to customers to make the act of queuing more interesting will be described. (Display of Object Image by Terminal Device 10)

[0084] In the following description, for example, a situation is assumed in which a customer who visits a certain store places a product he or she wants to purchase in a shopping cart and waits at a cash register to pay for the product. It is also assumed that the terminal device 10 used by the customer is a terminal with a display, such as a smartphone or a tablet terminal.

[0085] FIG. 7 is a schematic diagram showing an area around a cash register counter in a store according to this embodiment. In FIG. 7, marks 51 are provided on the floor of the store to help customers waiting for their turn at the cash register counter. The marks 51 may be referred to as marks for forming a queue. The marks 51 may be physical marks such as stickers or paint, or may be optical marks with a predetermined shape projected onto them. The marks 51 are provided at intervals that prevent droplets from scattering when people are having a normal conversation, for example, at intervals of about 2 m. The marks 51 may be arranged in a straight line or in a curved line.

[0086] Markers 52 are provided between the marks 51. The markers 52 are arranged at a predetermined distance and in a predetermined direction relative to the marks 51. The markers 52 may be physical, such as stickers or paint, or may be optical, with a predetermined shape projected onto them. Note that as long as the markers 52 are provided, the marks 51 do not necessarily have to be provided.

[0087] FIG. 8 is a flowchart showing an example of the operation of the terminal device 10 when generating an object image in a column.

[0088] First, the user operates the terminal device 10 to capture an image of the area around the checkout counter.

[0089] In step S11, the control unit 190 controls the camera 160 via the image capture control unit 193 to capture a video of an area where a queue may form, for example, an area around a checkout counter.

[0090] In step S12, the control unit 190 analyzes the captured image data by the image analysis unit 194. For example, the image analysis unit 194 detects the marker 52, which is an area for generating an object image by augmented reality, included in the captured image data. Also, when customers are lined up in front of the cash register counter, the image analysis unit 194 detects the customers waiting in line included in the captured image data.

[0091] In step S13, the control unit 190 causes the display control unit 195 to display an image on the display 141. For example, the display control unit 195 generates an object image based on information detected from the shooting data, and causes the display 141 to display the generated object image together with the shooting data.

[0092] (Example of object image display) FIG. 9 is a flowchart showing a specific example of the operation of the display control unit 195 according to the present embodiment when displaying an object image.

[0093] In step S21, the display control unit 195 determines whether or not a marker 52 located immediately after the customer at the end of the line or a marker 52 located between customers in the line has been detected. If a marker 52 located immediately after the customer at the end of the line or a marker 52 located between customers in the line has been detected (Yes in step S21), the display control unit 195 shifts the process to step S22. If a marker 52 located immediately after the customer at the end of the line or a marker 52 located between customers in the line has not been detected (No in step S21), the display control unit 195 shifts the process to step S24.

[0094] In step S22, the display control unit 195 creates an object image for each marker 52 detected in step S21. Specifically, the display control unit 195 acquires, for example, information related to the item "occurrence condition" of the object information 182 from the user information 181 and the environmental information 183. The display control unit 195 selects a record that satisfies the occurrence condition based on the acquired information. The display control unit 195 acquires image data based on data stored in the item "object data" of the selected record. When there are multiple objects that satisfy the occurrence condition, the display control unit 195 selects a record based on, for example, a priority that is set in advance. The display control unit 195 may select different objects for each detected marker 52, or may select the same object.

[0095] In step S23, the display control unit 195 displays an object image on the display 141. Specifically, the display control unit 195 superimposes an object image based on the acquired image data on the marker 52 detected in step S21. As a result, the object image is displayed on the marker 52 located immediately behind the customer at the end of the line and on the markers 52 located between the customers in the line.

[0096] Fig. 10 is a schematic diagram showing a display example of the display 141 according to this embodiment. In Fig. 10, the image analysis unit 194 detects the customer 100 who remains within the mark 51 as a customer who is waiting in line. Then, the image analysis unit 194 detects the marker 52 located immediately behind the customer 100. The display control unit 195 superimposes the object image 53 on the detected marker 52. In the example shown in Fig. 10, a character who is waiting in line with the customer and facing away is displayed as the object image 53. That is, the object image is displayed behind the customer.

[0097] In step S24, the display control unit 195 determines whether or not an event has occurred that satisfies the requirements for effect generation. Specifically, for example, the display control unit 195 determines whether or not an event has occurred that satisfies the item: "requirements" of the object information 182. If an event has occurred that satisfies the requirements for effect generation, the display control unit 195 proceeds to step S25.

[0098] In step S25, the display control unit 195 generates a performance according to the event. Specifically, for example, the display control unit 195 generates a performance stored in the item: "performance" corresponding to the requirement stored in the item: "requirement".

[0099] Specifically, for example, in step S24, when a new customer is detected lining up, the display control unit 195 determines that an event that satisfies the requirements for generating a performance has occurred. Then, in step S25, the display control unit 195 acquires image data as shown in step S22, and moves an object image based on the acquired image data toward the marker 52.

[0100] 11 and 12 are schematic diagrams showing a display example of the display 141 according to this embodiment. In Fig. 11 and Fig. 12, the image analysis unit 194 detects a customer 100 who has moved into the mark 51. Then, the image analysis unit 194 detects a marker 52 located just behind the customer 100. The display control unit 195 moves the object image 53 to the detected marker 52 so that it joins the line. The display control unit 195 may cause the object image 53, which has completed its movement, to perform a performance such as a greeting.

[0101] As shown in Fig. 12, when a new object image is displayed in addition to an object image that is already displayed, the newly displayed object image may be the same character as the object image that is already displayed, or may be a different character. Fig. 12 shows an example in which the character is the same.

[0102] Also, for example, in step S24, the display control unit 195 determines that an event that satisfies the requirements for generating a performance has occurred when the object image displayed on the display 141 is touched by the user via the touch-sensitive device 131. Then, in step S25, the display control unit 195 displays a predetermined performance for the touched object image.

[0103] 13 and 14 are schematic diagrams showing a display example of the display 141 according to this embodiment. In FIG. 13, the displayed object image 53 is poked by the user. The hand in FIG. 13 is a schematic representation of the user's contact with the object image 53. As shown in FIG. 14, the display control unit 195 produces an effect in which the object image 53 turns around and takes a pose. Note that the action in response to the user's interference is not limited to a predetermined pose, and a predetermined action may be performed, or a predetermined technique may be performed.

[0104] Also, for example, in step S24, when an object image is displayed at the end of the line and a customer in line moves forward toward the mark 51 on the cash register counter side, the display control unit 195 determines that an event that satisfies the requirements for generating a performance has occurred. Then, in step S25, the display control unit 195 moves the displayed object image toward the next marker 52 in the direction the customer moved.

[0105] Fig. 15 is a schematic diagram showing a display example of the display 141 according to this embodiment. For example, assume that the first customer in the state shown in Fig. 13 has finished the transaction, and the customer 100 in line has moved forward as shown in Fig. 15. The image analysis unit 194 detects the customer 100 who has moved to the mark 51, which is one step forward toward the cash register counter. The image analysis unit 194 then detects the marker 52 located immediately behind the moved customer 100. The display control unit 195 moves the object image 53 from the marker 52 on which the object image 53 was superimposed in Fig. 13 to the marker 52 which has moved one step toward the cash register counter.

[0106] Also, for example, in step S24, when an object image is displayed at the end of the line and a customer standing at the end of the line leaves the line, the display control unit 195 determines that an event that satisfies the requirements for generating a performance has occurred. Then, in step S25, the display control unit 195 moves the displayed object image so that it leaves the line.

[0107] Fig. 16 is a schematic diagram showing a display example of the display 141 according to this embodiment. For example, assume that the customer 100 at the end of the line in the state shown in Fig. 13 falls out of line as shown in Fig. 16. The image analysis unit 194 detects the customer 100 who has fallen out of line and moved away from the mark 51. The display control unit 195 moves the object image 53 from the marker 52 on which the object image 53 was superimposed in Fig. 13 in a direction away from the line.

[0108] (Display of object image by server 20) Next, a case will be described in which the state inside a store is photographed by an imaging device 41 installed inside the store and displayed on the display 30. In the following explanation, it is assumed that a customer visits a specific store, places the products he or she wants to purchase in a shopping cart, and waits at the cash register to pay for the purchase.

[0109] FIG. 17 is a flowchart showing an example of the operation of the server 20 when generating an object image in a sequence.

[0110] First, the image capturing device 41 installed in the store captures a video of the area around the cash register counter.

[0111] In step S 31 , the control unit 203 acquires, via the reception control module 2031 , image data captured by the image capturing device 41 .

[0112] In step S32, the control unit 203 analyzes the captured image data by the image analysis module 2033. For example, the image analysis module 2033 detects a marker 52, which is an area for generating an object image by augmented reality, included in the captured image data. Also, when customers are lined up in front of the cash register counter, the image analysis module 2033 detects the customers waiting in line included in the captured image data.

[0113] In step S33, the control unit 203 causes the display control module 2034 to display an image on the display 30. For example, the display control module 2034 generates an object image based on information detected from the shooting data, and displays the generated object image on the display 30 together with the shooting data.

[0114] Fig. 18 is a schematic diagram showing a display example of the display 30 according to this embodiment. In Fig. 18, the image analysis module 2033 detects customers who remain within the mark 51 as customers standing in line. Then, the image analysis module 2033 detects a marker 52 located immediately behind the customer standing at the end of the line and a marker 52 located between the customers standing in line. The display control module 2034 superimposes an object image 53 on the detected marker 52. In the example shown in Fig. 18, a character standing in line with the customer is displayed as the object image 53.

[0115] As described above, in the above embodiment, a plurality of second areas for generating an object image by augmented reality are arranged at predetermined intervals in a first area where a queue may be formed. The control unit 190 acquires photographed data of the first area. The control unit 190 detects the second area based on the acquired photographed data by the image analysis unit 194. The control unit 190 creates an object image corresponding to the detected second area by the display control unit 195. The display control unit 195 displays the created object image at a position corresponding to the second area. This allows the terminal device 10 or the server 20 according to this embodiment to generate an object by augmented reality in the queue, making it possible for customers waiting in line to enjoy the act of waiting.

[0116] Therefore, according to this embodiment, it is possible to increase the interest of waiting in line.

[0117] In the above embodiment, the display control unit 195 displays an image based on the shooting data, and displays the object image superimposed at a position corresponding to the second region in the image. This allows the user to easily view the object image generated in the line by holding a smartphone, tablet terminal, or the like over the region where a line may be formed.

[0118] In the above embodiment, the image analysis unit 194 detects people in a line and detects the second region using the detection result of the people. The display control unit 195 displays an object image at a position corresponding to the second region behind the person in the formation direction of the line. This results in an object image being generated behind the people in the line. In other words, people and objects appear to be lined up alternately. It becomes possible to see a line that does not actually exist, and the user can enjoy the action of lining up even more. Also, when multiple people are lined up, multiple objects are generated, and the user can enjoy the action of lining up even more.

[0119] In the above embodiment, the image analysis unit 194 detects the person at the end of the line, and detects the second area using the detection result of the last person. The display control unit 195 displays the object image behind the last person. This makes the object appear to be at the end, and the user will line up next to the object. By feeling the object close to the user, the action of lining up can be more enjoyable.

[0120] In the above embodiment, the display control unit 195 creates an object image based on the user information. This makes it possible to generate a different object every time the user visits the store. This increases the user's enjoyment of visiting the store, and motivates the user to visit the store again. The user information includes, for example, a store visit log, a purchase log, special days, or information about favorites.

[0121] In the above embodiment, the display control unit 195 creates an object image based on environmental information. This makes it possible to generate different objects according to the environment. This increases the user's enjoyment of visiting the store, which motivates the user to visit the store again. The environmental information includes, for example, information about the calendar or weather.

[0122] In the above embodiment, the display control unit 195 judges whether the requirements for effect generation are satisfied. The display control unit 195 causes the object image to perform an effect according to the satisfied requirements. This allows the object to perform an effect according to the requirements, so that the user can enjoy the lining up action more.

[0123] In the above embodiment, the display control unit 195 detects a person who has joined the queue. The display control unit 195 causes the object image to perform a performance of joining the queue. As a result, when a new customer joins the queue, the number of objects increases, further improving the interest.

[0124] In the above embodiment, the display control unit 195 detects a person moving forward along the line. The display control unit 195 causes the object image to perform an effect of moving forward along the line. This allows the object to behave distinctively when the line moves forward, so that the user can look forward to the line moving forward.

[0125] In the above embodiment, the display control unit 195 detects a person who has left the queue. The display control unit 195 causes the object image to perform a performance of leaving the queue. As a result, when a customer leaves the queue, the object behaves in a distinctive manner, so that the user can look forward to the action of the customer leaving the queue.

[0126] In the above embodiment, the display control unit 195 detects interference with the object image. The display control unit 195 causes the object image to perform an action according to the interference. This allows the object to operate due to the user's interference even if there is no change in the line, so that the user can enjoy the act of waiting even if there is no change in the line.

[0127] <Modification> In the above embodiment, when a certain condition is satisfied, an object image according to the condition is displayed, and when a certain requirement is satisfied, the object image performs a performance according to the requirement. At this time, it is not displayed what condition is satisfied to display the object image, or what requirement is satisfied to cause the object image to perform a certain performance.

[0128] However, it may be clearly stated what conditions are satisfied to display the object image. It may also be clearly stated what requirements are satisfied to cause the object image to perform a predetermined effect. For example, a statement such as "It occurred because the 'predetermined action' was cleared!" may be displayed in a manner that does not interfere with the display of the object image, for example, in a temporary pop-up. This makes it possible to understand the conditions for the generation of a characteristic object or the requirements for the generation of a characteristic effect, and allows the user to consciously reproduce the characteristic object or the characteristic effect. This allows the user to adjust his or her own actions in order to generate an object or effect that he or she desires.

[0129] Also, in the above embodiment, no hint about the next object to be displayed is given. However, a hint about the next object to be displayed may be given. For example, a statement such as "Next time, if you perform a 'predetermined action', a 'predetermined object or a predetermined effect' will occur!" may be displayed in a manner that does not interfere with the display of the object image, for example, as a temporary pop-up. Note that an expression that hides the occurrence may also be used. This allows the user to know the object or effect that may occur the next time, which makes the user look forward to the next visit.

[0130] In the above embodiment, the area for generating the object image may be installed at the checkout counter. The marker as the area placed at the checkout counter may include a predetermined code. The predetermined code is, for example, a code that is individually assigned to the store clerk who performs the transaction, and is a code for generating an object assigned to the store clerk. As a result, an object specific to the store clerk is generated at the head of the queue, so that the user can wait in line in anticipation of what the store clerk's object will be. In other words, the user will have more fun waiting in line.

[0131] In the above embodiment, the image analysis unit 194 or the image analysis module 2033 detects customers who are waiting in line. However, detection of customers who are waiting in line is not limited to the image analysis unit 194 or the image analysis module 2033. The control unit 190 may function as a queue state recognition unit by operating according to a program. Furthermore, the control unit 203 may function as a queue state recognition module by operating according to a program. The queue state recognition unit detects customers who are waiting in line, for example, based on point cloud data measured by a three-dimensional measurement unit. Furthermore, the queue state recognition module detects customers who are waiting in line, for example, based on point cloud data measured by a three-dimensional measurement module.

[0132] The queue state recognition unit or the queue state recognition module may use any existing technology to detect customers, etc., waiting in line.

[0133] In the above embodiment, for example, as shown in FIG. 10, a case where another customer is lined up behind a customer who is currently paying is described as an example. However, a character may be lined up immediately behind the customer who is currently paying. That is, for example, as shown in FIG. 19, an area for generating an object image by augmented reality may be provided between the cash register counter and the position where the customer at the head of the line is waiting. As a result, for example, when a customer is currently paying at the cash register counter, an object image is generated between the customer who is currently paying and the customer waiting at the head of the line. Also, as shown in FIG. 20, a situation may occur where only the object image is lined up behind the customer who is currently paying. This makes it possible for the customer to easily recognize where the line starts.

[0134] Also, for example, when a transaction is completed at a checkout counter and a customer moves away from the checkout counter, the object image may guide the customer at the front of the line. Specifically, for example, when a transaction is completed at a checkout counter and a customer moves away from the checkout counter, the image analysis unit 194 or the image analysis module 2033 detects that the customer has left the checkout counter. The display control unit 195 or the display control module 2034 causes the object image superimposed on the marker just before the checkout counter to guide the customer at the front of the line. FIG. 21 is a schematic diagram showing a display example of the display 141 according to this embodiment.

[0135] <4 Basic hardware configuration of computer> 22 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 91, a main storage device 92, an auxiliary storage device 93, and a communication IF (interface) 99. These are electrically connected to each other by a bus.

[0136] The processor 91 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0137] The main storage device 92 is for temporarily storing programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0138] The auxiliary storage device 93 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0139] The communication IF 99 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard.

[0140] The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, 5G mobile communication systems, LTE (Long Term Evolution), wireless networks that can connect to the Internet via a specified access point (e.g., Wi-Fi (registered trademark)), etc. In the case of wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), Bluetooth (registered trademark), etc. In the case of wired connection, the network also includes a network that is directly connected by a USB (Universal Serial Bus) cable or the like.

[0141] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration among multiple computers 90 and connecting them together via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0142] <Basic functional configuration of computer 90> A description will now be given of the functional configuration of a computer realized by the basic hardware configuration of a computer 90 shown in Fig. 22. The computer includes at least the functional units of a control unit, a storage unit, and a communication unit.

[0143] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 connected to each other via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0144] The control unit is realized by the processor 91 reading out various programs stored in the auxiliary storage device 93, expanding the programs in the main storage device 92, and executing processes according to the programs. The control unit can realize functional units that perform various information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0145] The storage unit is realized by a main storage device 92 and an auxiliary storage device 93. The storage unit stores data, various programs, and various databases. Furthermore, the processor 91 can secure a storage area corresponding to the storage unit in the main storage device 92 or the auxiliary storage device 93 in accordance with a program. Furthermore, the control unit can cause the processor 91 to execute processes of adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0146] A database refers to a relational database, which is used to manage data sets called tables, which are structured according to rows and columns, by relating them to each other. In a database, a table is called a table, a column in a table is called a column, and a row in a table is called a record. In a relational database, it is possible to set relationships between tables and associate them.

[0147] Usually, a column is set in each table as a key for uniquely identifying a record, but setting a key in the column is not essential. The control unit can cause the processor 91 to add, delete, or update records in a specific table stored in the storage unit according to various programs.

[0148] The communication unit is realized by the communication IF 99. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 91 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0149] Although several embodiments of the present disclosure have been described above, these embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are within the scope of the invention and its equivalents as described in the claims, as well as the scope and spirit of the invention.

[0150] <Additional Notes> The matters described in the above embodiments will be supplemented below. (Appendix 1) A program to be executed by a computer having a processor and a memory, the program causing the processor to execute the following steps: in a first area where customers may form a queue, multiple second areas for generating object images using augmented reality are arranged at predetermined intervals in a first area; acquiring photographic data of the first area; detecting the second area based on the acquired photographic data; creating an object image corresponding to the detected second area; and displaying the created object image in a position corresponding to the second area. (Appendix 2) The program according to claim 1, wherein in the step of displaying the object image, an image based on the shooting data is displayed, and the object image is superimposed and displayed at a position in the image corresponding to the second region. (Appendix 3) A program as described in (Appendix 1) or (Appendix 2), in which in the detection step, customers standing in line are detected, a second area is detected using the customer detection results, and in the object image display step, an object image is displayed at a position corresponding to the second area behind the customer in the direction of queue formation. (Appendix 4) A program described in Appendix 3, in which, in the detection step, a customer who is at the end of the line is detected, and a second area is detected using the detection result of the last customer, and, in the object image display step, the object image is displayed behind the last customer. (Appendix 5) The program according to any one of (Supplementary Note 1) to (Supplementary Note 4), wherein in the step of creating an object image, the object image is created based on user information. (Appendix 6) The program according to claim 5, wherein in the step of creating an object image, the object image is created based on information regarding a store visit log, a purchase log, a special day, or favorites included in the user information. (Appendix 7) The program according to any one of (Supplementary Note 1) to (Supplementary Note 4), wherein in the step of creating an object image, the object image is created based on environmental information. (Appendix 8) The program according to claim 7, wherein in the step of creating an object image, the object image is created based on information regarding a calendar or weather included in the environmental information. (Appendix 9) A program described in any of (Appendix 1) to (Appendix 8), which causes a processor to execute a step of determining whether the requirements for generating an effect are met, and in a step of displaying an object image, causes the object image to perform an effect according to the met requirements. (Appendix 10) The program according to claim 9, wherein in the determining step, a customer who has joined the queue is detected, and in the displaying step, the object image is made to perform an effect of joining the queue. (Appendix 11) The program according to claim 9, wherein in the determining step, a customer moving forward in line is detected, and in the displaying step, the object image is made to perform an effect of moving forward in line (Appendix 9). (Appendix 12) The program according to claim 9, wherein in the determining step, a customer who has fallen out of line is detected, and in the displaying step, the object image is made to perform an effect of falling out of line. (Appendix 13) The program according to claim 9, wherein in the determining step, interference with the object image is detected, and in the displaying step, the object image is made to perform an operation corresponding to the interference. (Appendix 14) A method executed on a computer having a processor and a memory, wherein the processor executes the steps of: acquiring photographic data of a first area in which a queue of customers may be formed, and a plurality of second areas for generating augmented reality object images arranged at predetermined intervals; detecting the second area based on the acquired photographic data; creating an object image corresponding to the detected second area; and displaying the created object image at a position corresponding to the second area. (Appendix 15) An information processing device comprising a control unit and a memory unit, wherein the control unit executes the steps of: a first area in which a queue of customers may form, and a plurality of second areas for generating object images using augmented reality are arranged at predetermined intervals; acquiring photographic data captured of the first area; detecting the second area based on the acquired photographic data; creating an object image corresponding to the detected second area; and displaying the created object image at a position corresponding to the second area. (Appendix 16) A system comprising a first area where customers may form a queue, and a second area for generating augmented reality object images, the second area being arranged at predetermined intervals within the first area. (Appendix 17) A system as described in appendix 16, in which a mark indicating the position where the customer will stop is placed in the first area separately from the second area. (Appendix 18) The system described in claim 17, wherein the markings and the second region are arranged alternately in the first region. (Appendix 19) A system as described in any one of (Appendix 16) to (Appendix 18), comprising: a means for acquiring shooting data captured of a first area; a means for detecting a second area based on the acquired shooting data; a means for creating an object image corresponding to the detected second area; and a means for displaying the created object image at a position corresponding to the second area. [Explanation of symbols]

[0151] 1. System 10...Terminal device 12…Communication Interface 120…Communications Department 13...Input device 131...Touch-sensitive devices 14...Output device 141…Display 15…Memory 150...Location information sensor 16…Storage 160…Camera 17...Audio processing unit 171…Mike 172…Speaker 180...Storage section 19…Processor 190...Control unit 20…Server

Claims

1. A program for causing a computer having a processor and a memory to execute the program, the program causing the processor to: acquiring photographed data of a first area in the store where customers may form a queue and a second area where customers may make a payment; Detecting the movement of customers in the second area; displaying, in an area between the first area and the second area, an object image that moves from the first area to the second area in accordance with the detected movement of the customer in the second area; A program that executes the following.

2. In the detecting step, it is detected that a customer has disappeared from the second area, The program according to claim 1, wherein, in the display step, if there are no customers in the second area, the object image is displayed to guide customers in the first area to the second area.

3. A method executed on a computer having a processor and a memory, wherein the processor executes all of the steps performed in the invention of claim 1 or claim 2.

4. An information processing device comprising a processor and a memory, wherein the processor executes all of the steps executed in the invention according to claim 1 or claim 2.

5. A system having means for executing all steps performed in the invention of claim 1 or claim 2.

Images