Information processing device, information processing method, and recording medium

Abstract

This information processing device comprises a position information acquisition unit and a display control unit. The position information acquisition unit acquires position information. The position information indicates the in-store position of a container that moves together with a customer in a store and accommodates goods that the customer intends to purchase. The display control unit uses the position information to carry out control related to the display of an image generated by an imaging unit that is mounted on the container and takes an image of the surroundings of the container.

Description

Information processing device, information processing method, and recording medium

[0001] The present invention relates to an information processing device, an information processing method, and a recording medium.

[0002] In recent years, services related to a store have been provided using images of the interior of the store. For example, Patent Document 1 discloses a display control device that can easily reflect changes in the actual store in a virtual store. This display control device detects changes in the actual store based on image data obtained by an image capture device moving around the actual store, and changes the content of the display screen so that the parts where the changes were detected are displayed. Patent Document 1 describes that this image capture device is installed in a shopping cart or shopping basket.

[0003] Japanese Patent Application Laid-Open No. 2021-131770

[0004] One way to use images taken inside a store is to monitor the inside of the store. One possible way is to install an imaging device in a container that moves with a customer, such as a shopping cart or basket, and use images generated by this imaging device to monitor the inside of the store. However, since there are often multiple customers inside a store, it is expected that multiple imaging devices will generate images simultaneously. Therefore, in order to efficiently monitor the inside of a store, it is necessary to devise a display format for the images generated by the multiple imaging devices.

[0005] An information processing device according to one aspect of the present disclosure includes: a location information acquisition means that moves with a customer within a store and acquires location information indicating the location within the store of a container that contains a product that the customer plans to purchase; and a display control means that uses the location information to control the display of an image generated by an imaging means that is attached to the container and captures an image of the area around the container.

[0006] An information processing method in one aspect of the present disclosure includes a computer that moves with a customer within a store, acquires location information indicating the location within the store of a container that contains a product that the customer plans to purchase, and uses the location information to control the display of an image generated by an imaging means attached to the container that captures an image of the area around the container.

[0007] A recording medium according to one aspect of the present disclosure has recorded thereon a program that causes a computer to have: a location information acquisition means that moves with a customer within a store and acquires location information indicating the location within the store of a container that contains a product that the customer plans to purchase; and a display control means that uses the location information to control the display of an image generated by an imaging means that is attached to the container and captures an image of the area around the container.

[0008] According to an example of the present disclosure, the inside of a store can be monitored efficiently.

[0009] 1 is a diagram illustrating an example of a usage environment of an information processing device according to the present disclosure; FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device; FIG. 3 is a diagram illustrating an example of a screen displayed on a display unit; FIG. 4 is a diagram illustrating a blind spot, which is an example of a predetermined area; FIG. 5 is a diagram illustrating an example of a hardware configuration of an information processing device; FIG. 6 is a flowchart illustrating an example of processing performed by an information processing device; FIG. 7 is a diagram illustrating an example of a functional configuration of an information processing device;

[0010] Hereinafter, in this disclosure, the drawings relate to one or more embodiments. In addition, in all drawings, similar components are given similar reference numerals and descriptions thereof will be omitted as appropriate.

[0011] As shown in FIG. 1, the information processing device 10 is used together with an imaging unit 20 and a display unit 30 , and performs processing for displaying an image captured by the imaging unit 20 on the display unit 30 .

[0012] The imaging unit 20 moves around the store together with the customer. For example, the imaging unit 20 is attached to a container, such as a cart or shopping basket, that contains the product that the customer intends to purchase. The imaging unit 20 is attached to the container so that the imaging range includes the periphery of the container, for example, at least the sides of the container. Furthermore, if the imaging range of the imaging unit 20 is wide-angle, the center of the imaging range may be the inside of the container. Furthermore, multiple imaging units 20 facing in different directions may be attached to the container.

[0013] The imaging unit 20 also has a communication function. For example, the imaging unit 20 transmits generated images to the information processing device 10 in real time.

[0014] Generally, a single store has a plurality of containers available, and therefore the information processing device 10 is often used together with a plurality of image capturing units 20.

[0015] The display unit 30 is viewed by, for example, a person monitoring the sales floor of a store, such as a store clerk or a supervisor. The display unit 30 may be, for example, a display placed in a room where the store clerk or supervisor is located, or a display on a portable communication device carried by the store clerk or supervisor. The display unit 30 may also be a display installed in a location in the store where customers pass through, such as a display installed at the exit or entrance of the store or on the sales floor.

[0016] When there are multiple customers in a store, each of the multiple imaging units 20 generates images while moving around the sales floor of the store and transmits them in real time to the information processing device 10. The information processing device 10 uses the positions of each of the multiple imaging units 20 to perform real-time control to display the images generated by the imaging units 20 on the display unit 30. Therefore, the information processing device 10 can perform control to display the images generated by the multiple imaging units 20 on the display unit 30 so as to efficiently monitor the sales floor of the store.

[0017] For example, the information processing device 10 may display on the display unit 30 an image of the imaging unit 20 that is capturing an image of a place that particularly requires monitoring, but may not display images of the other imaging units 20. The information processing device 10 may also display on the display unit 30 an image of the imaging unit 20 that is capturing an image of a place that particularly requires monitoring larger than images captured by the other imaging units 20. The information processing device 10 may also display on the display unit 30 an image of the imaging unit 20 that is capturing an image of a place that particularly requires monitoring longer than images captured by the other imaging units 20.

[0018] The time from when the imaging unit 20 generates an image until the image is displayed on the display unit 30 is, for example, 5 seconds or less, preferably 2 seconds or less, and more preferably 1 second or less.

[0019] There are various methods for identifying the position of the imaging unit 20 in the sales floor of the store, i.e., the position of the container held by the customer. For example, if multiple beacons are installed in the sales floor of the store, the imaging unit 20 may transmit information indicating which beacons are able to communicate to the information processing device 10 as location information indicating the location of the imaging unit 20. Alternatively, the imaging unit 20 may generate location information for the imaging unit 20 using, for example, a GPS, and transmit this location information to the information processing device 10.

[0020] 2 , the information processing device 10 has a location information acquisition unit 110 and a display control unit 120, and can use a storage unit 130. The storage unit 130 stores various types of information used by the information processing device 10. The storage unit 130 may be a part of the information processing device 10, or may be located outside the information processing device 10.

[0021] The location information acquisition unit 110 acquires location information indicating the location within the store of a container, such as a cart or shopping basket, that contains products that a customer plans to purchase. The location information is generated, for example, by the imaging unit 20. In this case, the location information acquisition unit 110 acquires the location information from the imaging unit 20.

[0022] The display control unit 120 uses the position information to control the display of the image generated by the imaging unit 20. For example, the display control unit 120 performs at least one of a determination process to determine whether or not to display the image on the display unit 30, and a determination process to determine at least one of the display position and size of the image on the display unit 30. This allows images whose shooting position satisfies certain conditions to be displayed on the display unit 30 in a manner that is easy for people to recognize.

[0023] For example, the display control unit 120 determines whether the location indicated by the location information is included in a predetermined area of ​​the store, and controls the display of images using the determination result.

[0024] For example, the display control unit 120 may cause the display unit 30 to display an image generated by the imaging unit 20 corresponding to the position information only when the position indicated by the position information is included in the predetermined area. In the example shown in Figure 3, the imaging unit 20 that generated image 1 is located in the predetermined area, while the imaging units 20 that generated the other images 2 to 4 are located in other areas. In this case, the display control unit 120 causes the display unit 30 to display only image 1.

[0025] Furthermore, when the position indicated by the position information is included in the predetermined area, the display control unit 120 may cause the image of the imaging unit 20 corresponding to that position information to be displayed on the display unit 30 larger than the images captured by the other imaging units 20. In the example shown in Fig. 4, image 1 is displayed larger than the other images 2 to 4. Note that images 1 to 4 are the same as those in the example shown in Fig. 3.

[0026] In addition, when the display control unit 120 switches between images generated by multiple imaging units 20 and displays them on the imaging unit 20, it may display the image of the imaging unit 20 corresponding to the location information on the display unit 30 for longer than the images captured by the other imaging units 20.

[0027] Information indicating the predetermined area is stored in advance in the storage unit 130. The display control unit 120 reads and uses this information from the storage unit 130. This information is set, for example, by a person who manages the sales floor of a store, such as a store clerk.

[0028] For example, as shown in Fig. 5, at least one surveillance imaging device 40 is usually installed in the sales floor of a store. Then, store clerks and surveillance personnel monitor the inside of the store by checking images captured by the imaging device 40. However, a plurality of shelves 50 are installed in a store. For this reason, there may be areas in the sales floor that cannot be captured by the imaging device 40, i.e., blind spots 60. It is preferable that the above-mentioned predetermined area includes the area that becomes the blind spot 60.

[0029] In addition, it may be possible to know in advance which products are likely to be shoplifted by customers. For this reason, it is preferable that the predetermined area includes an area where specific products, such as products that are likely to be shoplifted, are displayed. Note that the specific products may also include high-priced products.

[0030] The information processing apparatus 10 has, as a hardware configuration, for example, as shown in FIG. 6, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0031] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0032] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0033] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0034] The storage device 1040 is an auxiliary storage device realized by removable media such as a hard disk drive (HDD), a solid state drive (SSD), a memory card, or a read-only memory (ROM), and has a recording medium. The recording medium of the storage device 1040 stores program modules that realize each function of the information processing device 10 (e.g., the position information acquisition unit 110, the display control unit 120, and the reference setting unit 140 and image processing unit 150 described later). The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. Note that the storage device 1040 may also function as the memory unit 130.

[0035] The input / output interface 1050 is an interface for connecting the information processing device 10 to various input / output devices. For example, the information processing device 10 may communicate with the storage unit 130 via the input / output interface 1050.

[0036] The network interface 1060 is an interface for connecting the information processing device 10 to a network. This network is, for example, a local area network (LAN) or a wide area network (WAN). The network interface 1060 may connect to the network wirelessly or by wire. The information processing device 10 may communicate with at least one of the imaging unit 20, the display unit 30, and the storage unit 130 via the network interface 1060.

[0037] The information processing device 10 operates, for example, as shown in Fig. 7 . First, the position information acquisition unit 110 acquires the position information of the imaging unit 20. If multiple imaging units 20 are present in the sales floor of a store, the position information acquisition unit 110 acquires the position information of each of the multiple imaging units 20 (step S10). Then, the display control unit 120 controls the display of images generated by the imaging units 20 on the display unit 30 (step S20). The information processing device 10 repeatedly performs the processes shown in steps S10 and S20. The repetition interval is, for example, between 1 second and 10 seconds, but is not limited thereto.

[0038] In this way, the display control unit 120 of the information processing device 10 uses the positions of each of the multiple imaging units 20 to perform real-time control to display images generated by the imaging units 20 on the display unit 30. Therefore, the display control unit 120 can perform control to display images generated by the multiple imaging units 20 on the display unit 30 so that the sales floor of the store can be monitored efficiently.

[0039] The information processing device 10 may further include a reference setting unit 140, as shown in FIG. 8 . The reference setting unit 140 sets the above-mentioned "predetermined area" using layout information of the store's sales floor. The layout information includes, for example, the position, width, height, and orientation of the shelf 50, information identifying the products displayed on the shelf 50 (e.g., product identification information such as JAN codes), and the position, height, and orientation of the image capture device 40. For example, the reference setting unit 140 uses the layout information to identify an area that is a blind spot for the image capture device 40 and set this area as the "predetermined area." As another example, the reference setting unit 140 acquires information identifying a product to be monitored (e.g., product identification information for the product), identifies the shelf 50 corresponding to this information, and sets the area in front of the shelf 50 as the "predetermined area." Note that the information identifying the product to be monitored may be stored in advance in the storage unit 130 as, for example, a list, or may be input to the information processing device 10 by a store clerk or monitor when the "predetermined area" is set.

[0040] It is also preferable that areas of the sales floor where no store clerk is present be included in the above-mentioned predetermined area. For example, the standard setting unit 140 may acquire information indicating the current location of the store clerk, and set the "predetermined area" using this location information and store layout information. For example, the standard setting unit 140 may use the store clerk's location information to identify an area that can be seen from this location information, and set the part of the sales floor that is not included in this area as the "predetermined area."

[0041] In this way, the burden on store staff and security personnel when setting the "predetermined area" is reduced.

[0042] 9, the information processing device 10 may further include an image processing unit 150. The image processing unit 150 processes the image generated by the imaging unit 20 to generate various pieces of information.

[0043] For example, the image processing unit 150 processes the image generated by the imaging unit 20 to generate position information for the imaging unit 20. For example, the image processing unit 150 processes the image to identify the type of product included in the image, and generates position information using this type and store layout information. More specifically, the image processing unit 150 uses the layout information to identify the location where that type of product is displayed, and generates information indicating an area where the identified location can be photographed as position information. Note that if there is a landmark item in the sales floor of the store and the layout information includes information indicating the location of the item, the image processing unit 150 may determine whether the item is included in the image, and if the item is included in the image, generate information indicating an area where the item can be photographed as position information.

[0044] Furthermore, the image processing unit 150 may process an image to generate importance information indicating the importance of the imaging unit 20 that generated the image. In this case, the display control unit 120 may further use this importance information to control the display of the image.

[0045] For example, the display control unit 120 uses the position information to generate importance information related to the position of the imaging unit 20. For example, when the imaging unit 20 is located in a "predetermined area," the importance related to the position of the imaging unit 20 is higher than in other cases. Note that, when multiple "predetermined areas" are set, each of the multiple "predetermined areas" may be weighted. In this case, an imaging unit 20 located in a "predetermined area" with a relatively high weighting is given a higher importance than imaging units 20 located in other "predetermined areas." Note that information related to this weighting is also stored in advance in the storage unit 130.

[0046] The display control unit 120 then generates information indicating the priority of image display using the importance information based on the position of the imaging unit 20 and the importance information based on the image. If the importance information is an importance score, the imaging unit 20 may calculate the average or weighted average of the scores, and the higher this average, the higher the priority.

[0047] Note that a specific example of image display control in this example is the same as the example described using Figures 1 and 2. For example, the display control unit 120 may cause the display unit 30 to display only images generated by the imaging unit 20 whose importance exceeds a reference value. Furthermore, the higher the importance, the larger the image generated by that imaging unit 20 may be displayed on the display unit 30. Furthermore, the higher the importance, the longer the image generated by that imaging unit 20 may be displayed on the display unit 30.

[0048] The importance information based on the image is calculated, for example, based on whether or not a person is included in the image. For example, the image processing unit 150 may increase the importance of an image if a person is included in the image. Here, the image processing unit 150 may further increase the importance of the image if the person is carrying a container other than a cart or shopping basket provided by the store, such as a bag or purse that the person brought into the store.

[0049] The image processing unit 150 may further identify at least one of the person's movement and posture using the image and generate importance information using this. For example, when the person's movement or posture meets a predetermined criterion, the importance of the image capture unit 20 that generated the image is increased compared to the importance of other image capture units 20. An example of the predetermined criterion is that the person is behaving in a way that assumes they are going to take away the product, such as looking around or staying in the same place for a long time.

[0050] When the image processing unit 150 generates importance information, the display control unit 120 can use this importance information to control the display of images generated by multiple imaging units 20 on the display unit 30 so that the store's sales floor can be monitored more efficiently.

[0051] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0052] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0053] Some or all of the above embodiments can be described as, but are not limited to, the following notes. 1. An information processing device comprising: position information acquisition means that moves with a customer within a store and acquires position information indicating the position within the store of a container that contains a product the customer plans to purchase; and display control means that uses the position information to control the display of an image generated by imaging means that is attached to the container and captures an image of the area around the container. 2. The information processing device described in above 1, wherein the display control means performs at least one of a determination process of determining whether or not to display the image on a display means, and a determination process of determining at least one of a display position and a size of the image on the display means. 3. The information processing device described in above 1 or 2, wherein the display control means determines whether or not the position indicated by the position information is included in a predetermined area of ​​the store, and controls the display of the image using the determination result. 4. The information processing device described in above 3, wherein the predetermined area includes an area that is a blind spot of a surveillance camera installed in the store. 5. 10. The information processing device according to any one of claims 1 to 6, further comprising: a reference setting unit that sets the predetermined area using layout information of the store; a display control unit that controls the display of the image using the layout information of the store; a display control unit that controls the display of the image using the layout information of the store; a display control unit that controls the display of the image using the layout information of the store; a display control unit that controls the display of the image using the layout information of the store; a display control unit that controls the display of the image using the layout information of the store; a display control unit that controls the display of the image using the layout information of the store; a display control unit that controls the display of the image using the layout information of the store; a display control unit that controls the display of the image using the layout information of the store;10. The information processing device described in above 9, wherein the image processing means generates the importance information using whether or not a person is included in the image. 11. The information processing device described in above 10, wherein the image processing means generates the importance information using at least one of the person's movement and posture. 12. An information processing method in which a computer moves within a store together with a customer and acquires position information indicating the position within the store of a container containing a product the customer plans to purchase, and uses the position information to control the display of an image generated by an imaging means attached to the container and capturing an image of the surrounding area of ​​the container. 13. A recording medium having recorded thereon a program that causes a computer to have: position information acquisition means that moves within the store together with the customer and acquires position information indicating the position within the store of a container containing a product the customer plans to purchase, and display control means that uses the position information to control the display of an image generated by an imaging means attached to the container and capturing an image of the surrounding area of ​​the container. 14. The program described in Appendix 13.

[0054] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 11, which are dependent on Supplementary Note 1, may also be dependent on Supplementary Notes 12 to 14 in the same dependent relationship as Supplementary Notes 2 to 11. Furthermore, not limited to Supplementary Notes 1 and Supplementary Notes 12 to 14, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0055] REFERENCE SIGNS LIST 10 Information processing device 20 Imaging unit 30 Display unit 40 Imaging device 50 Shelf 60 Blind spot 110 Position information acquisition unit 120 Display control unit 130 Storage unit 140 Reference setting unit

Claims

1. An information processing device comprising: a location information acquisition means that moves with a customer within a store and acquires location information indicating the location within the store of a container that contains a product that the customer plans to purchase; and a display control means that uses the location information to control the display of an image generated by an imaging means that is attached to the container and captures an image of the surroundings of the container.

2. An information processing device according to claim 1, wherein the display control means performs at least one of a judgment process for determining whether or not to display the image on the display means, and a decision process for determining at least one of the display position and size of the image on the display means.

3. An information processing device according to claim 1 or 2, wherein the display control means determines whether the location indicated by the location information is included in a predetermined area of ​​the store, and controls the display of the image using the result of the determination.

4. An information processing device according to claim 3, wherein the predetermined area includes an area that is a blind spot of a surveillance camera installed in the store.

5. An information processing device according to claim 3 or 4, wherein the predetermined area includes an area where specific products are displayed.

6. An information processing device according to any one of claims 3 to 5, further comprising a standard setting means for setting the predetermined area using the layout information of the store.

7. An information processing device according to any one of claims 1 to 6, comprising image processing means for generating the position information by processing the image.

8. An information processing device according to claim 7, wherein the image processing means identifies the type of product included in the image and generates the location information using the type and layout information of the store.

9. An information processing device according to any one of claims 1 to 6, comprising image processing means for generating importance information indicating the importance of the imaging means that generated the image by processing the image, and wherein the display control means further controls the display of the image using the importance information.

10. An information processing device according to claim 9, wherein said image processing means generates said importance information using whether or not said image contains a person.

11. An information processing device according to claim 10, wherein said image processing means generates said importance information using at least one of the movement and posture of said person.

12. An information processing method in which a computer moves with a customer within a store, acquires location information indicating the location within the store of a container containing a product the customer plans to purchase, and uses the location information to control the display of an image generated by an imaging means attached to the container that captures an image of the area around the container.

13. A recording medium having recorded thereon a program that equips a computer with: a location information acquisition means that moves with a customer within a store and acquires location information indicating the location within the store of a container that contains a product that the customer plans to purchase; and a display control means that uses the location information to control the display of an image generated by an imaging means that is attached to the container and takes an image of the area around the container.

Images