Display control system and display control program

Abstract

To provide a display control system that guides a customer to a non-congested facility without displaying degree of congestion of a facility in a display control system and a display control program.SOLUTION: Each signage installed in each store calculates, on the basis of on selection conditions including gender and age of a customer in front of the signage and a degree of congestion at other signage-installed stores, content selection information (content selection probability of the other stores) for the signage (S4), and displays, on the basis of the content selection information, content (interrupted content) on the other signage-installed stores (other stores) (S6). Thereby, each signage can guide, without displaying congestion level of a signage-installed store (especially information that the store is vacant), the customer to a non-congested and safe store that meets the above selection conditions among other signage-installed stores (other stores).SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a display control system and a display control program. [Background technology]

[0002] Recently, due to the increase in the number of people infected with the so-called novel coronavirus (COVID-19), the number of customers visiting shopping complexes and other facilities has decreased. This means that customers are avoiding crowded places and places where many people are not wearing masks.

[0003] In the above-mentioned situation where customers are avoiding crowded places, there are systems that display the degree of congestion (vacancy status) of a facility in order to guide customers to a facility that is not crowded. An example of such a system that displays the degree of congestion of a facility is the system described in Patent Document 1 below. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6153177 Summary of the Invention [Problem to be solved by the invention]

[0005] However, many facility managers are opposed to methods such as the system shown in Patent Document 1 above, which clearly display information about whether the facility is available (degree of congestion), because they feel it looks bad, and such systems have not become widespread.

[0006] The present invention is intended to solve the above-mentioned problems, and aims to provide a display control system and a display control program that can guide customers to facilities that are not crowded without displaying the congestion level of the facility. [Means for solving the problem]

[0007] In order to solve the above problem, a display control system according to a first aspect of the present invention includes display devices having an image capturing means, which are installed in each of at least two or more facilities; estimation means for detecting people in front of each of the display devices from an image captured by the image capturing means and estimating attributes of the detected people; congestion degree determination means for determining a congestion degree of the facility in which each of the display devices is installed, based on the number of people detected by the estimation means who have entered the facility in which each of the display devices is installed; and content to be displayed on each of the display devices installed in the two or more facilities, based on selection conditions including attributes of people in front of the display device estimated by the estimation means from an image captured by the image capturing means of the display device and congestion degrees of facilities in which other display devices than the display device are installed, determined by the congestion degree determination means. a content selection probability, which is the probability of selecting content related to a facility in which a display device other than the display device is installed, of calculation Select the content you want to Probability calculations a display device installed in each of the two or more facilities, probability Based on Between displaying contents related to the facility in which the display device is installed, The display device further includes a display control means for controlling the display device so as to display content relating to a facility in which the display device is installed.

[0009] In this display control system, the content selection Probability calculations The means selects, for each of the display devices installed in the two or more facilities, a selection condition including an attribute of a person in front of the display device, and a congestion degree of the facility in which the other display device is installed, as well as the congestion degree of the facility in which the other display device is installed, as determined by the congestion degree determination means. Content Selection Probability of calculation It is desirable to do so.

[0010] In this display control system, the content selection Probability calculations It is desirable that the selection conditions used by the means include, in addition to the attributes of the person in front of the display device, at least one of the following conditions: the time of day, the weather in the area in which the facility in which the display device is installed is located, and the location of the facility in which the display device is installed.

[0011] In this display control system, the content selection Probability calculations When the selection conditions are vectorized and input, Content Selection Probability The neural network may be configured to output:

[0012] In the case where the display control system is provided with the above-mentioned neural network, the display control system further comprises: a face authentication information generation means for generating face authentication information for face authentication of a person from a face image of the person in an image captured by the imaging means of the display device, for each of the display devices installed in the two or more facilities; a movement detection means for detecting whether or not a person who was in front of a facility where a certain display device is installed has moved to a facility where another display device is installed, using the face authentication information generated by the face authentication information generation means; and a movement detection means for detecting whether or not a person who was in front of a certain display device has moved to a facility where another display device is installed, based on a selection condition including an attribute of the person who was in front of the certain display device estimated by the estimation means from the image captured by the imaging means of the certain display device, and a result of detection by the movement detection means of whether or not a person who was in front of the certain display device has moved to a facility where another display device is installed. Neural Networks It is preferable that the present invention further comprises a training data set generating means for automatically generating the training data set.

[0013] A display control program according to a second aspect of the present invention is a display control program for controlling switching of content to be displayed on a display device having an image capturing means installed in each of at least two or more facilities, comprising: an estimation means for detecting a person in front of each of the display devices from an image captured by the image capturing means and estimating attributes of the detected person; a congestion degree determination means for determining a congestion degree of the facility in which each of the display devices is installed, based on the number of people detected by the estimation means who have entered the facility in which each of the display devices is installed; and a congestion degree determination means for determining, for each of the display devices installed in the two or more facilities, a selection condition including an attribute of a person in front of the display device estimated by the estimation means from an image captured by the image capturing means of the display device, and a congestion degree of a facility in which other display devices than the display device are installed, determined by the congestion degree determination means. a content selection probability, which is the probability of selecting content related to a facility in which a display device other than the display device is installed, of calculation Select the content you want to Probability calculations a display device installed in each of the two or more facilities, probability Based on Between displaying contents related to the facility in which the display device is installed, The display device functions as a display control means for controlling the display device to display content relating to a facility in which another display device is installed. Effect of the Invention

[0014] According to a display control system according to a first aspect of the present invention and a display control program according to a second aspect of the present invention, each of display devices installed in two or more facilities selects a person in front of the display device based on a selection condition including an attribute of a person in front of the display device and a congestion degree of the facility in which the other display devices are installed. a content selection probability that the content related to the facility in which the display device other than the display device is installed is selected as the content to be displayed on the display device; Content Selection probability Based on Between displaying contents related to the facility in which the display device is installed, The display device is controlled to display content related to facilities where other display devices are installed. This allows each of the display devices to guide customers to less crowded facilities that meet the selection conditions and that are not crowded, among the facilities where other display devices are installed, without displaying the degree of congestion of the facility where the display device is installed (particularly, information that the facility is vacant). Furthermore, each of the above display devices can display content related to the facility in which the display device is installed, and can guide customers to facilities that are not crowded and are safe among the facilities in which other display devices are installed. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram showing a schematic configuration of a display control system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a block diagram showing a schematic hardware configuration of the signage in FIG. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of a signage management server in FIG. [Figure 4] FIG. 2 is a functional block diagram of the signage and the signage management server. [Diagram 5] FIG. 5 is a functional block diagram of a content selection information generating unit in FIG. 4. [Figure 6]4 is an explanatory diagram of a content list distributed from the signage management server to each signage. FIG. [Figure 7] 11 is an explanatory diagram of a display control process in the signage according to the content list. [Figure 8] 10 is a flowchart of a display control process of interrupting content on each signage, which is performed in the display control system. [Figure 9] 13 is an explanatory diagram of a customer attribute estimation process in the signage. [Figure 10] FIG. 11 is an explanatory diagram of a flow up to interrupt content selection in the content selection information generating unit. [Figure 11] FIG. 4 is an explanatory diagram of content display on the signage. [Figure 12] A configuration diagram of the functional blocks required for automatic generation of learning datasets and learning in the above signage and signage management server. [Figure 13] Time chart of the process of automatically generating a training dataset for the content selection NN in Figure 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] A display control system and a display control program according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a display control system 10 according to this embodiment. As shown in FIG. 1, the display control system 10 mainly includes signages 1a, 1b, etc., which are tablet terminals for digital signage installed in each store ("facility" in the claims) Sa, Sb, etc., and a signage management server 5 connected to the signages 1a, 1b, etc. via the Internet. In the following description, the signage 1 is a general term for the signages 1a, 1b, etc., and the store S is a general term for the stores Sa, Sb, etc. The display control system 10 includes one or more signages 1 ("display device" in the claims) and a wireless LAN router 3 in each store S. The signage 1 includes a built-in camera 2 ("imaging means" in the claims).

[0017] The signage 1 is installed in a location where it can capture images of the area around the entrance or exit of store S, and displays content such as advertisements on its touch panel display 14 (see Figure 2) aimed at customers ("people" in the claims) who have visited the store (standing in front of the signage 1), and based on the frame image from its built-in camera 2, detects customers who appear in the frame image and estimates the attributes of the detected customers.

[0018] The signage management server 5 is a server for managing signage installed in the management department (head office, etc.) of store S. Details will be described later, but the signage management server 5 receives customer information such as the attributes of the above customers from each signage 1, and generates content selection probabilities ("content selection information" in the claims) that are information for selecting content to be displayed on each signage 1 based on the customer information. The signage 1 and signage management server 5 correspond to the "computer" in the claims.

[0019] Next, the hardware configuration of the tablet-type signage 1 will be described with reference to Fig. 2. In addition to the built-in camera 2, the signage 1 includes a System-on-a-Chip (SoC) 11, a touch panel display 14, a speaker 15, a memory 16 for storing various data and programs, a communication unit 17, a secondary battery 18, and a charging terminal 19. The SoC 11 includes a CPU 12 for controlling the entire device and performing various calculations, and a GPU 13 used for inference processing of various trained Deep Neural Networks (DNN) models.

[0020] The programs stored in the memory 16 include a signage display control program 20. This signage display control program 20 includes various inference models. The communication unit 17 is equipped with a communication IC and an antenna. The signage 1 is connected to a signage management server 5 on the cloud via the communication unit 17 and the Internet. The secondary battery 18 is a battery that can be used repeatedly by charging, such as a lithium ion battery, and stores power from a commercial power source after it is converted into DC power by an AC / DC converter, and supplies the power to each unit of the signage 1.

[0021] Next, the hardware configuration of signage management server 5 will be described with reference to Fig. 3. Signage management server 5 includes a CPU 21 that controls the entire device and performs various calculations, a hard disk 22 that stores various data and programs, a RAM (Random Access Memory) 23, a display 24, an operation unit 25, and a communication unit 26.

[0022] The programs stored in the hard disk 22 include a server-side display control program 27. This server-side display control program 27 and the signage-side display control program 20 correspond to the display control program in the claims.

[0023] 4 shows functional blocks of the above-mentioned signage 1 and signage management server 5 (during normal operation). In addition to the above-mentioned built-in camera 2, touch panel display 14, and speaker 15, the signage 1 includes, as functional blocks, a video input unit 31, an estimation unit 32 ("estimation means" in the claims), and a content display control unit 34 (part of "display control means" in the claims). The signage management server 5 includes, as a functional block, a content selection information generation unit 37 ("content selection information generation means" in the claims). The content selection information generation unit 37 includes a (trained) content selection Neural Networks (NN) 38 ("neural networks" in the claims).

[0024] 2 (including an I / O chipset, not shown), receives and decodes (the video stream data) input from the built-in camera 2 of each signage 1 to generate (the frame image data). The estimation unit 32 detects a person (customer) in front of the signage 1 from the frame image (captured image) captured by the built-in camera 2 of the signage 1, and estimates the attributes of the detected customer. The customer attributes estimated by the estimation unit 32 include gender, age, whether or not a mask is worn, and whether or not hand sanitization (with alcohol) has been performed.

[0025] The content selection information generation unit 37 on the signage management server 5 side generates content selection information (content selection probability) for each signage 1 installed in two or more stores S, based on selection conditions including the gender and age of the customer in front of the signage 1 estimated by the estimation unit 32 from the frame image (captured image) by the built-in camera 2 of the signage 1, and the degree of congestion of the stores S in which other signages 1 other than the signage 1 are installed. The content selection information generation unit 37 generates content selection information (content selection probability) for selecting content to be displayed on the signage 1, based on selection conditions including the gender and age of the customer in front of the signage 1 estimated by the estimation unit 32 from the frame image (captured image) by the built-in camera 2 of the signage 1, and the degree of congestion of the stores S in which other signages 1 other than the signage 1 are installed. For example, for a signage 1a installed in a store Sa in Figure 1, the content selection information generation unit 37 generates content selection information, which is information for selecting content to be displayed on the signage 1a, based on selection conditions including the gender and age of a person in front of the signage 1a, which are estimated by the estimation unit 32 from a frame image (captured image) by the built-in camera 2 of the signage 1a, and the degree of congestion of stores Sb, etc. (other stores) in which other signages 1b, etc. are installed, which is determined based on the number of customers visiting the store transmitted from other signages 1b, etc. other than the signage 1a.

[0026] The content display control unit 34 on the signage 1 side, together with the server-side display control unit 47 in Fig. 5 described later, controls each signage 1 installed in each store S to display content related to the store S (own store) in which the signage 1 is installed, based on the above content selection information (selection probability of content of other stores). For example, the content display control unit 34 and the server-side display control unit 47 control the signage 1 in Fig. 1 to display content related to stores Sb, etc. (other stores around the own store) in which other signages 1b, etc. other than the signage 1a are installed, based on the above content selection information (selection probability of content of other stores), based on the above content selection information (selection probability of content of other stores), between displaying content related to the store Sa (own store) in which the signage 1a is installed.

[0027] Among the functional blocks of the signage 1 described above, video input unit 31, estimation unit 32, and content display control unit 34 are mainly realized by SoC 11 in Fig. 2. Moreover, content selection information generation unit 37 on the signage management server 5 side is mainly realized by CPU 21 and communication unit 26 in Fig. 3.

[0028] 5 shows an example of functional blocks in content selection information generation unit 37 of signage management server 5. In this example, content selection information generation unit 37 of signage management server 5 includes a signage customer information receiving unit 41, a congestion level / risk level determination unit 42 (congestion level determination means), a content selection probability calculation unit 43, an interruption content determination unit 44, an interruption content delivery unit 45, and a content list delivery unit 46. Of these functional blocks, interruption content determination unit 44, interruption content delivery unit 45, and content list delivery unit 46 constitute a server-side display control unit 47 indicated by a dashed line. This server-side display control unit 47, together with content display control unit 34 on the signage 1 side, functions as the "display control means" in the claims. Furthermore, the signage management server 5 stores on its hard disk 22 (see FIG. 3) a signage customer information file (referred to as "signage customer information F" in FIG. 5 and the following description) 48 and a content information file (referred to as "content information F" in FIG. 5 and the following description) 49. The signage customer information F48 stores customer information such as customer attributes and detection information of customers visiting the store, transmitted from the signage group (signages 1a, 1b, etc.). The content information F49 stores various contents to be sent to each signage 1, and a content list that is a list of contents related to each store S that are displayed on the signage 1 of that store.

[0029] 5 shows an example in which the functional blocks of signage customer information receiving unit 41, congestion level / risk level determination unit 42, content selection probability calculation unit 43, and server-side display control unit 47 (interruption content determination unit 44, interruption content delivery unit 45, and content list delivery unit 46) are provided within content selection information generation unit 37 of signage management server 5, but among these functional blocks, signage customer information receiving unit 41, congestion level / risk level determination unit 42, and server-side display control unit 47 do not necessarily need to be provided within content selection information generation unit 37. Therefore, in the claims, content selection information generating means corresponding to content selection information generation unit 37, congestion level determination means corresponding to congestion level / risk level determination unit 42, and display control means corresponding to server-side display control unit 47 and content display control unit 34 are described as separate components.

[0030] For convenience of explanation, the following will begin with the explanation of the functional blocks in Fig. 5, starting with content list distribution unit 46. When each signage 1 requests signage management server 5 to download a content list upon device startup and at a predetermined time (e.g., midnight), content list distribution unit 46 reads out a content list related to the store in which each signage 1 is installed from content information F49, and distributes it to each signage 1.

[0031] Next, the above content list will be described. The content list is information indicating the (display) order, display time, etc. of the content to be played on the signage 1 of each store S. The content selection information generating unit 37 (mainly the content list distributing unit 46) of the signage management server 5 distributes the content list related to the store in which each signage 1 is installed to each signage 1, thereby controlling (instructing) each signage 1 (its content display control unit 34) to repeatedly display each piece of content related to its own store for a predetermined time each.

[0032] For example, if the content list 51 of restaurant A in which signage 1a is installed is as shown in Fig. 6(a), the signage 1a installed in restaurant A repeats the process of displaying (the contents of) the menu of restaurant A for 10 seconds and then displaying video of the inside of restaurant A for 15 seconds in accordance with the content list 51, as shown in Fig. 7. Also, if the content list 52 of movie theater B in which signage 1b is installed is as shown in Fig. 6(b), the signage 1b installed in movie theater B repeats the process of displaying card member information for movie theater B for 30 seconds, a process of displaying an introductory video for movie 1 for 60 seconds, and a process of displaying an introductory video for movie 2 for 30 seconds in accordance with the content list 52.

[0033] Returning to the explanation of each functional block in Fig. 5, the signage customer information receiving unit 41 receives customer information such as customer attributes and detection information of customers (customers who have entered the store) transmitted from the signage group (signages 1a, 1b, etc.), and stores it in signage customer information F48. Among the functional blocks in Fig. 5, the functions of the congestion level / risk level determining unit 42, content selection probability calculating unit 43, interrupting content determining unit 44, and interrupting content distributing unit 45 will be described in the explanation of Fig. 8 below.

[0034] Next, a mechanism for displaying interruption content (content related to other stores) on each signage 1 in the display control system 10 will be described with reference to the time chart of Fig. 8. First, the SoC 11 (estimation unit 32) of each signage 1 (signages 1a, 1b, etc.) detects customers in front of each signage 1 from a frame image (captured image) input from its own built-in camera 2 (S1), as shown in Fig. 9, and estimates the attributes of the detected customers (gender, age, whether or not they wear a mask, and whether or not they have disinfected their hands with alcohol, etc.) (S2). In addition, the SoC 11 (estimation unit 32) tracks the customers detected in S1, and detects customers (visitors) who have entered the store in which the signage 1 is installed, among the customers in front of each signage 1. The SoC 11 and communication unit 17 of each signage 1 transmit customer information, including the detection information of the visiting customer and the customer attribute information estimated in S2 above, to the signage management server 5 at regular time intervals (for example, every 30 seconds).

[0035] The signage management server 5 (its signage customer information receiving unit 41) stores the above customer information received from each signage 1 in the signage customer information F48. Then, the crowdedness / risk determination unit 42 (see FIG. 5) of the signage management server 5 determines the crowdedness of the store in which each signage 1 is installed based on the number of customers obtained from the above customer information (i.e., the number of customers who entered the store in which each signage 1 is installed among the customers detected by the estimation unit 32 of each signage 1), and determines the risk of the store in which each signage 1 is installed based on the number of customers obtained from the above customer information and attribute information of the customers (whether or not they wore masks and whether or not they were disinfected) (S3).

[0036] A specific example of the store congestion / risk determination process in S3 is as follows. First, for the store congestion determination process, the manager of the display control system 10 or the like stores (when the signage 1 is installed in each store, when the store is renovated, etc.) capacity information such as "if the number of customers in the store is 10 or less, it is empty, if the number of customers is more than 10, it is slightly crowded, and if the number of customers is more than 15, it is crowded" and the average in-store stay time of customers in the store in the hard disk 22 of the signage management server 5 for each store in which the signage 1 is installed. Then, the congestion / risk determination unit 42 of the signage management server 5 calculates the current number of people in the store for each store in which the signage 1 is installed, from the number of customers obtained from the above customer information and the average in-store stay time of customers stored in the hard disk 22, and determines the congestion level of each store from this current number of people in the store and the capacity information of each store.

[0037] As a variation of the above-mentioned congestion level determination process, the SoC11 (estimation unit 32) of the signage 1 may detect the entry and exit of customers from frame images captured by the built-in camera 2 of the signage 1 installed at the entrance and exit of the store, and transmit this customer entry and exit information to the signage management server 5. The CPU21 (crowding level / risk determination unit 42) of the signage management server 5 may then calculate the current number of people inside each store based on the customer entry and exit information, and determine the congestion level of each store based on this current number of people inside the store.

[0038] In addition, with regard to the process of assessing the risk level of a store, an administrator of the display control system 10, etc., stores (retains) in advance (when the signage 1 is installed in each store, when the store is renovated, etc.) risk capacity information such as "it is dangerous if three or more people in the store are not wearing masks," "it is dangerous if three or more people in the store are not disinfecting," or "it is dangerous if three or more people in the store are not wearing masks or three or more people are not disinfecting," as well as the average time customers spend in the store, in the hard disk 22 of the signage management server 5. Then, the congestion / risk determination unit 42 of the signage management server 5 calculates, for each store in which the signage 1 is installed, the current number of customers not wearing masks and / or not disinfected in the store based on the number of customers not wearing masks and / or not disinfected obtained from the customer information above and the average time customers spend in the store stored on the hard disk 22, and determines the risk level of each store based on this current number of customers not wearing masks and / or not disinfected and the risk capacity information of each store.

[0039] When the store's congestion and danger level determination process in S3 is completed, the content selection probability calculation unit 43 (see Figure 5) of the signage management server 5 calculates (S4) a content selection probability ("content selection information" in the claims) for a certain signage 1a (for example, a signage installed in restaurant A shown in Figure 6, etc.) based on "selection conditions" including the gender and age of customers in front of the signage 1a (for example, a signage installed in restaurant A shown in Figure 6, etc.) and the congestion and danger level of the store in which the signage 1b is installed (for example, movie theater B shown in Figure 6, etc.) determined in S3 based on customer information transmitted from other signage 1b.

[0040] The above "selection conditions" include the (estimated) gender and age of the customer in front of the signage 1a, which are included in the customer information transmitted from the signage 1a, as well as conditions such as the time period at that time (when the content selection probability calculation process is executed), the weather in the area where the store where the signage 1a is installed is located, and the location of the store where the signage 1a is installed. For example, the selection conditions are "13:00 to 15:00 (time period), (weather is) rainy, (store location is) District A, (estimated gender and age is) female in her 30s." Among the above "selection conditions," information such as the time period at that time, the weather in the area where the store where the signage 1a is installed is located, and the location of the store where the signage 1a is installed is collected by the signage management server 5 (CPU 21) from a timer, information stored in the hard disk 22, information on the network, etc.

[0041] The content selection probability calculation unit 43 may be implemented using the trained neural network (content selection NN38) shown in Fig. 4, or may be implemented using a simple table. Using the table or trained neural network (content selection NN38), the content selection probability calculation unit 43 outputs the content selection probability (for a certain signage 1) that best matches the "selection conditions" including the gender and age of a customer in front of the signage 1.

[0042] When a certain "selection condition" is input, the content selection probability calculation unit 43 uses the learned content selection NN 38 (see Figure 4) or table to find the probability (selection probability) that content related to other stores (e.g., stores Sb, Sc, Sd) in the vicinity of a certain store (e.g., store Sa) will be selected as content to be displayed on the signage 1 of the certain store (e.g., store Sa) under certain "selection conditions," such as store information (such as a food menu) for store Sb being 5%, store information for store Sc being 3%, and store information for store Sd being 2%.

[0043] When the content selection probability calculation unit 43 is implemented using AI (content selection NN 38) as shown in FIG. 4, the above-mentioned "selection conditions" are embedded in an input vector to the content selection NN 38. For example, among the "selection conditions", the time periods of 9:00 to 10:00, 10:00 to 11:00, and 12:00 to 13:00 are converted into codes 1, 2, and 3, respectively, and embedded (taken in) into the input vector. In addition, the weather of the region where a store having a certain signage 1a is located is also converted into codes such as 1 (sunny), 2 (cloudy), and 3 (rainy), and embedded into the input vector. When the trained content selection NN 38 receives a vector in which coded selection conditions for a certain signage 1 (e.g., signage 1a) are embedded, as described above, it outputs the selection probability of content related to other stores having other signages installed other than the certain signage 1 (e.g., stores Sb, Sc, etc. around the signage 1a).

[0044] The content selection probability calculation unit 43 multiplies the selection probability of the content related to the other stores (e.g., stores Sb, Sc, etc.) by a coefficient (hereinafter referred to as the "safety coefficient") that is calculated from the congestion and danger of the other stores and represents the safety of the other stores, and calculates and outputs the final selection probability of the content related to the other stores. The "safety coefficient" is, for example, a value in the range from 0 to 1, and approaches 0 as the congestion and danger of the other stores (facilities) are higher, and approaches 1 as the congestion and danger are lower.

[0045] For example, when the selection probabilities of content related to the above other stores, which are directly calculated by the content selection probability calculation unit 43 using the content selection NN 38 or a table, are 5% for the store guide of store Sb, 3% for the store guide of store Sc, and 2% for the store guide of store Sd, the final content selection probabilities output by the content selection probability calculation unit 43 are as follows: That is, the content selection probability in this case is 5% × the "safety factor" of store Sb for the store guide of store Sb, 3% × the "safety factor" of store Sc for the store guide of store Sd, and 2% × the "safety factor" of store Sd for the store guide of store Sb. For example, if the "safety factors" of stores Sb, Sc, and Sd are 0.6, 0, and 1, respectively, the final content selection probability output by the content selection probability calculation unit 43 will be 3% (5% x 0.6) for the store guide for store Sb, 0% (3% x 0) for the store guide for store Sc, and 2% (2% x 1) for the store guide for store Sd.

[0046] Returning to the description of Fig. 8, when the content selection probability calculation process in S4 above is completed, interruption content determination unit 44 of signage management server 5 determines (selects) interruption content for a certain signage 1 (for example, signage 1a) (S5).

[0047] Next, referring to FIG. 10, the flow up to the selection of the interrupt content in the content selection information generation unit 37 will be described. As described above, the congestion degree / risk degree determination unit 42 of the signage management server 5 determines the congestion degree and risk degree of the store where each signage 1 is installed based on the number of customers coming to the store obtained from the customer information received from each signage 1 and the attribute information of the customers (presence or absence of wearing a mask and presence or absence of disinfection). Then, the content selection probability calculation unit 43 calculates the selection probability of the interrupt content for the above signage 1a (for example, the signage installed in Restaurant A shown in FIG. 6 etc.) based on the "selection conditions" including the gender and age of the customers in front of a certain signage 1a and the congestion degree and risk degree of the store where this signage 1b is installed (a store other than the store where a certain signage 1a is installed and around it, for example, Theater B shown in FIG. 6 etc.) determined by the congestion degree / risk degree determination unit 42 based on the customer information transmitted from other signage 1b. By multiplying the selection probability of this interrupt content by the above safety factor, the final content selection probability is obtained.

[0048] The interrupt content determination unit 44 generates a random number R and determines the interrupt content for the above signage 1a from the above final content selection probability and the random number R. Specifically, for example, as shown in FIG. 10, when the final content selection probability output (calculated) by the content selection probability calculation unit 43 is such that the introduction video of Movie 1 in Theater B is 35% and the introduction video of Movie 2 in Theater B is 10%, the interrupt content determination unit 44 generates a random number R between 0 and 99. When the generated random number R satisfies 0 < R < 34, the introduction video of Movie 1 in Theater B is selected as the interrupt content for the above signage 1a. When 35 < R < 44, the introduction video of Movie 2 in Theater B is selected as the interrupt content for the above signage 1a. The interrupt content distribution unit 45 in FIG. 5 transmits the interrupt content selected by the above interrupt content determination unit 44 to the signage 1a. Note that when the generated random number R satisfies 45 < R < 99, the interrupt content determination unit 44 does not select the interrupt content. Therefore, at this time, the interrupt content distribution unit 45 does not transmit the interrupt content to the signage 1a.

[0049] In the explanation of Figure 10 above, an example is shown in which only content related to one other store (Movie theater B) in the vicinity of the store where signage 1a is installed is displayed as interruption content. However, if multiple other stores in the vicinity of the store where signage 1a is installed (for example, Movie theater B where signage 1b is installed and general store C where signage 1c is installed) are all safer and less dangerous than restaurant A where signage 1a is installed, content related to these multiple stores may be displayed as interruption content.

[0050] Returning to the explanation of Fig. 8, when the interruption content determination process in S5 above is completed and interruption content distribution unit 45 transmits the interruption content to signage 1a, signage 1a displays interruption content, which is content related to the store in which other signage 1b, etc. are installed, between displays of content related to the store in which signage 1a is installed (S6).

[0051] Next, the display of content on the signage 1 will be described with reference to Fig. 11. When the signage 1 receives interruption content from the signage management server 5 (interruption content distribution unit 45 thereof), it interrupts the playback of content (related to the store where the signage 1 is installed) according to the above content list, and displays the interruption content (related to the store where the other signage 1 is installed). For example, when the signage 1 installed in restaurant A receives an introductory video for movie 1 in movie theater B from the signage management server 5 as interruption content as shown in Fig. 11, the signage 1 interrupts the playback of content related to restaurant A according to the content list (for example, (content of) the menu of restaurant A), and displays the introductory video for movie 1 in movie theater B (interruption content) on the touch panel display 14.

[0052] The signage 1 installed in Restaurant A may receive from the signage management server 5 information on the degree of congestion and / or danger of the store in which the device is installed (Restaurant A) in addition to the above interruption content (an introduction video of Movie 1 in Cinema B), and superimpose the received information indicating the degree of congestion and / or danger of the store in which the device is installed (own store) on the interruption content. In this case, since it is inappropriate to display information that the inside of the store is dangerous, it is better for the signage 1 to display that the own store (the store in which the device is installed) is crowded even when the received degree of danger of the own store is high, just as it does when the received degree of congestion of the own store is high.

[0053] As shown in FIG. 11, when the display of the interrupting content is completed, the content display control unit 34 (see FIG. 4) of the signage 1 resumes the playback of the content (related to the store where the signage 1 is installed) according to the content list.

[0054] The display control system 10 has a function of automatically generating a training data set for the content selection NN 38 shown in FIG. 4. This training data set automatic generation function will be described with reference to FIG. 12 and FIG. 13. FIG. 12 shows functional blocks required for the automatic generation of the training data set and for learning using the automatically generated training data set in the signage 1 and the signage management server 5. The signage 1 includes a face authentication information generation unit 62 ("face authentication information generation means" in the claims) in addition to the built-in camera 2, the video input unit 31, and the estimation unit 32 shown in FIG. 4 as functional blocks required for the training data set automatic generation function. The face authentication information generation unit 62 generates face authentication information for customer face authentication from the customer's face image in the frame image (photographed image) by the built-in camera 2 of each signage 1. Specifically, the facial recognition information generation unit 62 generates facial recognition information for facial recognition of a customer from a cut-out image (facial image) of the customer's face detected by the face detection unit 61 (a trained DNN (Deep Neural Networks) model for face detection (hereinafter abbreviated as "face detection model")) possessed by the estimation unit 32.

[0055] Meanwhile, signage management server 5 includes a movement detection unit 64 ("movement detection means" in the claims), a learning data set generation unit 65 ("learning data set generation means" in the claims), and a learning unit 67 as functional blocks required for automatic generation of a learning data set and learning. Also, signage management server 5 includes a learning data set 66 that collects teacher data (training data) for content selection NN 38. This learning data set 66 is stored in hard disk 22 (see FIG. 3). Using face authentication information generated by face authentication information generation unit 62, the movement detection unit 64 detects whether a customer who was in front of a store in which a certain signage 1 is installed has moved to a store in which another signage 1 is installed. The training dataset generation unit 65 automatically generates a training dataset 66 for the content selection NN 38 based on selection conditions including attributes (gender, age) of a customer in front of a certain signage 1 estimated by the estimation unit 32 from an image captured by the built-in camera 2 of the certain signage 1, and a result of detection by the movement detection unit 64 as to whether a customer in front of the store where the certain signage 1 is installed has moved to another store where the signage 1 is installed. In addition, the learning unit 67 uses the training dataset 66 automatically generated by the training dataset generation unit 65 to train the content selection NN 38.

[0056] FIG. 13 is a time chart showing the automatic generation process of the learning data set 66 of the content selection NN 38. A specific example of the automatic generation process of the learning data set 66 will be described with reference to FIG. 13. In this example, the estimation unit 32 on each signage 1 side is provided with a gender / age estimation model as a trained DNN model for inference processing, in addition to the face detection model used in the face detection unit 61. Also, the face authentication information generation unit 62 is provided with a vectorization model. Meanwhile, on the signage management server 5 side, the movement detection unit 64 is provided with a DNN model for customer identification processing (customer ReID (Person Re-Identification) processing) based on the customer's face vector as a trained DNN model for inference processing.

[0057] In the above configuration, first, the face detection unit 61 (face detection model) of each signage 1 (e.g., signages 1a, 1b, and 1c) in the display control system 10 detects the face of a customer reflected in a frame image input from the built-in camera 2 (S11), and outputs the coordinate position of the detected face (e.g., coordinates representing the center of the face and a coordinate range representing the horizontal and vertical widths of the face). Next, the gender and age estimation model included in the estimation unit 32 performs an estimation process of the attributes of the customer (gender and age (generation)) using the cut-out image of the customer's face detected by the face detection unit 61 (S12). Then, the vectorization model included in the face authentication information generation unit 62 performs a vectorization process on the cut-out image (face image) of the customer's face detected by the face detection unit 61 (S13). The SoC 11 and the communication unit 17 of each signage 1 transmit the vector resulting from the above vectorization process to the signage management server 5 as a face vector ("face authentication information" in the claims).

[0058] The movement detection unit 64 of the signage management server 5 uses the DNN model for the above customer identification process (customer ReID process) to detect whether a customer who was in front of a store where a certain signage 1 is installed has moved to a store where another signage 1 is installed, based on the face vector transmitted from each signage 1 (S14). Then, the learning dataset generation unit 65 of the signage management server 5 automatically generates a learning dataset 66 for the content selection NN 38 based on the selection conditions including the gender and age of the customer who was in front of the certain signage 1 and the result of detection by the movement detection unit 64 of whether a customer who was in front of the store where the certain signage 1 is installed has moved to a store where the other signage 1 is installed (S15).

[0059] The automatic generation process of S15 described above allows for the collection of training data consisting of a combination of the selection conditions including the gender and age of a customer in front of a certain signage 1 and a detection result of whether or not the customer moved from this store (e.g., restaurant A) to another store (e.g., movie theater B) where the signage 1 is installed, based on the selection conditions including the gender and age of the customer in front of the signage 1 and the detection result of whether or not the customer moved from this store (e.g., restaurant A) to another store (e.g., movie theater B) where the signage 1 is installed under the selection conditions. By using a training data set 66 that collects such training data to train the content selection NN 38, when the trained content selection NN 38 inputs "selection conditions" including the gender and age of a customer in front of a certain signage 1 (e.g., a signage installed in restaurant A), it is possible to output a content selection probability according to the probability that a customer who meets the selection conditions moved from this store (e.g., restaurant A) to another store (e.g., movie theater B) where the signage 1 is installed in the past.

[0060] As described above, according to the display control system 10 and the display control program (signage-side control program 20 (see FIG. 2) and server-side display control program 27 (see FIG. 3)) of the present embodiment, each of the signages 1 installed in two or more stores is controlled to display content (interruption content) related to the store in which the other signages 1 are installed, based on selection conditions including the gender and age of the customer in front of the signage 1 and the congestion and danger levels of the stores in which the other signages 1 are installed. This allows each of the signages 1 to guide customers to a safe store that meets the above selection conditions, is not crowded, and is not affected by the selection conditions, among the stores in which the other signages 1 are installed, without displaying the congestion level of the store in which the signage 1 is installed (particularly, information that the store is vacant).

[0061] Furthermore, according to the display control system 10 of this embodiment, the content display control unit 34 on the signage 1 side and the server-side display control unit 47 control each signage 1 installed in each store to display content related to the store (own store) in which the signage 1 is installed, based on the content selection information (probability of selecting content of other stores). This allows the signage 1 installed in each store to display content related to the store (own store) in which the signage 1 is installed, and to guide customers to stores (other stores) in which other signage 1 is installed that are not crowded and are safe.

[0062] Furthermore, according to the display control system 10 of this embodiment, the selection conditions used by the content selection information generation unit 37 (particularly, the content selection probability calculation unit 43) include, in addition to the gender and age of the customer in front of the signage 1, conditions such as the time of day at that time (when the content selection probability calculation process is executed), the weather in the region where the store where the signage 1 is installed is located, and the location of the store where the signage 1 is installed. This allows the signage 1 to display content related to other stores (stores where other signages 1 other than the signage 1 are installed) that meet conditions such as the time of day at that time (when the content selection probability calculation process is executed), the weather in the region where the store where the signage 1 is installed is located, and the location of the store where the signage 1 is installed, in addition to the gender and age of the customer in front of the signage 1.

[0063] Furthermore, according to the display control system 10 of the present embodiment, a training data set 66 for the content selection NN 38 is automatically generated based on selection conditions including the gender and age (estimated by the estimation unit 32) of a customer in front of a certain signage 1 and a result of detection using face authentication information (face vector) as to whether a customer in front of a store in which the certain signage 1 is installed has moved to a store in which another signage 1 is installed. By training the content selection NN 38 using such a training data set 66, when the trained content selection NN 38 inputs selection conditions including the gender and age of a customer in front of a certain signage 1, it can output a content selection probability according to the probability that a customer matching this selection condition has moved from in front of a store in which the signage 1 is installed to a store in which another signage 1 is installed in the past.

[0064] Variations: The present invention is not limited to the configurations of the above-described embodiments, and various modifications are possible without departing from the spirit of the invention. Next, modifications of the present invention will be described.

[0065] Variation 1: In the above embodiment, the content selection information generating unit 37 (mainly, the content selection probability calculating unit 43) generates content selection information (content selection probability) for each signage 1 installed in two or more stores based on selection conditions including the gender and age of the customer in front of the signage 1 and the congestion and risk of the store (other store) in which the other signage 1 is installed. However, the content selection information generating unit does not necessarily need to generate the above content selection information (content selection probability) based on both the congestion and risk of the other store. In other words, the content selection information generating unit may generate content selection information (content selection probability) for each signage installed in two or more stores based only on selection conditions including the gender and age of the customer in front of the signage and the congestion of the store (other store) in which the other signage is installed.

[0066] Variation 2: The content selection information generating unit may generate content selection information (content selection probability) for each signage installed in two or more stores based on selection conditions including the gender and age of the customer in front of the signage, and the congestion and danger levels of the stores (other stores) in which the other signages are installed, as well as the congestion and danger levels of the store (own store) in which the signage is installed, as determined by the congestion and danger level determining unit. This allows control so that, for each signage installed in two or more stores, when the congestion and danger levels of the own store in which the signage is installed are high, the signage increases the probability of displaying content (interrupting content) related to a store that meets the above selection conditions and has a low congestion and danger level among the other stores in which the other signages are installed, and conversely, when the congestion and danger levels of the own store in which the signage is installed are low, the signage decreases the probability of displaying the above interrupting content.

[0067] In the above case, as in the case of Modification 1, the content selection information generating unit does not necessarily need to generate the above content selection information (content selection probability) based on both the congestion degree and the risk degree. In other words, the content selection information generating unit may generate content selection information (content selection probability) for each signage 1 installed in two or more stores based on selection conditions including the gender and age of the customer in front of the signage 1, and the congestion degree of the store in which the signage 1 is installed (their own store) in addition to the congestion degree of the store in which the other signage 1 is installed (other stores).

[0068] Variation 3: In addition, in the above embodiment, an example was shown in which the attributes included in the selection conditions used by the content selection information generation unit 37 (mainly the content selection probability calculation unit 43) were only the gender and age of the customer, but the attributes included in the above selection conditions are not limited to these and may be, for example, the customer's clothing, whether the customer is wearing accessories (for example, sunglasses, earrings, etc.), the customer's group attributes (type of group, such as a family of three or more people, a couple, friends of the same age, or alone), etc.

[0069] Variation 4: Also, in the above embodiment, the congestion / risk determination unit 42 of the signage management server 5 determines the congestion and risk of the store in which each signage 1 is installed based on the number of customers and attribute information of the customers (whether or not they are wearing masks, and whether or not they have been disinfected) obtained from the customer information received from each signage 1. However, the present invention is not limited to this configuration, and a congestion / risk determination unit may be provided on the signage side, and the congestion / risk determination unit on this signage 1 side may determine the congestion and risk of the store in which each signage 1 is installed based on the number of customers estimated by the estimation unit and the attribute information of the customers (whether or not they are wearing masks, and whether or not they have been disinfected), and transmit information on the congestion and risk of the store to the signage management server 5.

[0070] Variation 5: In the above embodiment, an example was shown in which the content selection information in the present invention is the selection probability (content selection probability) of the content of a store (other store) in which a signage 1 other than the signage 1 is installed, but the content selection information is not limited to this, and may be any information that can control the signage to display the content of the store (other store) in which the other signage is installed based on this information. For example, the content selection information may be information on (a combination of) content display times, such as 11 seconds for the content related to the store in which the signage is installed (for example, restaurant A) (according to the content list), 7 seconds for the first content related to the store in which the other signage is installed (for example, an introduction video of movie 1 in movie theater B), and 2 seconds for the second content related to the store in which the other signage is installed (for example, an introduction video of movie 2 in movie theater B).

[0071] Variation 6: In the above embodiment, in the automatic generation process of the learning dataset 66 of the content selection NN 38, the movement detection unit 64 detects whether a customer who was in front of a store where a signage 1 is installed has moved to a store where another signage 1 is installed, based on a face vector obtained by vectorizing a face image of the customer. That is, the movement detection unit 64 uses a face vector obtained by vectorizing a face image of the customer as authentication information used to identify the customer. However, the present invention is not limited to this, and the movement detection unit may use a customer vector obtained by vectorizing an image of the entire body of the customer as authentication information used to identify the customer, or may use some feature amount that represents the features of the customer's face or body (for example, the face outline, the texture of the face (spots, wrinkles, sagging), the distance between the eyes, etc.). [Explanation of symbols]

[0072] 1, 1a, 1b, 1c Signage (display device) 2 Built-in camera (photography means) 20 Signage display control program (display control program) 27 Server-side display control program (display control program) 32 Estimation part (estimation means) 34 Content display control unit (display control means) 37 Content selection information generation unit (content selection information generation means) 38 Content Selection NN (Neural Network) 42 Congestion degree / danger degree determination unit (congestion degree determination means) 47 Server side display control unit (display control means) 62 Face authentication information generation unit (face authentication information generation means) 64 Movement detection unit (movement detection means) 65 Learning data set generation unit (learning data set generation means) 66 Training Dataset S, Sa, Sb, Sc, Sd Stores (facilities)

Claims

1. A display device having an image capturing means, the display device being installed in each of at least two or more facilities; an estimation means for detecting a person in front of each of the display devices from an image captured by the imaging means and estimating an attribute of the detected person; a congestion level determination means for determining a congestion level of a facility in which each of the display devices is installed based on the number of people who have entered the facility in which each of the display devices is installed among the people detected by the estimation means; a content selection probability calculation means for calculating, for each of the display devices installed in the two or more facilities, a content selection probability which is the probability of selecting content relating to a facility in which a display device other than the display device in question is installed, as content to be displayed on the display device, based on a selection condition including attributes of a person in front of the display device, estimated by the estimation means from an image captured by the imaging means of the display device in question, and a congestion degree of the facility in which the display device in question is installed, determined by the congestion degree determination means; A display control system comprising: a display control means for controlling each of the display devices installed in the two or more facilities to display content related to facilities in which other display devices are installed, between displaying content related to the facility in which the display device is installed, based on the content selection probability.

2. The display control system described in claim 1, characterized in that the content selection probability calculation means calculates the content selection probability for each display device installed in the two or more facilities based on selection conditions including attributes of the person in front of the display device and the congestion degree of the facility in which the other display devices are installed, as well as the congestion degree of the facility in which the other display devices are installed, as determined by the congestion degree determination means.

3. The display control system according to claim 1 or claim 2, characterized in that the selection conditions used by the content selection probability calculation means include, in addition to the attributes of the person in front of the display device, at least one of the following conditions: the time of day at that time, the weather in the area in which the facility in which the display device is installed is located, and the location of the facility in which the display device is installed.

4. 4. The display control system according to claim 1, wherein the content selection probability calculation means includes a neural network that outputs the content selection probability when the selection conditions are vectorized and input.

5. a face authentication information generating means for generating face authentication information for face authentication of a person from a face image of the person in an image captured by an image capturing means of the display device, for each of the display devices installed in the two or more facilities; a movement detection means for detecting whether or not a person who was in front of a facility in which a display device is installed has moved to a facility in which another display device is installed, using the face authentication information generated by the face authentication information generation means; The display control system according to claim 4, further comprising a learning dataset generation means for automatically generating a learning dataset for the neural network based on selection conditions including attributes of a person in front of the certain display device estimated by the estimation means from an image captured by the imaging means of the certain display device, and a result of detection by the movement detection means as to whether a person in front of the facility in which the certain display device is installed has moved to a facility in which the other display device is installed.

6. A display control program for controlling switching of content to be displayed on a display device having an image capturing means, the display device being installed in each of at least two or more facilities, comprising: Computer, an estimation means for detecting a person in front of each of the display devices from an image captured by the imaging means and estimating an attribute of the detected person; a congestion level determination means for determining a congestion level of a facility in which each of the display devices is installed based on the number of people who have entered the facility in which each of the display devices is installed among the people detected by the estimation means; a content selection probability calculation means for calculating, for each of the display devices installed in the two or more facilities, a content selection probability which is the probability of selecting content relating to a facility in which a display device other than the display device in question is installed, as content to be displayed on the display device, based on a selection condition including attributes of a person in front of the display device, estimated by the estimation means from an image captured by the imaging means of the display device in question, and a congestion degree of the facility in which the display device in question is installed, determined by the congestion degree determination means; A display control program for causing each of the display devices installed in the two or more facilities to function as a display control means that controls the display device to display content related to facilities in which other display devices are installed, between displaying content related to the facility in which the display device is installed, based on the content selection probability.

Images