Information processing device, information processing system, information processing method, and program

Abstract

To improve user experience value.SOLUTION: An information processing device comprises: an acquisition unit for acquiring an analysis result obtained by analyzing a sensing result of an environment in which a display device is installed, using a recognition model; and an output control unit configured to control outputs of the display device and an output device associated with the display device based on application of preset rules for the analysis result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] In recent years, technologies have been developed to output various contents to display devices such as digital signage. The contents output to such display devices are changed according to sensing results. For example, Patent Document 1 discloses sensing the flow of people with a sensor and switching the content output by the display device according to the sensing results. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-97749 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, the content that has a predetermined correspondence with the sensing result is merely output to the display device. However, there is a demand for creating an atmosphere that is more suitable for the environment in which the display device is installed.

[0005] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a new and improved technology that can improve the user's experience value. [Means for solving the problem]

[0006] In order to solve the above problem, according to one aspect of the present invention, there is provided an information processing device comprising: an acquisition unit that acquires an analysis result obtained by analyzing the sensing result of the environment in which a display device is installed using a recognition model; and an output control unit that controls the output of the display device and an output device associated with the display device based on applying predetermined rules to the analysis result.

[0007] The output control unit may control outputs from a plurality of output devices associated with the display device, and the plurality of output devices may include a lighting device and a sound output device.

[0008] The acquisition unit may acquire an analysis result obtained by analyzing an image of a person present in the environment, which is the sensing result, using the recognition model.

[0009] The acquisition unit may acquire an analysis result including at least one of an age, a sex, and a clothing of the person.

[0010] The acquisition unit may acquire an analysis result including weather or brightness of the environment.

[0011] The display device and the output device may be installed within a store, and the acquisition unit may acquire an analysis result obtained by analyzing the displayed products using the recognition model from an image of a sales floor for products sold in the store, which is the sensing result.

[0012] The output control unit controls the output of a plurality of display devices and a plurality of output devices installed within the store, and the plurality of display devices and the plurality of output devices may be associated with each of the areas in which the plurality of display devices and the plurality of output devices are installed.

[0013] The output control unit may generate control information for controlling output for each of the display device and the output device from the same analysis result.

[0014] The acquisition unit may acquire multiple analysis results analyzed using multiple recognition models, and the output control unit may generate control information for controlling output for each of the display device and the output device from the multiple analysis results.

[0015] The output control unit may control the output of the output device by controlling the control information about the output device to be transmitted to a device having a virtual assistant function.

[0016] The output control unit may apply the rule to statistical data obtained from a plurality of analysis results.

[0017] The output control unit may obtain a keyword by applying a preset rule to the analysis result, and control the display device and the output device to output content searched for based on the keyword.

[0018] In addition, according to another aspect of the present invention to solve the above problem, an information processing system is provided, comprising: an acquisition unit that acquires an analysis result obtained by analyzing the sensing result of the environment in which a display device is installed using a recognition model; and an output control unit that controls output of the display device and an output device associated with the display device based on application of predetermined rules to the analysis result.

[0019] In addition, according to another aspect of the present invention to solve the above problem, there is provided an information processing method executed by a computer, which includes obtaining an analysis result by analyzing a sensing result of an environment in which a display device is installed using a recognition model, and controlling output of the display device and an output device associated with the display device based on applying a predetermined rule to the analysis result.

[0020] In addition, according to another aspect of the present invention to solve the above problem, a program is provided that causes a computer to function as an acquisition unit that acquires analysis results obtained by analyzing sensing results of the environment in which a display device is installed using a recognition model, and an output control unit that controls output of the display device and an output device associated with the display device based on application of predetermined rules to the analysis results. Effect of the Invention

[0021] As described above, according to the present invention, it is possible to improve the user's experience value. [Brief description of the drawings]

[0022] [Figure 1] 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram for explaining an example of an arrangement of output devices according to an embodiment of the present invention. [Diagram 3] 1 is a block diagram showing an example of the configuration of a recognition device 20 according to an embodiment of the present invention. [Figure 4] 2 is a block diagram showing an example of a configuration of a control device 21 according to the embodiment of the present invention. FIG. [Diagram 5] 2 is a block diagram showing a configuration example of a GW device 30 according to an embodiment of the present invention. FIG. [Figure 6] 2 is a block diagram showing an example of the configuration of a lighting server 40 according to an embodiment of the present invention. FIG. [Figure 7] 1 is a block diagram showing an example of the configuration of a sound server 50 according to an embodiment of the present invention. [Figure 8] FIG. 3 is a diagram for explaining control of the output of each output device when a customer P1 visits a store B having the layout shown in FIG. [Figure 9] 9 is a diagram for explaining an example of generating an output request based on a video captured by the camera C1 shown in FIG. 8. FIG. [Figure 10]FIG. 2 is a sequence diagram showing an example of a flow of operation processing of the information processing system according to the present embodiment. [Figure 11] FIG. 2 is a diagram for explaining an example of an arrangement of output devices according to an embodiment of the present invention. [Figure 12] 12 is a diagram for explaining the control of the output of each output device when a customer P2 visits a store B having the layout shown in FIG. [Figure 13] 13 is a diagram for explaining an example of generating an output request based on a video captured by the camera C2 shown in FIG. 12. FIG. [Figure 14] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device 900 that embodies the information processing system 1 according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and duplicated explanations will be omitted.

[0024] In addition, in this specification and drawings, multiple components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numeral. However, when there is no need to particularly distinguish between multiple components having substantially the same functional configuration, each of the multiple components is given only the same reference numeral.

[0025] <<1. Overview>> First, an overview of an information processing system according to an embodiment of the present invention will be described. Fig. 1 is a diagram for explaining an example of the configuration of an information processing system according to an embodiment of the present invention. As shown in Fig. 1, an information processing system 1 according to an embodiment of the present invention includes a digital signage system 2, a GW device 30, a lighting server 40, a sound server 50, a plurality of cameras C (C1, C2...), a plurality of digital signages D (D1, D2...), a plurality of lighting devices L (L1, L2...), and a plurality of sound output devices S (S1, S2...).

[0026] (Digital Signage System 2) The digital signage system 2 is a system that is connected to a plurality of cameras C and a plurality of digital signages D, and controls the output to each of the plurality of digital signages D.

[0027] The digital signage D is an example of a display device that is installed in a building such as a store and displays content such as video, images, or text to users. The installation location of the digital signage D is not particularly limited, and it may be installed indoors in an office building or the like, or outdoors on a pedestrian road, an underground passage, a station, or a building wall. In this embodiment, an example in which the digital signage D is installed in a store will be mainly described. Note that the digital signage system 2 may control the output of another display device, such as a tablet terminal, that displays content to users moving around the store, instead of the digital signage D.

[0028] Here, the store may be divided into a plurality of areas. The divisions may be set arbitrarily, and may be, for example, divided into areas for each aisle, or into areas based on the installation locations of the digital signage D or the camera C. The digital signage D, the lighting device L, and the sound output device S are associated with each area. Hereinafter, when the digital signage D, the lighting device L, and the sound output device S are not particularly distinguished from each other, they may be referred to as "output devices." The digital signage system 2 controls the output of each output device installed in an area in a linked manner based on the analysis results of the image output by the camera C that captures the image in the area.

[0029] As shown in FIG. 1, the digital signage system 2 includes a recognition device 20 and a control device 21.

[0030] (Recognition device 20) The recognition device 20 acquires sensing results from various sensors and analyzes the sensing results. The type of sensor is not particularly limited, and may be, for example, a camera, a microphone, a temperature sensor, an illuminance sensor, or a people flow sensor. The recognition device 20 may acquire sensing results from multiple types of sensors. In this embodiment, an example in which the recognition device 20 acquires images as sensing results from multiple cameras C will be mainly described.

[0031] The camera C in this embodiment captures an image of the environment in which the digital signage D is installed. For example, the camera C may capture a person image that is an image of a person present in a store. The camera C may also capture a landscape image outside the store by capturing an image of a glass window or an entrance in the store.

[0032] The camera C may be a 360-degree camera, in which case the camera C may simultaneously capture a person image and a landscape image. Alternatively, the person image and the landscape image may each be captured by a different camera C.

[0033] The recognition device 20 analyzes the sensing result using a recognition model. For example, the recognition device 20 may analyze a person video using the recognition model and output attributes of the person appearing in the video, such as the age or gender, as an analysis result. The recognition device 20 may also analyze a person video using the recognition model and output clothing of the person appearing in the video as an analysis result.

[0034] Furthermore, the recognition device 20 may analyze the scenery video using the recognition model, and output the weather or brightness outside the store as the analysis result.

[0035] The recognition device 20 outputs the analysis result to the control device 21 .

[0036] (Control device 21) The control device 21 is an example of an information processing device that controls the outputs of the digital signage D, the lighting device L, and the sound output device S based on the analysis result acquired from the recognition device 20.

[0037] The control device 21 generates an output request, which is control information for controlling each output device, by applying output rules, which are rules for output requests to each output device, that are preset for the acquired analysis results.

[0038] The control device 21 controls the output of the connected digital signage D based on a display request, which is an output request generated for the digital signage D. The control device 21 also transmits a light output request, which is an output request generated for the lighting device L, and a sound output request, which is an output request generated for the sound output device S, to the GW device 30. Based on this, the outputs of the lighting device L and the sound output device S are controlled.

[0039] (GW device 30) The GW device 30 is a gateway device that is communicatively connected to the control device 21, the lighting server 40, and the sound server 50, and transmits an output request received from the control device 21 to the lighting server 40 or the sound server 50.

[0040] Communication between the GW device 30 and the control device 21, the lighting server 40 and the sound server 50 may be realized via a wired or wireless LAN (Local Area Network), or via a wireless communication network such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0041] The GW device 30 may be, for example, a smart speaker with a virtual assistant function. By configuring the GW device 30 as a smart speaker, it becomes possible to easily connect to various services and devices, and it is possible to expand the range of content to be output to each output device. However, the GW device 30 is not particularly limited as long as it is a device that can transmit an output request received from the control device 21 to the lighting server 40 and the sound server 50. In addition, the function of the GW device 30 may be included in the control device 21.

[0042] (Lighting Server 40) The lighting server 40 is a server that controls the output of the connected lighting device L based on a light output request received from the GW device 30. The method of connection between the lighting server 40 and the lighting device L is not particularly limited, and may be, for example, Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), or the like.

[0043] The lighting device L is controlled by the lighting server 40 for light color, brightness, etc. The lighting device L may be, for example, an LED (Light-Emitting Diode) lighting device, or may be a lighting device configured with a light bulb or other light-emitting element such as a laser diode. The shape of the lighting device L is not particularly limited, and may be a light bulb type or a tape type.

[0044] (Sound Server 50) The sound server 50 is a server that controls the output of the connected sound output device S based on a sound output request received from the GW device 30. The method of connection between the sound server 50 and the sound output device S is not particularly limited, and may be, for example, Wi-Fi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), or the like.

[0045] The sound output from the sound output device S is controlled by the sound server 50. The sound output device S is, for example, a speaker.

[0046] (Example of output device layout) So far, we have described an example of the configuration of the information processing system 1. Here, we will explain an example of the arrangement of the output devices and the like included in the information processing system 1 with reference to FIG.

[0047] Fig. 2 is a diagram for explaining an example of the arrangement of each output device according to an embodiment of the present invention. Fig. 2 shows a digital signage D1, a sound output device S1, a lighting device L1, and a camera C1, which are installed in an area A1 near an entrance E of a store B.

[0048] Camera C1 captures an image of customer P1 entering store B through entrance E. Camera C1 also captures an image of the scenery outside store B through window W. Digital signage system 2 then controls the output of digital signage D1, sound output device S1, and lighting device L1 based on the analysis results of the person and scenery images acquired from camera C1. This allows digital signage system 2 to control the output from each output device to suit the analysis results of camera C1, making it possible to present area A1 in a way that suits customer P1 and the surrounding environment.

[0049] The overview of the information processing system 1 according to the embodiment of the present invention has been described above. Next, an example of the configuration of each device included in the information processing system 1 according to the embodiment of the present invention will be described.

[0050] <<2. Configuration Example>> <2-1. Example of the configuration of the recognition device 20> First, a configuration example of the recognition device 20 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing a configuration example of the recognition device 20 according to an embodiment of the present invention. As shown in Fig. 3, the recognition device 20 includes a video acquisition unit 201, a control unit 202, a storage unit 206, and a communication unit 209.

[0051] (Video acquisition unit 201) The image acquisition unit 201 is connected to a camera C, and acquires an image from the camera C, which is an example of a sensing result.

[0052] (Control unit 202) The control unit 202 is mainly configured with a CPU (Central Processing Unit) and controls the overall operation of the recognition device 20. For example, the control unit 202 stores the video acquired by the video acquisition unit 201 in a video DB 207 included in the storage unit 206. The control unit 202 also has an attribute recognition engine 203, a clothing recognition engine 204, and an environment recognition engine 205.

[0053] (Attribute recognition engine 203, clothing recognition engine 204, environment recognition engine 205) The attribute recognition engine 203, the clothing recognition engine 204, and the environment recognition engine 205 output analysis results obtained by analyzing the video captured by the video capture unit 201 using a pre-constructed AI recognition model. Hereinafter, when there is no need to distinguish between the attribute recognition engine 203, the clothing recognition engine 204, and the environment recognition engine 205, they may be simply referred to as "recognition engines."

[0054] The attribute recognition engine 203 analyzes the person video acquired by the video acquisition unit 201, and analyzes the attributes of the person appearing in the person video. The attribute recognition engine 203 may output, for example, the gender or age of the person as the analysis result.

[0055] The clothing recognition engine 204 analyzes the person video acquired by the video acquisition unit 201, and analyzes the clothing of the person appearing in the person video. The clothing recognition engine 204 may output the type of clothes, shoes, and belongings worn by the person as an analysis result. The clothing recognition engine 204 may also output the dress code of the person's clothing (casual, semi-formal, formal, etc.) as an analysis result. The clothing recognition engine 204 may also output the type of the person's clothing (dressy, young casual, hip hop, etc.) as an analysis result.

[0056] When multiple people appear in a person video, the clothing recognition engine 204 may output the analysis results for each person, or may extract features from the analysis results for the multiple people and output the analysis results.

[0057] The environment recognition engine 205 analyzes the scenery image acquired by the image acquisition unit 201 to analyze the environment depicted in the image. For example, the environment recognition engine 205 may analyze the scenery image and output the weather, brightness, or the like as an analysis result.

[0058] (Storage unit 206) The storage unit 206 has a function of temporarily or permanently storing various data such as data related to the processing of the control unit 202 and operation program files of the control unit 202. The storage unit 206 may also have a video DB 207 and a recognition result DB 208.

[0059] (Video DB207, Recognition result DB208) The video DB 207 is a database that stores the video acquired by the video acquisition unit 201. The recognition result DB 208 is a database that stores the analysis results by each recognition engine. The data stored in the video DB 207 and the recognition result DB 208 may be evaluated separately and used to adjust each recognition model. The data stored in the video DB 207 and the recognition result DB 208 may be stored outside the recognition device 20.

[0060] (Communications Department 209) The communication unit 209 has functions as a transmitting unit and a receiving unit that transmits and receives information to and from the control device 21. The communication unit 209 can transmit and receive data to and from the control device 21 via, for example, a network. The network may be realized by various types of LANs or the like.

[0061] The communication unit 209 transmits the analysis results by each recognition engine to the control device 21. The communication unit 209 may transmit camera identification information, which identifies which of the multiple cameras C the analysis results are based on video captured by, in association with the analysis results. The communication unit 209 may also transmit area identification information, which identifies in which area the analysis results are based on video captured by the camera C in association with the analysis results.

[0062] The communication unit 209 may transmit the analysis results each time an analysis is performed, or may transmit all of the analysis results recognized within a certain period of time that are stored in the recognition result DB 208 .

[0063] <2-2. Configuration example of the control device 21> Next, a configuration example of the control device 21 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing a configuration example of the control device 21 according to an embodiment of the present invention. As shown in Fig. 4, the control device 21 includes a communication unit 211, a control unit 212, a storage unit 215, and a display output unit 218.

[0064] (Communication unit 211) The communication unit 211 has functions as a transmission unit and a reception unit that transmits and receives information between the recognition device 20 and the GW device 30. The communication unit 211 can transmit and receive data to and from the recognition device 20 and the GW device 30 via a network, for example. The network may be realized by various LANs or the like.

[0065] The communication unit 211 is an example of an acquisition unit that acquires a result of video analysis from the recognition device 20. In addition, the communication unit 211 transmits, to the GW device 30, a lighting output request and a sound output request generated by a later-described optimization unit 213.

[0066] (Control unit 212) The control unit 212 is mainly configured with a CPU, for example, and controls the overall operation of the control device 21. The control unit 212 also has a optimization unit 213 and a display control unit 214. The control unit 212 is an example of an output control unit that controls the output of each output device based on the generation of an output request by the optimization unit 213. The control unit 212 controls the output by the lighting device L and the sound output device S, for example, by controlling the communication unit 211 to transmit an output request to the GW device 30.

[0067] (Optimization unit 213) The optimization unit 213 applies output rules stored in a rule DB 216 (described later) to the analysis results acquired by the communication unit 211, thereby generating output requests for controlling the output of each output device. The optimization unit 213 controls the output of each output device by generating output requests. The optimization unit 213 can provide a sense of unity to the outputs from the multiple output devices by generating output requests for each output device using the same analysis results.

[0068] The output request will be described in detail later, but as an example, the output request may be information representing a keyword. The keyword may be a tag for searching for content output by each output device. Furthermore, the keyword may be information indicating a pattern of content output by each output device. Furthermore, the output request may be a condition for searching for content. Furthermore, the output request may be content identification information for specifying content.

[0069] By applying output rules to generate output requests, it is possible to flexibly adjust the behavior using the output from the recognition model, making it possible to create a more suitable environment. Also, by using the output from the recognition model, it is possible to create a suitable environment without having to set detailed rules.

[0070] The optimization unit 213 may generate an output request for each output device that belongs to the same area as the camera C, based on the area identification information of the area to which the camera C belongs, capturing the video that is the analysis source of the analysis result acquired by the communication unit 211. This makes it possible to create a suitable environment for each area. Also, by controlling the output for each area using the same output rule, it is possible to centrally manage adjustments within the store.

[0071] (Display control unit 214) The display control unit 214 controls the output of the digital signage D based on the display request generated by the optimization unit 213. The display control unit 214 may search for content suitable for a keyword indicated by the display request, for example, from among content stored in a display DB 217 described later.

[0072] Furthermore, the display control unit 214 may control the communication unit 211 to acquire content suitable for a keyword indicated by a display request from an external device.

[0073] (Storage unit 215) The storage unit 215 has a function of temporarily or permanently storing various data such as data related to the processing of the control unit 212 and operation program files of the control unit 212. For example, the storage unit 215 may store information such as an IP address, a host name, and an installation location assigned to the control device 21. The storage unit 215 may also store information on connection information and specifications of each device connected to the communication unit 211. The storage unit 215 may also store information on each device of the lighting server 40 and the sound server 50 that communicate with the communication unit 211 via the GW device 30. The storage unit 215 may also accumulate analysis results acquired by the communication unit 211.

[0074] The storage unit 215 may store area information that indicates which area each output device belongs to. This allows each output device to be associated with each area. For example, the storage unit 215 may store that the digital signage D1, the lighting device L1, and the sound output device S1 in the example shown in FIG. 2 belong to the area A1.

[0075] The storage unit 215 may also include a rule DB 216 , a display DB 217 , and a content DB 218 .

[0076] (Rule DB216) The rule DB 216 is a database that stores output rules for generating an output request for controlling the output of the output device based on the analysis result acquired by the communication unit 211. The output rules stored in the rule DB 216 are set manually in response to an input from, for example, a store manager or a system manager.

[0077] The output rules stored in the rule DB 216 may include output rules that use only the analysis results from a single recognition engine as conditions. Also, the output rules stored in the rule DB 216 may include output rules that use the analysis results from multiple recognition engines as conditions.

[0078] Furthermore, the priority order of a plurality of output rules stored may be included in the rule DB 216. The optimization unit 213 may search for an output rule that matches the analysis result in order of priority order.

[0079] In addition, when there are multiple output rules that match the analysis result, the optimization unit 213 may generate only an output request based on the output rule with the highest priority. In addition, when there are multiple output rules that match the analysis result, the optimization unit 213 may generate output requests for all output rules that match the analysis result.

[0080] A specific example of the output rule will be described. As an example, the output rule may include a condition that the analysis result of a person's attributes is that the gender is "female" and the age is "20s", and the recognition result of the clothing is that the dress code is "casual" and the type is "dressy". Then, the output rule may include search keywords "Dressed-Casual" and "Young-Vivid" for searching for content to be displayed on the digital signage D as a display request generated when such conditions are met.

[0081] Furthermore, the output rules stored in the rule DB 216 may include conditions for each output device. For example, the storage unit 215 may store identification information for further identifying output devices belonging to the same area. In this case, the output rules may include conditions for such identification information, and the optimization unit 213 generates an output request for an output device that matches the identification information.

[0082] Furthermore, the output rules stored in the rule DB 216 may include conditions for statistical data obtained from a plurality of analysis results. For example, conditions for statistical data obtained from analysis results of attributes of a plurality of persons acquired within a predetermined time period may be included.

[0083] As a more specific example, the output rules may include a condition that statistical data such as "there are many customers in their twenties" is obtained from the analysis results of the attributes of multiple people acquired within a specified time. Then, as a display request generated when such a condition is met, the output rules may include a condition for searching for content to be displayed on the digital signage D, such as "excluding information for seniors."

[0084] (Display DB217) The display DB 217 is a database that stores information regarding control of the display of content on the digital signage D. The display DB 217 may store, for example, the association between the display request generated by the optimization unit 213 and the content to be displayed.

[0085] Furthermore, the display DB 217 may store video data or image data of the content to be displayed on the digital signage D. The display control unit 214 may refer to such correspondence between the display request and the content, and search for content suitable for the keyword indicated by the display request from among the content stored in the display DB 217.

[0086] Furthermore, the display DB 217 may store each piece of content by linking it to content identification information for identifying the content or a tag indicating the characteristics of the content. In this case, the optimization unit 213 may extract content that matches the display request from the display DB 217 by referring to the content identification information or the tag.

[0087] The data stored in the rule DB 216 and the display DB 217 may be stored outside the control device 21.

[0088] (Display output unit 218) The display output unit 218 connects to the digital signage D and transmits video data or image data of the content to the digital signage D. Under the control of the display control unit 214, the display output unit 218 transmits content data to each of the multiple digital signages D to which it is connected.

[0089] <2-3. Example of the configuration of the GW device 30> Next, a configuration example of the GW device 30 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing a configuration example of the GW device 30 according to an embodiment of the present invention. As shown in Fig. 5, the GW device 30 includes an output request receiving unit 310, a light output request transmitting unit 320, a sound output request transmitting unit 330, a control unit 340, and a storage unit 350.

[0090] (Output request receiving unit 310) The output request receiving unit 310 is connected to the control device 21 and receives a light output request and a sound output request from the control device 21 .

[0091] (Optical output request transmission unit 320) The light output request sending unit 320 is connected to the lighting server 40 and sends the light output request acquired by the output request receiving unit 310 to the lighting server 40. The light output request sending unit 320 may be connected to a plurality of lighting servers 40. In this case, the light output request sending unit 320 sends the light output request received by the output request receiving unit 310 to the lighting server 40 corresponding to the light output request.

[0092] (Sound output request sending unit 330) The sound output request sending unit 330 connects to the sound server 50 and transmits the sound output request acquired by the output request receiving unit 310 to the sound server 50. Note that the sound output request sending unit 330 may be connected to a plurality of sound servers 50. In this case, the sound output request sending unit 330 transmits the sound output request received by the output request receiving unit 310 to the sound server 50 corresponding to the sound output request.

[0093] (Control unit 340) The control unit 340 is mainly configured with, for example, a CPU and the like, and controls the overall operation of the GW device 30. For example, the control unit 340 controls the transmission of an output request received from the output request receiving unit 310 to the light output request transmitting unit 320 or the sound output request transmitting unit 330. The control unit 340 may provide an API (Application Programming Interface) and may control the transmission and reception of output requests via the API.

[0094] (Storage unit 350) The storage unit 350 has a function of temporarily or permanently storing various data such as data related to processing by the control unit 340 and operation program files of the control unit 340. For example, the storage unit 350 may store information such as an IP address, a host name, and an installation location assigned to the GW device 30. The storage unit 350 may also store information on the connection information and specifications of each device connected to the output request receiving unit 310, the light output request transmitting unit 320, and the sound output request transmitting unit 330.

[0095] <2-4. Example of the configuration of the lighting server 40> Next, a configuration example of the lighting server 40 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing a configuration example of the lighting server 40 according to an embodiment of the present invention. As shown in Fig. 6, the lighting server 40 includes a light output request acquisition unit 410, a storage unit 420, a control unit 430, and a lighting information output unit 440.

[0096] (Optical output request acquisition unit 410) The optical output request acquisition unit 410 is connected to the GW device 30 and receives an optical output request generated by the control device 21 from the GW device 30.

[0097] (Storage unit 420) The storage unit 420 has a function of temporarily or permanently storing various data such as data related to processing by the control unit 430 and operation program files of the control unit 430. For example, the storage unit 420 may store information such as an IP address, a host name, and an installation location assigned to the lighting server 40. The storage unit 420 may also store information on the connection information and specifications of each device to which the light output request acquisition unit 410 and the lighting information output unit 440 are connected.

[0098] The storage unit 420 may also store information regarding control of light output by the lighting device L. The storage unit 420 may store, for example, a correspondence between a light output request received by the light output request acquisition unit 410 and a lighting pattern of the lighting device L. The lighting pattern indicates, for example, the color, brightness, and lighting pattern (e.g., blinking at a predetermined cycle) of the lighting device L.

[0099] Furthermore, the storage unit 420 may store the illumination patterns of the illumination device L in association with tags that indicate features of the illumination patterns.

[0100] Furthermore, the storage unit 420 may store a plurality of lighting devices L in a group. In this case, the storage unit 420 may store the lighting patterns for the plurality of lighting devices L as the lighting patterns of the group.

[0101] (Control unit 430) The control unit 430 is mainly composed of, for example, a CPU, and controls the overall operation of the lighting server 40. For example, based on the light output request received by the light output request acquisition unit 410, the control unit 430 controls the light output of each of the multiple lighting devices L connected to the lighting information output unit 440.

[0102] The control unit 430 may, for example, refer to the correspondence between the light output request and the lighting pattern of the lighting device L, which is stored in the storage unit 420, to control the output of light by the lighting device L. More specifically, the control unit 430 may control the output of light by the lighting device L by controlling the lighting information output unit 440 to transmit, to the lighting device L, lighting information indicating a lighting pattern matching the light output request received by the light output request acquisition unit 410.

[0103] The control unit 430 may provide an API and may transmit and receive light output requests and lighting information via the API.

[0104] (Lighting information output unit 440) The illumination information output unit 440 transmits the illumination information generated by the control unit 430 to the connected illumination device L. The illumination device L outputs light in accordance with the illumination information.

[0105] <2-5. Example of the configuration of the sound server 50> Next, a configuration example of the sound server 50 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing a configuration example of the sound server 50 according to an embodiment of the present invention. As shown in Fig. 7, the sound server 50 includes a sound output request acquisition unit 510, a storage unit 520, a control unit 530, and a sound information output unit 540.

[0106] (Sound output request acquisition unit 510) The sound output request acquisition unit 510 is connected to the GW device 30 and receives a sound output request generated by the control device 21 from the GW device 30.

[0107] (Storage unit 520) The storage unit 520 has a function of temporarily or permanently storing various data such as data related to processing by the control unit 530 and operation program files of the control unit 530. For example, the storage unit 520 may store information such as an IP address, a host name, and an installation location assigned to the sound server 50. The storage unit 520 may also store information on the connection information and specifications of each device to which the sound output request acquisition unit 510 and the sound information output unit 540 are connected.

[0108] Furthermore, the storage unit 520 may store information regarding control of sound output by the sound output device S. The storage unit 520 may store, for example, a correspondence between a sound output request received by the sound output request acquisition unit 510 and sound information output by the sound output device S. The sound information may be, for example, information for the sound output device S to play music, sound effects, or the like.

[0109] Furthermore, the storage unit 520 may store sound information in association with a sound tag that indicates a feature of the sound information.

[0110] (Control unit 530) The control unit 530 is mainly configured with, for example, a CPU, and controls the overall operation of the sound server 50. For example, the control unit 530 controls the output of sound by each of the multiple sound output devices S to which the sound information output unit 540 is connected, based on the sound output request received by the sound output request acquisition unit 510.

[0111] The control unit 530 may, for example, control the output of sound by the sound output device S by referring to a sound tag representing a feature of sound information stored in the storage unit 520. More specifically, the control unit 530 may control the output of sound by the sound output device S by controlling the sound information output unit 540 to transmit to the sound output device S sound information corresponding to a sound tag that matches the sound output request received by the sound output request acquisition unit 510.

[0112] Note that the method by which control unit 530 controls the output of sound suitable for a sound output request is not limited to the above method. For example, control unit 530 may acquire sound information such as music suitable for a keyword indicated by the sound output request from an external source.

[0113] Control unit 530 may provide an API, and may transmit and receive a sound output request and sound information via the API.

[0114] (Sound information output unit 540) The sound information output unit 540 transmits the sound information generated by the control unit 530 to the connected sound output device S. The sound output device S outputs sound in accordance with the sound information.

[0115] <<3. Application Examples>> So far, configuration examples of the devices included in the information processing system 1 according to the embodiment of the present invention have been described. From here, application examples of the information processing system 1 according to the embodiment of the present invention will be described.

[0116] Here, an application example of control of output by output devices by the information processing system 1 according to the embodiment of the present invention will be described with reference to Fig. 8 and Fig. 9. Fig. 8 and Fig. 9 show an application example in which the output of each output device is controlled in the layout of store B shown in Fig. 2.

[0117] Fig. 8 is a diagram for explaining the control of the output of each output device when a customer P1 visits a store B having the layout shown in Fig. 2. As shown in Fig. 8, a customer P1 is present in an area A1 near an entrance E of the store B, and a camera C1, a digital signage D1, a lighting device L1, and a sound output device S1 are installed in the area A1.

[0118] Fig. 9 is a diagram for explaining an example of generating an output request based on an image captured by the camera C1 shown in Fig. 8. First, the recognition device 20 acquires a person image, which is an image of a customer P1, by the camera C1 and a landscape image seen through a window W shown in Fig. 8.

[0119] The recognition device 20 then performs attribute recognition on the customer P1 from the person video, and obtains an analysis result that the gender of the customer P1 is "female" and the age group is "20s." The recognition device 20 also performs clothing recognition on the customer P1 from the person video, and obtains an analysis result that the dress code of the clothes worn by the customer P1 is "casual" and the type is "dressy."

[0120] Furthermore, the recognition device 20 recognizes the environment from the landscape image and obtains the analysis result that the weather is "sunny" and the brightness is "high."

[0121] Then, the control device 21 acquires each analysis result from the recognition device 20 and generates an output request for each output device by applying an output rule to the analysis result. For example, the control device 21 in this application example generates search keywords "Dressed-Casual", "Young-Vivid", and "Clear Weather", which are display requests, by applying an output rule to the analysis results for attribute recognition, clothing recognition, and environment recognition.

[0122] 8, the control device 21 controls the display of the content d1 found based on the search word on the digital signage D1. The control device 21 also controls the output of the lighting device L1 and the sound output device S1 by transmitting a sound output request and a light output request to the lighting server 40 and the sound server 50 via the GW device 30. In this way, by controlling the output of multiple output devices using the same analysis result, the multiple output devices can be linked to create an atmosphere suitable for the environment in which the digital signage D is installed.

[0123] <<4. Operation Processing>> Next, the flow of operation processing of the information processing system 1 according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a sequence diagram showing an example of the flow of operation processing of the information processing system 1 according to this embodiment.

[0124] First, the image acquisition unit 201 of the recognition device 20 acquires a person image of a person captured by the camera C (S101). The image acquisition unit 201 also acquires a landscape image captured by the camera C (S102).

[0125] The control unit 202 analyzes the person video and the scenery video using a recognition engine (S103). For example, the control unit 202 may analyze the attributes of the person shown in the person video using an attribute recognition engine 203. The control unit 202 may also analyze the clothes of the person shown in the person video using a clothing recognition engine 204. Furthermore, the control unit 202 may analyze the weather and brightness of the environment shown in the scenery video using an environment recognition engine 205.

[0126] Then, the communication unit 209 transmits the analysis results from each recognition engine to the control device 21 (S104).

[0127] The optimization unit 213 of the control device 21 applies the output rule to the analysis result acquired by the communication unit 211 (S105). As a result, an output request for each output device is generated.

[0128] The communication unit 211 transmits the generated light output request and sound output request to the GW device 30 (S106). Meanwhile, the display control unit 214 extracts content data suitable for the generated display request (S107). For example, the display control unit 214 may refer to the association stored in the display DB 217 and extract content data suitable for the display request from a plurality of content data stored in the display DB 217.

[0129] The display output unit 218 transmits the extracted content data to the digital signage D (S108). The digital signage D displays the content using the received content data (S109).

[0130] Meanwhile, the light output request transmitting unit 320 of the GW device 30 transmits the light output request acquired by the output request receiving unit 310 to the lighting server 40 (S110). The control unit 430 of the lighting server 40 controls the light output of the connected lighting device L based on the light output request acquired by the light output request acquiring unit 410 (S111). For example, the control unit 430 may refer to the correspondence between the light output request and the lighting pattern stored in the storage unit 420, and control the lighting device L to output light in a lighting pattern suitable for the light output request.

[0131] Furthermore, the optical output request transmitting unit 320 of the GW device 30 transmits the sound output request acquired by the output request receiving unit 310 to the sound server 50 (S112). The control unit 530 of the sound server 50 controls the output of sound from the connected sound output device S based on the sound output request acquired by the sound output request acquiring unit 510 (S113). For example, the control unit 530 may search for sound information suitable for the sound output request from a plurality of pieces of sound information stored in the storage unit 520, and control the sound output device S to output the searched sound information.

[0132] <<5. Modifications>> The operational process of the embodiment of the present invention has been described above. Next, a modified example of the embodiment of the present invention will be described.

[0133] So far, an example has been described in which the environment recognition engine 205 of the recognition device 20 outputs the weather, brightness, etc. from the environment shown in the landscape image as the analysis result of the environment. However, the environment analyzed by the environment recognition engine 205 is not limited to this.

[0134] In this modification, an example will be described in which the environment recognition engine 205 analyzes a display image of a sales floor of products sold in a store, captured by a camera C.

[0135] First, an example of the arrangement of the output devices according to this modified example will be described. Fig. 11 is a diagram for explaining an example of the arrangement of the output devices according to the embodiment of the present invention. Fig. 11 shows digital signages D2-D4, a sound output device S2, and lighting devices L2-L8, and a camera C2, which are installed in area A2 in store B. Product display shelves DS1 and DS2 are installed in area A2.

[0136] Camera C2 captures an image of customer P2 in area A2. Camera C2 also captures images of products displayed on display shelves DS1 and DS2. Digital signage system 2 then controls the output of digital signage D2-D4, sound output device S2, and lighting devices L2-L8 based on the analysis results of the person video and display video acquired by camera C2. This allows digital signage system 2 to control the output from each output device to suit the analysis results by camera C2. More specifically, digital signage system 2 can present area A2 to suit customer P2 and the products displayed on display shelves DS1 and DS2.

[0137] 11, the output devices may be installed across multiple areas, for example, digital signage D4 and lighting device L8 are installed across areas A2 and A3. In this case, digital signage system 2 may control the output of digital signage D4 and lighting device L8 based on the analysis results of the video captured by camera C2 and the video captured by camera C3.

[0138] The environment recognition engine 205 of the recognition device 20 analyzes the display image acquired by the image acquisition unit 201 to analyze the environment shown in the image. The environment recognition engine 205 may analyze the display image and output the types of products displayed on the display shelves as the analysis results. With this configuration, products are automatically recognized from the display image, so that the administrator's work can be saved compared to the case where product and shelf allocation information is individually registered.

[0139] When the display shelf is divided into a plurality of blocks by partitions, etc., the environment recognition engine 205 may output the type of displayed products for each block as an analysis result. The environment recognition engine 205 may execute the analysis for a shelf or block registered in advance, or may automatically recognize the shelf and block and execute the analysis.

[0140] Furthermore, the storage unit 206 in this modification may store the correspondence between the display shelves or blocks and each output device. For example, the block on the left side of the display shelf DS1 in FIG. 11 as seen from the camera C2 side may be stored in correspondence with the lighting device L2. Also, the display shelf DS1 may be stored in correspondence with the digital signage D3. Such correspondence may be set in advance by an administrator or the like, or may be automatically recognized based on the display image captured by the camera C2 and the position information of each output device. Such correspondence may be stored in the storage unit 420 of the lighting server 40 or the storage unit 520 of the sound server 50.

[0141] Furthermore, the output rules stored in the rule DB 216 of the control device 21 may include conditions based on output requests generated for other output devices. As a specific example, the rule DB 216 may store an output rule for generating a light output request to the lighting device L based on a display request to the digital signage D. For example, the rule DB 216 may store a light output rule that, when a product name is included in a search word that is a display request to the digital signage D, the lighting device L corresponding to the display shelf DS is caused to blink on the condition that the display shelf DS on which a product with that product name is displayed is associated with the display shelf DS.

[0142] Next, an application example of control of output by output devices by the information processing system 1 according to this modification will be described with reference to Fig. 12 and Fig. 13. Fig. 12 and Fig. 13 show an application example in which the output of each output device is controlled in the layout of store B shown in Fig. 11.

[0143] Fig. 12 is a diagram for explaining the control of the output of each output device when customer P2 visits store B having the layout shown in Fig. 11. As shown in Fig. 12, customer P2 is present in area A2 of store B, where camera C2, digital signages D2-D4, lighting devices L2-L8, and sound output device S2 are installed. Here, it is assumed that block BL1 on the right side of display shelf DS1 as seen from customer P2 and lighting device L3 are associated in advance. It is also assumed that display shelf DS1 and digital signage D3 are associated in advance.

[0144] Fig. 13 is a diagram for explaining an example of generating an output request based on an image captured by the camera C2 shown in Fig. 12. First, the recognition device 20 acquires a person image, which is an image of a customer P2, by the camera C2 and a display image of the products displayed on the display shelves DS1 and DS2 shown in Fig. 12.

[0145] The recognition device 20 then performs attribute recognition on the customer P2 from the person image, and obtains an analysis result that the gender of the customer P2 is "male" and the age group is "20s." The recognition device 20 also performs clothing recognition on the customer P2 from the person image, and obtains an analysis result that the dress code of the clothes worn by the customer P2 is "formal" and the type is "dressy."

[0146] The recognition device 20 also recognizes the displayed products from the display image and obtains an analysis result that "jackets" are displayed in "Block 1" indicating block BL1. Similar analysis may be performed for the other blocks and shelves, but a description thereof will be omitted here.

[0147] The control device 21 acquires each analysis result from the recognition device 20, and generates an output request for each output device by applying an output rule to the analysis result. For example, the control device 21 in this application example generates search keywords "Dressed-Formal," "Young-Stylish," and "Men's Casual Wear," which are display requests, by applying an output rule to the analysis results of attribute recognition, clothing recognition, and display product recognition. Furthermore, the control device 21 generates a search word "jacket" for "display 3" representing the digital signage D3. Note that search words may also be generated individually for the other digital signages D.

[0148] In addition, the control device 21 in this application example outputs tags "Daylight" and "Calm Pattern" that are light output requests by applying output rules to the analysis results of attribute recognition, clothing recognition, and display product recognition. As a result, the lighting devices L2 to L8 set as a group may be controlled by the lighting server 40 to output light according to the lighting pattern narrowed down based on the tags.

[0149] Furthermore, based on the search word "jacket" generated as the display request, the control device 21 generates a light output request "blink indicating block 1" which indicates to blink the lighting device L corresponding to the block BL1 in which the jackets are displayed. As a result, the lighting device L3 installed on the upper level of the block BL1 is controlled to blink, so that it is possible to emphasize to the customer P the location where the product displayed on the digital signage D3 is displayed.

[0150] <<6. Hardware configuration example>> The embodiments of the present invention have been described above. Various types of information processing performed by the above-mentioned information processing system 1 are realized by cooperation between software and the hardware of the information processing system 1 which will be described below.

[0151] FIG. 14 is a block diagram showing an example of a hardware configuration of an information processing device 900 that embodies the information processing system 1 (more specifically, for example, the recognition device 20, the control device 21, the GW device 30, the lighting server 40, or the sound server 50) according to this embodiment.

[0152] As shown in FIG. 14, the information processing device 900 includes a CPU 901, a ROM 902, a RAM 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, a drive 911, and a communication device 913.

[0153] The CPU 901 functions as an arithmetic processing device and control device, and controls the overall operation of the information processing device 900 in accordance with various programs. The CPU 901 may be a microprocessor. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901, and parameters that change appropriately in the execution of the programs. The CPU 901, ROM 902, and RAM 903 are connected to each other by a host bus 904 that is composed of a CPU bus and the like. The functions of the control unit 140 described above are realized by the cooperation of the CPU 901, ROM 902, and RAM 903.

[0154] The host bus 904 is connected to an external bus 906, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 905. Note that the host bus 904, bridge 905, and external bus 906 do not necessarily need to be separated, and the functions of these may be implemented in a single bus.

[0155] The input device 908 is composed of an input means for a user to input information, such as a mouse, a keyboard, a touch panel, a button, a switch, or a microphone, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. A user who operates the information processing device 900 can input various data to the information processing device 900 and instruct the information processing device 900 to perform processing operations by operating the input device 908.

[0156] The input device 908 may be configured with a device that detects information related to the work. For example, the input device 908 may be configured to include various sensors such as an image sensor (e.g., a camera), a depth sensor (e.g., a stereo camera or a ToF sensor), an acceleration sensor, a gyro sensor, a geomagnetic sensor, a light sensor, a sound sensor, a distance sensor, and a force sensor.

[0157] The output device 909 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, an OLED (Organic Light Emitting Diode) device, or a lamp, or a sound output device such as a speaker.

[0158] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 910 may realize the function of the storage unit 110, for example.

[0159] The drive 911 is a reader / writer for a storage medium, and is externally attached to the information processing device 900. The drive 911 reads out information recorded on a removable storage medium 912, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 903. The drive 911 is also capable of writing information to the removable storage medium 912.

[0160] The communication device 913 is a communication interface configured with a communication device for performing communication, etc. The communication device 913 may be a wireless LAN (Local Area Network) compatible communication device, or may be a wired communication device that performs wired communication.

[0161] Note that the hardware configuration of the information processing device 900 is not limited to the configuration shown in Fig. 14. For example, the information processing device 900 may not include the input device 908 or the output device 909. Also, a part or all of the configuration shown in Fig. 14 may be realized by one or more integrated circuits (ICs).

[0162] <<7. Supplementary Information>> Although the preferred embodiment of the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such an example. It is clear that a person having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present invention.

[0163] For example, in the above embodiment, an example has been described in which the association of each output device for each area is stored in the storage unit 215 of the control device 21, but the association of each output device for each area is not limited to being stored in the storage unit 215, and area identification information may be stored in the control device 21, the lighting server 40, or the sound server 50 for each type of output device. For example, area identification information indicating the area in which the lighting device L is present may be stored in the storage unit 420 of the lighting server 40. Moreover, each output device does not need to be associated with each area, and may be associated arbitrarily according to the design.

[0164] In the above embodiment, the attributes and clothing are mainly analyzed from the person image, and the weather and brightness are analyzed from the landscape image. However, the captured image and the analysis content are not limited to this. For example, the inside of a store or an area may be captured, and the number of customers present in the store or area may be analyzed from the captured image by a recognition model. In this case, an output rule including a condition of the number of customers or the customer density in the store or area may be set. Then, the output rule may be set so that an output request is generated to control each output device to output a display, light, or sound that guides the flow of customers according to the condition.

[0165] In addition, in the above embodiment, examples of output devices being digital signage D, lighting device L, and sound output device S have been mainly described, but the type of output device is not particularly limited and may include, for example, an air conditioning device, an odor output device, or a tactile presentation device.

[0166] Furthermore, the series of processes performed by the information processing device described in this specification may be realized using any of software, hardware, and a combination of software and hardware. The programs constituting the software are stored in advance, for example, in a storage medium (non-transitory medium) provided inside or outside each device. Then, each program is loaded into a RAM, for example, when executed by a computer, and executed by a processor such as a CPU. The storage medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. Furthermore, the computer program may be distributed, for example, via a network, without using a storage medium. [Explanation of symbols]

[0167] 1. Information Processing Systems 2. Digital signage system 20 Recognition device 201 Video Acquisition Unit 202 Control section 203 Attribute Recognition Engine 204 Clothing Recognition Engine 205 Environmental Recognition Engine 206 Memory section 207 Video DB 208 Recognition result DB 209 Communications Department 21 Control device 211 Communications Department 212 Control section 213 Optimization Section 214 Display control unit 215 Storage section 216 Rules DB 217 Display DB 218 Display output section 30 GW equipment 40 Lighting Server 50 Sound Server D Digital signage L Lighting device S Sound output device

Claims

1. an acquisition unit that acquires an analysis result obtained by analyzing a sensing result of an environment in which the display device is installed using a recognition model; an output control unit that controls output of the display device and an output device associated with the display device based on application of a preset rule to the analysis result; An information processing device comprising:

2. The output control unit controls outputs of the plurality of output devices associated with the display device; The plurality of output devices include a lighting device and a sound output device. The information processing device according to claim 1 .

3. The information processing apparatus according to claim 1 , wherein the acquisition unit acquires an analysis result obtained by analyzing an image of a person present in the environment, which is the sensing result, using the recognition model.

4. The information processing apparatus according to claim 3 , wherein the acquisition unit acquires an analysis result including at least one of an age, a sex, and a clothing of the person.

5. The information processing apparatus according to claim 1 , wherein the acquisition unit acquires an analysis result including a weather or brightness of the environment.

6. The display device and the output device are installed in a store, The acquisition unit acquires an analysis result obtained by analyzing the displayed products using the recognition model from an image of a sales floor of the products sold in the store, which is the sensing result. The information processing device according to any one of claims 1 to 4.

7. The output control unit controls outputs of a plurality of display devices and a plurality of output devices installed in the store, The plurality of display devices and the plurality of output devices are associated with each other for each area in which the plurality of display devices and the plurality of output devices are installed. The information processing device according to any one of claims 1 to 4.

8. 5. The information processing device according to claim 1, wherein the output control unit generates control information for controlling output for each of the display device and the output device from the same analysis result.

9. The acquisition unit acquires a plurality of analysis results analyzed by a plurality of recognition models, the output control unit generates control information for controlling output for each of the display device and the output device based on the plurality of analysis results. The information processing device according to claim 8.

10. The output control unit controls the output of the output device by controlling the control information about the output device to be transmitted to a device having a virtual assistant function. The information processing device according to claim 8.

11. The information processing device according to claim 1 , wherein the output control unit applies the rule to statistical data obtained from a plurality of analysis results.

12. The information processing device according to any one of claims 1 to 4, wherein the output control unit obtains keywords by applying predetermined rules to the analysis results, and controls the display device and the output device to output content searched for from the keywords.

13. an acquisition unit that acquires an analysis result obtained by analyzing a sensing result of an environment in which the display device is installed using a recognition model; an output control unit that controls output of the display device and an output device associated with the display device based on application of a preset rule to the analysis result; An information processing system comprising:

14. Obtaining an analysis result obtained by analyzing a sensing result of an environment in which the display device is installed using a recognition model; controlling an output of the display device and an output device associated with the display device based on applying a preset rule to the analysis result; An information processing method implemented by a computer.

15. Computer, an acquisition unit that acquires an analysis result obtained by analyzing a sensing result of an environment in which the display device is installed using a recognition model; an output control unit that controls output of the display device and an output device associated with the display device based on application of a preset rule to the analysis result; A program that functions as a

Images