Information processing system and information processing method

Abstract

To detect an object considering incident of external light.SOLUTION: An information processing system according to the present application has a server device and an illumination device including a camera, and the information processing system has a prediction unit, a creation unit, a composition unit, and a detection unit. The prediction unit predicts a mode in which external light is incident on an area in which the illumination device is installed, on the basis of information on a position of the illumination device and opening information indicating a position of an opening in the area. The creation unit creates a reference image to be a reference of the area on the basis of mode information indicating the mode predicted by the prediction unit. The composition unit creates a composite image obtained by combining the reference image created by the creation unit and a picked-up image picked up by the camera. The detection unit detects an object included in the picked-up image on the basis of the composite image created by the composition unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] FIELD An embodiment of the present invention relates to an information processing system and an information processing method. [Background technology]

[0002] Conventionally, there are known techniques for detecting objects using a camera. As one example of such a technique, a technique has been proposed in which an image captured by a lighting device having a camera is acquired, a moving object and attributes in the captured image are detected, and a notification is given based on the detection result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-056286 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned technology does not disclose how to detect an object while taking into account the incidence of external light.

[0005] For example, in the above-mentioned technology, when a captured image is input, moving objects and attributes in the captured image are detected using a trained model generated to output detection results of moving objects and attributes, but the technology does not disclose how to detect objects taking into account the incidence of external light.

[0006] The present application has been made in view of the above, and has an object to detect an object while taking into account the incidence of external light. [Means for solving the problem]

[0007] The information processing system of this embodiment is an information processing system having a server device and a lighting device equipped with a camera, and is characterized by having: a prediction unit that predicts the manner in which external light will be incident on an area based on position information of the lighting device and opening information indicating the position of an opening in the area in which the lighting device is installed; a generation unit that generates a reference image that serves as a reference for the area based on the appearance information indicating the appearance predicted by the prediction unit; a synthesis unit that generates a composite image by synthesizing the reference image generated by the generation unit and an image captured by the camera; and a detection unit that detects an object included in the image captured based on the composite image generated by the synthesis unit. Effect of the Invention

[0008] According to the present invention, it is possible to detect an object while taking into account the incidence of external light. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of an information processing system 1 according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of the information processing device 10 according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of the lighting device information database 31 according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of the configuration of a lighting device 100 according to an embodiment. [Diagram 5] FIG. 5 is a flowchart showing an example of the operation of the information processing system 1 according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, a mode for carrying out an information processing device according to the present application (hereinafter, referred to as an "embodiment") will be described in detail with reference to the drawings. Note that the information processing device according to the present application is not limited to this embodiment. Also, the same components in the following embodiments are given the same reference numerals, and duplicated descriptions will be omitted.

[0011] The information processing system 1 according to the embodiment described below is an information processing system having a server device (information processing device 10) and a lighting device 100 equipped with a camera, and includes a prediction unit 41, a generation unit 141, a synthesis unit 142, and a detection unit 143. The prediction unit 41 predicts the manner in which external light is incident on the area based on the position information of the lighting device 100 and opening information indicating the positions of openings in the area where the lighting device 100 is installed. The generation unit 141 generates a reference image that serves as a reference for the area based on the manner information indicating the manner predicted by the prediction unit 41. The synthesis unit 142 generates a composite image by synthesizing the reference image generated by the generation unit 141 and the captured image captured by the camera. The detection unit 143 detects an object included in the captured image based on the composite image generated by the synthesis unit 142.

[0012] In the information processing system 1 according to the embodiment described below, the information processing device 10 includes a prediction unit 41 and a transmission unit 42, and the lighting device 100 includes a generation unit 141, a synthesis unit 142, and a detection unit 143. The prediction unit 41 predicts the manner in which external light is incident on the area based on the position information of the lighting device 100 and opening information indicating the position of the opening in the area where the lighting device 100 is installed. The transmission unit 42 transmits to the lighting device 100 the manner information indicating the manner predicted by the prediction unit 41. The generation unit 141 generates a reference image that serves as a reference for the area based on the manner information. The synthesis unit 142 generates a composite image by synthesizing the reference image generated by the generation unit 141 and the captured image captured by the camera. The detection unit 143 detects an object included in the captured image based on the composite image generated by the synthesis unit 142.

[0013] The prediction unit 41 according to the embodiment described below predicts the aspect for each predetermined time period. The synthesis unit 142 generates a synthetic image by synthesizing the captured image with a reference image generated based on aspect information indicating an aspect corresponding to the time period when the captured image was captured.

[0014] The prediction unit 41 according to the embodiment described below predicts the weather conditions for each weather. The composition unit 142 generates a composite image by combining the captured image with a reference image generated based on the weather conditions information indicating the weather conditions at the position indicated by the position information when the captured image was captured.

[0015] The prediction unit 41 according to the embodiment described below predicts the aspect for each season. The composition unit 142 generates a composite image by combining the captured image with a reference image generated based on aspect information indicating an aspect corresponding to the season at the position indicated by the position information when the captured image was captured.

[0016] Moreover, the prediction unit 41 according to the embodiment described below further predicts the state based on installation information related to an installation that has been installed in advance in the area.

[0017] The prediction unit 41 according to the embodiment described below further predicts the state based on moving object information related to the moving object moving in the area. The composition unit 142 generates a composite image by combining the captured image with a reference image generated based on state information indicating the state corresponding to the position of the moving object when the captured image was captured.

[0018] Furthermore, the prediction unit 41 according to the embodiment described below further predicts the manner in which the light source illuminates the region based on installation position information indicating the position at which the light source is installed in the region. Furthermore, the generation unit 141 generates a reference image based on mode information indicating the manner in which external light is incident on the region and the manner in which the light source installed in the region illuminates the region.

[0019] [Embodiment] (An example of an information processing system) An example of an information processing system 1 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of an information processing system 1 according to an embodiment. In the example shown in Fig. 1, the information processing system 1 has an information processing device 10 and a lighting device 100. The information processing device 10 and the lighting device 100 are connected to each other via a predetermined communication network (not shown) so as to be able to communicate with each other by wire or wirelessly. Note that the information processing system 1 shown in Fig. 1 may include a plurality of information processing devices 10 and a plurality of lighting devices 100.

[0020] The information processing device 10 is an example of a server device according to the present application, and is an information processing device (computer) that functions as a so-called cloud server. For example, the information processing device 10 manages location information indicating the location where the lighting device 100 is installed. As a specific example, the information processing device 10 manages GPS (Global Positioning System) information (e.g., latitude and longitude) of the lighting device 100 as the location information.

[0021] Furthermore, the information processing device 10 manages opening information relating to the positions and shapes of openings (windows and the like) through which external light enters in a building (e.g., a factory, a warehouse, and the like) #1 in which the lighting device 100 is installed. For example, the information processing device 10 manages data (e.g., Building Information Modeling (BIM) data) for reproducing the positions and shapes of openings in software that reproduces an object in a three-dimensional shape, as the opening information.

[0022] The lighting device 100 is a lighting device with a camera. For example, the lighting device 100 is a device in which a camera configuration is integrated into a lighting fixture. For example, the lighting device 100 is a lighting device (for example, a base light, ceiling light, downlight, or spotlight) that is installed on the ceiling of a building #1 and illuminates the floor surface when turned on.

[0023] Furthermore, the camera (camera unit 120) included in the lighting device 100 is an imaging device that captures an image of a predetermined area around the lighting device 100. In the example of Fig. 1, the camera included in the lighting device 100 captures an image of an area #1 corresponding to the angle of view.

[0024] Hereinafter, the information processing performed by the information processing device 10 will be described with reference to FIG. 1. First, the information processing device 10 predicts the manner in which external light is incident on the region #1 based on the position information of the lighting device 100 and the opening information of the building #1 (step S1). For example, the information processing device 10 predicts the manner in which external light is incident on the region #1 for each weather in each time period (for example, morning time period (5:00 to 10:00), daytime time period (10:00 to 15:00), evening time period (15:00 to 18:00), etc.) based on information such as the position of the sun and the intensity of sunlight for each weather in each time period at the position indicated by the position information of the lighting device 100, and the opening information of the building #1. As a specific example, the information processing device 10 predicts the change in color, saturation, brightness, etc. of the region #1 due to the external light as the manner in which external light is incident on the region #1.

[0025] Next, the information processing device 10 transmits to the lighting device 100 aspect information indicating an aspect in which external light is incident on the area #1 (step S2). Next, the lighting device 100 generates a reference image serving as a reference for the area #1 for each weather in each time period based on the aspect information transmitted from the information processing device 10 (step S3). For example, the lighting device 100 generates an image of the area #1 affected by external light as a reference image for each weather in each time period. As a specific example, the lighting device 100 generates a reference image in a state in which no object (product, etc.) is placed on the building #1.

[0026] Next, the lighting device 100 generates a composite image by combining the reference image with the captured image captured by the lighting device's camera (step S4). For example, the lighting device 100 generates a composite image by combining a reference image corresponding to the time and weather when the captured image was captured with the captured image from among the reference images. As a specific example, the lighting device 100 generates an image as the composite image by controlling the parameters of the reference image (e.g., color, saturation, brightness, etc.) based on the parameters of the captured image.

[0027] Next, the lighting device 100 detects an object included in the captured image based on the composite image (step S5). For example, the lighting device 100 detects an object included in the captured image from the composite image using a known method for object detection.

[0028] In the above example, the information processing device 10 may further predict the manner in which external light is incident on the region #1 for each season. Then, the lighting device 100 may generate a composite image by combining the captured image with a reference image corresponding to the season when the captured image was captured, and detect an object included in the captured image.

[0029] Furthermore, the information processing device 10 may further predict the manner in which external light is incident on the area #1 based on installation information (e.g., the position of an installation object) related to an installation object (e.g., a shelf or an object other than a product) previously installed in the area #1. Then, the lighting device 100 may generate a composite image by combining the captured image with a reference image generated based on the appearance information indicating the predicted appearance, and detect an object included in the captured image.

[0030] The information processing device 10 may further predict the manner in which external light is incident on the region #1 based on moving body information (e.g., the position of a moving body) related to a moving body (e.g., a transport machine or a robot that transports an object) moving in the region #1. The lighting device 100 may generate a reference image corresponding to the position of the moving body when the captured image was captured, generate a composite image by combining the captured image with the reference image, and detect an object included in the captured image. The information processing device 10 may receive the captured image from the lighting device 100, and predict the manner in which external light is incident on the region #1 based on the position of the moving body indicated by the captured image (in other words, the real-time position of the moving body).

[0031] Furthermore, the information processing device 10 may further predict the manner in which a light source (e.g., the lighting device 100 or another lighting device) illuminates the area #1 based on installation position information indicating the position at which the light source is installed in the area #1. The lighting device 100 may generate a composite image by combining the captured image with a reference image generated based on aspect information indicating the manner in which external light is incident on the area #1 and the aspect of illumination of the area #1 by the light source installed in the area #1, and detect an object included in the captured image.

[0032] Furthermore, the above-described processing by the lighting device 100 may be performed by the information processing device 10. For example, the information processing device 10 may generate a reference image based on aspect information indicating an aspect in which external light is incident on the region #1, obtain a captured image captured by the lighting device 100, and detect an object included in the captured image based on a composite image obtained by combining the reference image and the captured image.

[0033] As described above, the information processing system 1 according to the embodiment predicts the manner in which external light will be incident on the area in which the lighting device 100 is installed, and detects an object included in the captured image based on a composite image obtained by combining a reference image based on the predicted manner with a captured image captured by the camera of the lighting device 100.

[0034] Previously, in order to improve the accuracy of object detection without being affected by external light, it was necessary to eliminate the influence of external light by closing curtains over openings, etc. Furthermore, when performing object detection while being affected by external light, there were issues such as the apparent color of an object changing due to sunlight, which changes depending on the time of day and weather, resulting in the same object being mistaken for a different object or making object detection difficult.

[0035] In contrast, the information processing system 1 according to the embodiment predicts the way (state) of external light entering and detects an object based on the predicted state, thereby reducing the influence of external light and preventing erroneous detection of an object. Therefore, the information processing system 1 according to the embodiment has the effect of accurately detecting an object in an image in an environment influenced by external light.

[0036] (An example of an information processing device) Next, the configuration of the information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing device 10 according to an embodiment. As shown in Fig. 2, the information processing device 10 has a communication unit 20, a storage unit 30, and a control unit 40.

[0037] [About Communications Section 20] The communication unit 20 is realized by, for example, a network interface card (NIC), etc. The communication unit 20 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the lighting device 100 and the like.

[0038] [Regarding the memory unit 30] The storage unit 30 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 2, the storage unit 30 has a lighting device information database 31.

[0039] [About Lighting Equipment Information Database 31] The lighting device information database 31 stores various types of information related to the lighting device 100. An example of information stored in the lighting device information database 31 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the lighting device information database 31 according to an embodiment. In the example of Fig. 3, the lighting device information database 31 has items such as "lighting device ID", "area ID", "position information", "opening information", "installed object information", "mobile object information", "installation position information", and "mode information".

[0040] "Lighting device ID" indicates identification information for identifying lighting device 100. "Area ID" indicates identification information for identifying the area in which lighting device 100 is installed (in other words, the area corresponding to the angle of view of the camera of lighting device 100). "Location information" indicates the location where lighting device 100 is installed. "Opening information" indicates information about an opening of the building in which lighting device 100 is installed. "Installation information" indicates information about an installation that is installed in advance in the area. "Mobile object information" indicates information about a mobile object that moves in the area. "Installation position information" indicates the position where a light source is installed in the area. "Mode information" indicates the mode in which external light enters the area, the mode in which area #1 is illuminated by the light source installed in the area, etc.

[0041] That is, FIG. 3 shows an example in which a lighting device 100 identified by a lighting device ID "DID#1" is installed in an area identified by an area ID "AID#1", the location information of the lighting device 100 is "location information #1", the opening information of the building in which the lighting device 100 is installed is "opening information", the installation object information in the area is "installation object information #1", the moving object information is "moving object information #1", the installation position information is "installation position information #1", and the aspect information is "aspect information #1".

[0042] [Regarding the control unit 40] The control unit 40 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 10 using a RAM as a working area. The control unit 40 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). As shown in FIG. 3, the control unit 40 according to the embodiment has a prediction unit 41 and a transmission unit 42, and realizes or executes the functions and actions of information processing described below.

[0043] [About Prediction Section 41] The prediction unit 41 predicts the manner in which external light is incident on an area based on the position information of the lighting device and the opening information indicating the position of the opening in the area where the lighting device is installed. For example, in the example of Fig. 1, the prediction unit 41 refers to the lighting device information database 31 and predicts the manner in which external light is incident on the area #1 based on the position information of the lighting device 100 and the opening information of the building #1.

[0044] The prediction unit 41 may also predict the state for each predetermined time period. For example, in the example of Fig. 1, the prediction unit 41 predicts the state in which external light enters the area #1 for each time period based on information such as the position of the sun and the intensity of sunlight for each weather in the morning, afternoon, and evening time periods, and information on the openings of the building #1.

[0045] The prediction unit 41 may also predict the state of each weather. For example, in the example of Fig. 1, the prediction unit 41 predicts the state of incidence of external light into the area #1 for each weather based on information such as the position of the sun and the intensity of sunlight for each weather and information on the openings of the building #1.

[0046] The prediction unit 41 may also predict the state for each season. For example, in the example of Fig. 1, the prediction unit 41 predicts the state in which external light is incident on the area #1 for each season.

[0047] The prediction unit 41 may further predict the state based on information about objects installed in the area in advance. For example, in the example of Fig. 1, the prediction unit 41 predicts the state in which external light is incident on the area #1 based on the positions of shelves and objects other than products installed in the area #1 in advance.

[0048] The prediction unit 41 may further predict the state based on moving body information related to a moving body moving in the region. For example, in the example of Fig. 1, the prediction unit 41 predicts the state in which external light is incident on the region #1 based on the positions of a transport machine or a robot that transports an object in the region #1.

[0049] Furthermore, the prediction unit 41 may further predict the manner in which the light source irradiates the region based on installation position information indicating the position at which the light source is installed in the region. For example, in the example of Fig. 1, the prediction unit 41 predicts the manner in which the light source irradiates the region #1 based on installation position information indicating the position at which the light source is installed in the region #1.

[0050] [About transmitter 42] The transmitter 42 transmits, to the lighting device, aspect information indicating the aspect predicted by the predictor 41. For example, in the example of FIG. 1, the transmitter 42 transmits, to the lighting device 100, aspect information indicating an aspect in which external light is incident on the region #1.

[0051] (Example of lighting equipment) Next, the configuration of the lighting device 100 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the lighting device 100 according to the embodiment. For example, the lighting device 100 is an information processing device (computer) that functions as a lighting device with a camera. As shown in Fig. 4, the lighting device 100 has a communication unit 110, a camera unit 120, a lighting unit 130, a control unit 140, and a storage unit 150.

[0052] The communication unit 110 is realized by, for example, a NIC etc. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the information processing device 10 etc.

[0053] Camera unit 120 includes camera 121, camera control unit 122, and storage unit 123. Camera 121 includes an imaging element that electronically acquires an image, such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and captures a predetermined angle of view to generate image data (also called an "image"). Camera 121 is disposed at a predetermined position and captures an image of a specific space. The image data may be video data or still image data.

[0054] The camera control unit 122 controls the imaging by the camera 121. In addition, the camera control unit 122 outputs image data generated by the imaging by the camera 121 to the control unit 140.

[0055] Storage unit 123 stores programs and data for implementing various controls of camera unit 120. Storage unit 123 may store images captured by camera 121 or image data generated by capturing images by camera 121. Storage unit 123 is implemented by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

[0056] The lighting unit 130 includes a light source 131, a lighting control unit 132, and a storage unit 133. The light source 131 includes a semiconductor light emitting element such as an LED (Light Emitting Diode). The light source 131 turns on or off in a manner corresponding to a control signal output from the lighting control unit 132.

[0057] The lighting control unit 132 controls turning on and off the light source 131. The lighting control unit 132 turns on the light source 131 that is to be turned off, or turns off the light source 131 that is to be turned on. In the case of the lighting unit 130 in which the light source 131 can be dimmed, the lighting control unit 132 may control the dimming degree of the light source 131.

[0058] The storage unit 133 stores programs and data for implementing various controls of the lighting unit 130. The storage unit 133 is implemented by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Note that the storage unit 133 may be provided in the storage unit 150 described later.

[0059] The storage unit 150 stores various information related to information processing. The storage unit 150 stores programs for implementing various controls of the control unit 140. The storage unit 150 is implemented by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 150 may store information stored in the storage unit 123 or information stored in the storage unit 133.

[0060] The control unit 140 is realized, for example, by a CPU, an MPU, or the like executing various programs stored in an internal memory using a RAM as a working area. The control unit 140 can also be realized, for example, by an integrated circuit such as an ASIC or an FPGA. The control unit 140 has a generation unit 141, a synthesis unit 142, and a detection unit 143, and realizes or executes the functions and actions of information processing described below.

[0061] [About the generation unit 141] The generation unit 141 generates a reference image serving as a reference for the region based on the aspect information indicating the aspect predicted by the prediction unit 41. For example, in the example of Fig. 1, the generation unit 141 generates a reference image serving as a reference for the region #1 based on the aspect information.

[0062] The generating unit 141 may generate a reference image based on aspect information indicating the manner in which external light is incident on the region and the manner in which the region is irradiated by a light source installed in the region. For example, in the example of FIG. 1, the generating unit 141 generates a reference image based on aspect information indicating the manner in which external light is incident on the region #1 and the manner in which the region #1 is irradiated by a light source installed in the region #1.

[0063] [About the synthesis unit 142] The synthesis unit 142 generates a synthetic image by synthesizing the reference image generated by the generation unit 141 and the captured image captured by the camera. For example, in the example of Fig. 1, the synthesis unit 142 generates a synthetic image by synthesizing the reference image and the captured image captured by the camera 21.

[0064] The composition unit 142 may generate a composite image by combining a reference image generated based on aspect information indicating an aspect corresponding to a time period when the captured image was captured with the captured image. For example, in the example of Fig. 1, the composition unit 142 generates a composite image by combining a reference image corresponding to a time period when the captured image was captured with the captured image, among the reference images.

[0065] The composition unit 142 may generate a composite image by combining a reference image generated based on aspect information indicating an aspect corresponding to the weather at the position indicated by the location information when the captured image was captured with the captured image. For example, in the example of Fig. 1, the composition unit 142 generates a composite image by combining a reference image corresponding to the weather at the time when the captured image was captured with the captured image, among the reference images.

[0066] The composition unit 142 may generate a composite image by combining the captured image with a reference image generated based on aspect information indicating an aspect corresponding to the season at the position indicated by the location information when the captured image was captured. For example, in the example of Fig. 1, the composition unit 142 generates a composite image by combining the captured image with a reference image corresponding to the season when the captured image was captured.

[0067] The composition unit 142 may generate a composite image by combining a reference image generated based on aspect information indicating an aspect corresponding to the position of the moving body when the captured image was captured with the captured image. For example, in the example of Fig. 1, the composition unit 142 generates a reference image corresponding to the position of the moving body when the captured image was captured and generates a composite image by combining the captured image with the reference image.

[0068] [About detection unit 143] The detection unit 143 detects an object included in the captured image based on the composite image generated by the composition unit 142. For example, in the example of Fig. 1, the detection unit 143 detects an object included in the captured image from the composite image by using a known method for object detection.

[0069] (Processing flow of information processing system) Next, a flow of processing of the information processing system 1 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the operation of the information processing system 1 according to the embodiment.

[0070] 5, the information processing system 1 predicts the manner in which external light is incident on a region based on the position information of the lighting device and the opening information indicating the position of the opening in the region where the lighting device is installed (step S101). Next, the information processing system 1 generates a reference image that serves as a reference for the region based on the aspect information indicating the predicted aspect (step S102). Next, the information processing system 1 determines whether the lighting device 100 has captured an image (step S103). If an image has not been captured (step S103; No), the information processing system 1 waits until an image is captured.

[0071] On the other hand, if an image has been captured (step S103; Yes), the information processing system 1 generates a composite image by combining the reference image and the captured image captured by the camera (step S104). Next, the information processing system 1 detects an object included in the captured image based on the composite image (step S105) and ends the process.

[0072] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as the scope and spirit of the invention. [Explanation of symbols]

[0073] 1. Information Processing Systems 10. Information processing device 20 Communications Department 30 Storage section 31 Lighting equipment information database 40 Control section 41 Prediction Department 42 Transmitter 100 Lighting Equipment 110 Communications Department 120 Camera Unit 121 Camera 122 Camera control unit 123 Storage section 130 Lighting Unit 131 Light source 132 Lighting control section 133 Storage section 140 Control section 141 Generation part 142 Synthesis section 143 Detection unit 150 Storage section

Claims

1. An information processing system having a server device and a lighting device equipped with a camera, a prediction unit that predicts a manner in which external light is incident on an area in which the lighting device is installed, based on position information of the lighting device and opening information indicating a position of an opening in the area; a generating unit that generates a reference image serving as a reference for the region based on aspect information indicating the aspect predicted by the predicting unit; a synthesis unit that generates a synthetic image by synthesizing the reference image generated by the generation unit and the captured image captured by the camera; a detection unit that detects an object included in the captured image based on the composite image generated by the synthesis unit; An information processing system comprising:

2. The server device includes: a prediction unit that predicts a manner in which external light is incident on an area in which the lighting device is installed, based on position information of the lighting device and opening information indicating a position of an opening in the area; a transmission unit that transmits, to the lighting device, aspect information indicating the aspect predicted by the prediction unit; having The lighting device includes: a generation unit that generates a reference image serving as a reference for the region based on the aspect information; a synthesis unit that generates a synthetic image by synthesizing the reference image generated by the generation unit and the captured image captured by the camera; a detection unit that detects an object included in the captured image based on the composite image generated by the synthesis unit; 2. The information processing system according to claim 1, further comprising:

3. The prediction unit is Predicting the above-mentioned behavior for each predetermined time period; The synthesis unit is A composite image is generated by combining the reference image, which is generated based on the aspect information indicating the aspect corresponding to the time period when the captured image was captured, with the captured image.

2. The information processing system according to claim 1 .

4. The prediction unit is Predict the above-mentioned aspects for each weather condition; The synthesis unit is A composite image is generated by combining the captured image with the reference image, which is generated based on the aspect information indicating the aspect corresponding to the weather at the position indicated by the position information when the captured image was captured.

2. The information processing system according to claim 1 .

5. The prediction unit is Forecasting the seasonal patterns; The synthesis unit is A composite image is generated by combining the captured image with the reference image, which is generated based on the aspect information indicating the aspect corresponding to the season at the position indicated by the position information when the captured image was captured.

2. The information processing system according to claim 1 .

6. The prediction unit is Furthermore, the state is predicted based on installation information regarding installations that have been installed in advance in the area.

2. The information processing system according to claim 1 .

7. The prediction unit is Furthermore, the behavior is predicted based on moving object information regarding a moving object moving in the area; The synthesis unit is A composite image is generated by combining the captured image with the reference image generated based on the aspect information indicating the aspect corresponding to the position of the moving body when the captured image was captured.

2. The information processing system according to claim 1 .

8. The prediction unit is Furthermore, a state of illumination of the region by the light source is predicted based on installation position information indicating a position where the light source is installed in the region; The generation unit is The reference image is generated based on the aspect information indicating an aspect of external light entering the region and an aspect of illumination of the region by a light source installed in the region.

2. The information processing system according to claim 1 .

9. An information processing method executed by an information processing system, comprising: a prediction step of predicting a state in which external light is incident on an area in which the lighting device is installed, based on position information of the lighting device equipped with a camera and opening information indicating positions of openings in the area in which the lighting device is installed; a generating step of generating a reference image serving as a reference for the region based on aspect information indicating the aspect predicted by the predicting step; a synthesis step of generating a synthetic image by synthesizing the reference image generated in the generation step and the captured image captured by the camera; a detection step of detecting an object included in the captured image based on the composite image generated by the synthesis step; 13. An information processing method comprising:

Images