Information processing device and information processing method

Abstract

To facilitate background image settings.SOLUTION: An information processing device according to an embodiment includes an acquisition unit, a storage unit, and a classification unit. The acquisition unit acquires an image of a target area. The storage unit stores the image acquired by the acquisition unit as a candidate background image to be used for object detection. The classification unit classifies the candidate images into categories according to a similarity of their color information.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing method.

Background Art

[0002] Conventionally, a technique for detecting a specified object from an image taken from the ceiling in a facility such as a factory has been disclosed. For example, in the prior art, a system has been disclosed that detects an object such as a worker who has entered a preset area such as a manufacturing line and gives a notification (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, there has been room for improvement in facilitating the setting of the background image. Specifically, when detecting an object based on the difference from the background image, the background image changes due to factors such as sunlight and shadows, so an appropriate setting of the background image according to the situation is required.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing apparatus and an information processing method capable of facilitating the setting of the background image.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, an information processing apparatus according to the present invention includes an acquisition unit, a storage unit, and a classification unit. The acquisition unit acquires an image obtained by photographing a target area. The storage unit stores the image acquired by the acquisition unit as a candidate image for a background image used for object detection. The classification unit classifies the candidate images for each category according to the similarity of the color information of each of the candidate images.

Advantages of the Invention

[0007] According to the present invention, the setting of the background image can be facilitated.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0009] The information processing apparatus 20 according to the embodiment described below includes an acquisition unit 23a that acquires an image obtained by photographing a target area, a storage unit 22 that stores the image acquired by the acquisition unit 23a as a candidate image for a background image to be used for object detection, and a classification unit 23b that classifies the candidate images for each category according to the similarity of the color information of each candidate image.

[0010] Further, the information processing apparatus 20 according to the embodiment described below includes a determination unit 23c that determines a background image for each category from the candidate images for each category, and a detection unit 23d that detects an object existing in the target area from the difference between the target image, which is an image of the object to be detected, and the background image determined by the determination unit 23c.

[0011] Further, in the information processing apparatus 20 according to the embodiment described below, the determination unit 23c presents the candidate images for each category to the user, and determines the candidate image selected by the user as the background image.

[0012] Further, in the information processing apparatus 20 according to the embodiment described below, the determination unit 23c determines a plurality of background images for each category, and the detection unit 23d detects an object using the background image whose color information is similar to that of the target image among the plurality of background images.

[0013] Further, the information processing apparatus 20 according to the embodiment described below includes a notification unit 23e that notifies an alert when an object is detected by the detection unit 23d in a set area within the target area.

[0014] Further, in the information processing apparatus 20 according to the embodiment described below, the classification unit 23b classifies the candidate images for each category according to the similarity of the histograms of the color information of the candidate images.

[0015] Further, in the information processing apparatus 20 according to the embodiment described below, the acquisition unit 23a acquires an image from a camera included in a lighting device installed in the facility.

[0016] In addition, the information processing method according to the embodiment described below includes an acquisition step of acquiring an image obtained by photographing a target area, a storage step of storing the image acquired in the acquisition step as a candidate image for a background image to be used for object detection, and a classification step of classifying the candidate images for each category according to the similarity of the color information of each candidate image.

[0017] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that each of the embodiments described below does not limit the technology disclosed by the present invention. In addition, the same reference numerals are given to the same parts in each embodiment, and redundant explanations are omitted.

[0018] First, the outline of the lighting system according to the embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing the outline of the lighting system according to the embodiment. As shown in FIG. 1, the lighting system 1 includes a lighting device 10, an information processing device 20, and a notification device 30.

[0019] The lighting system 1 is introduced into a factory. A plurality of lighting devices 10 and a plurality of notification devices 30 are provided at a work site where a manufacturing line or the like exists. The lighting device 10 is a lighting device having a camera (a lighting device with a camera).

[0020] The lighting device 10 is installed, for example, on the ceiling of the work site. Therefore, the camera of the lighting device 10 can photograph an image looking down on the work site. Therefore, the lighting system 1 can analyze such an image to detect non-safe behaviors that may lead to accidents at the work site, and make proposals for improving work efficiency through, for example, pedestrian flow analysis.

[0021] The information processing device 20 remotely controls the lighting device 10 and the notification device 30 via the network N. In this example, the network is a wide area network such as the Internet, and the information processing device 20 is an external server device or the like provided outside the factory. Note that the information processing device 20 may be provided inside the factory. The functions of the information processing device 20 may be distributed and provided in a device (for example, a management device) provided inside the factory and an external server device provided outside the factory. The external server device may be realized on a cloud system.

[0022] The notification device 30 is composed of, for example, a speaker, a warning light, etc., and outputs sounds and the like provided from the information processing device 20 via the network N. For example, the notification device 30 outputs a voice regarding an alert.

[0023] Next, with reference to FIG. 2, the outline of object detection according to the embodiment will be described. FIG. 2 is a schematic explanatory diagram of object detection according to the embodiment. Object detection according to the embodiment detects an object based on a difference image Gd that is the difference between the background image Gc and the real-time image Gr.

[0024] In the example shown in FIG. 2, the real-time image Gr shows the objects L1 and L2, and the background image Gc shows only the object L2. Therefore, since the object L1 remains in the difference image Gd between the background image Gc and the real-time image Gr, the object L1 can be detected.

[0025] For example, in object detection according to the embodiment, an object shown in the real-time image Gr is detected from the color difference between the background image Gc and the real-time image Gr. By the way, since the background changes due to factors such as sunlight and shadows, an appropriate setting of the background image Gc according to the situation is required.

[0026] Therefore, while it is necessary to collect a plurality of background images Gc in response to environmental changes, it is time-consuming to collect and set these background images Gc. Thus, the information processing apparatus 20 according to the embodiment automatically acquires candidate images that are candidates for the background image Gc, and classifies the candidate images for each category.

[0027] As a result, for example, the user can select the background image Gc from the candidate images classified for each category, so that the setting of the background image can be facilitated.

[0028] Next, with reference to FIG. 3, a configuration example of the information processing apparatus 20 according to the embodiment will be described. FIG. 3 is a block diagram of the information processing apparatus 20 according to the embodiment. As shown in FIG. 3, the information processing apparatus 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0029] The communication unit 21 is a communication module that performs data communication with the lighting device 10 and the notification device 30.

[0030] The storage unit 22 is, for example, a semiconductor memory element such as a flash memory, or a storage device such as an HDD or an optical disk. In the example shown in FIG. 2, the storage unit 22 includes an image storage unit 22a, a background image storage unit 22b, and a notification condition storage unit 22c.

[0031] The image storage unit 22a stores images. For example, the image storage unit 22a stores candidate images that are candidates for the background image Gc. FIG. 4 is a diagram showing an example of information stored in the image storage unit 22a. Note that FIG. 4 shows information stored for one location (camera).

[0032] As shown in FIG. 4, the image storage unit 22a stores information on items such as "image ID", "shooting time", "image data", "histogram", and "category" in association with each other.

[0033] In the "Image ID" item, an identifier for identifying each image is stored. In the "Shooting Time" item, information regarding the date and time when the image identified by the corresponding Image ID was shot is stored.

[0034] In the "Image Data" item, data of the image itself identified by the corresponding Image ID is stored. In the "Histogram" item, a histogram of the image identified by the corresponding Image ID is stored.

[0035] In this embodiment, the histogram is a histogram regarding the color information of the image. The color information of the image includes hue, saturation, and brightness. That is, in the "Histogram" item, a histogram obtained by aggregating the color information of each pixel included in the image is stored.

[0036] In the "Category" item, information regarding the category of the image identified by the corresponding Image ID is included. The category is information indicating to which category of candidate images the corresponding image belongs.

[0037] Returning to the description of FIG. 3, the background image storage unit 22b will be described. The background image storage unit 22b stores background images. FIG. 5 is a diagram showing an example of information stored in the background image storage unit 22b according to the embodiment.

[0038] As shown in FIG. 5, the background image storage unit 22b stores information of items such as "Category", "Image Data", and "Histogram" in association with each other. In the "Category" item, information regarding the category is stored.

[0039] In the "Image Data" item, image data of the corresponding category is stored. In the "Histogram" item, a histogram of the corresponding image data is stored. In the example shown in FIG. 5, it shows that a plurality of image data such as image data "D001" and image data "D004" are included for the category "G001". Note that it is not limited to this, and one image data may be associated with one category.

[0040] Returning to the description of FIG. 3, the notification condition storage unit 22c will be described. The notification condition storage unit 22c stores notification conditions. The notification condition is a condition for notification through the notification device 30 (see FIG. 1) when an object is detected.

[0041] FIG. 6 is a diagram showing an example of information stored in the notification condition storage unit 22c according to the embodiment. As shown in FIG. 6, the notification condition storage unit 22c stores information such as "condition ID", "condition 1", and "condition 2" in association with each other.

[0042] The "condition ID" is an identifier for identifying the corresponding condition. "Condition 1" and "condition 2" are conditions for notification through the notification device 30 (see FIG. 1), respectively. In the example of FIG. 6, it indicates that notification is to be made when two conditions are satisfied, such as "condition 1" and "condition 2".

[0043] For example, regarding "condition 1" and "condition 2", they can be arbitrarily changed according to the facility where the lighting device 10 is installed, and the number of conditions may be one or three or more.

[0044] In the example shown in FIG. 6, for condition ID "A001", it indicates that the notification condition is satisfied when a worker is detected in area #1, which is a preset area (for example, a restricted access area). For condition ID "A002", it indicates that the notification condition is satisfied when a state where the worker is lying down (that is, the worker has fallen) is detected.

[0045] Note that the notification conditions shown in FIG. 6 are only examples and can be arbitrarily changed. Also, the object to be detected is not limited to a worker and may include machines, forklifts, carts, etc.

[0046] Returning to the description of FIG. 3, the control unit 23 will be described. The control unit 23 includes, for example, a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), input / output ports, and various circuits.

[0047] The control unit 23 includes an acquisition unit 23a, a classification unit 23b, a determination unit 23c, a detection unit 23d, and a notification unit 23e. Each function of the control unit 23 is realized, for example, by the CPU of the control unit 23 reading and executing a program stored in the RAM, ROM, or storage unit 22 of the control unit 23.

[0048] Note that part or all of the acquisition unit 23a, the classification unit 23b, the determination unit 23c, the detection unit 23d, and the notification unit 23e may be configured by hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0049] The acquisition unit 23a acquires an image from each lighting device 10 via the communication unit 21. The acquisition unit 23a stores the acquired image in the storage unit 22 and passes a target image (corresponding to the real-time image Gr) to be the object of object detection to the detection unit 23d.

[0050] Also, as will be described later, when a request for acquiring the background image Gc is made, the acquisition unit 23a stores a plurality of images taken at a predetermined period (interval of 30 minutes) in the image storage unit 22a. Such images are used as candidate images that are candidates for the background image Gc.

[0051] The classification unit 23b classifies the candidate images for each category according to the similarity of the color information of each candidate image. Specifically, the classification unit 23b generates a histogram regarding the color information of the images (i.e., candidate images) stored in the image storage unit 22a.

[0052] First, the classification unit 23b generates a histogram for each candidate image by aggregating the hues of each pixel in the candidate image. Subsequently, the classification unit 23b compares each histogram exhaustively, classifies candidate images with similar histograms into the same category, and classifies candidate images with dissimilar histograms into separate categories.

[0053] Note that the classification unit 23b may also classify candidate images into each category based on the analysis of each candidate image, such as the presence or absence of shadows and the presence or absence of light reflection on the floor surface. That is, in the present embodiment, color information is a concept that includes, in addition to hue, saturation, and lightness, the presence or absence of shadows and the presence or absence of light reflection on the floor surface. Further, the classification unit 23b may divide a candidate image into a plurality of regions and classify the candidate image into each category for each region.

[0054] The determination unit 23c determines the background image Gc for each category from the candidate images for each category. Specifically, the determination unit 23c presents candidate images for each category to the user (for example, the manager at the work site), and determines the candidate image selected by the user as the background image Gc.

[0055] Here, with reference to FIG. 7, the selection screen for the background image Gc will be described. FIG. 7 is a diagram showing an example of the selection screen according to the embodiment. As shown in FIG. 7, buttons B1 to B4 and an image display area A1 are displayed on the selection screen.

[0056] Button B1 is a button for requesting the start of acquisition of the background image. In the present embodiment, when the user selects button B1, a series of processes such as acquisition of candidate images is started. Button B2 is a button for adding an image (background image Gc) from the LIVE video.

[0057] For example, when the user selects button B2, the current real-time image Gr can be added to the candidate images. Button B3 is a button for adding an image saved in a file to the candidate images, and button B4 is a button for deleting the candidate images.

[0058] Also, as shown in FIG. 7, in the image display area A1, a plurality of candidate images corresponding to the category selected by the user are displayed. In the example shown in FIG. 7, the case where Category 1 is selected is shown, and the case where 16 candidate images corresponding to Category 1 are displayed is shown.

[0059] For example, the user can select any number of images from the candidate images displayed in the image display area A1 as the background image, and the determination unit 23c determines the image selected by the user as the background image.

[0060] At this time, it is preferable that the user selects, as the background image Gc, an image in which, for example, no person is reflected among the candidate images. That is, the determination unit 23c can determine the background image Gc with a simple process without the need to confirm by image processing or the like that a person is reflected, etc., by leaving the determination of the final background image to the visual inspection of the user. Note that the determination unit 23c may automatically determine the background image Gc from the candidate images using an arbitrary algorithm.

[0061] Through the above processing, the processing until the background image Gc is determined is completed. Hereinafter, returning to the description of FIG. 3, the actual object detection processing using the background image will be described. The detection unit 23d detects an object existing in the target area from the difference between the target image, which is an image to be the object detection target, and the background image Gc determined by the determination unit 23c.

[0062] First, the detection unit 23d acquires the real-time image Gr, which is the target image, from the acquisition unit 23a, and generates a histogram regarding the color information of the real-time image Gr. Subsequently, the detection unit 23d compares the histogram of the real-time image Gr with the histogram of the background image Gc, and determines the background image Gc with the most similar histogram as the background image Gc for the current real-time image Gr.

[0063] In this way, the detection unit 23d can improve the object detection accuracy by selecting the background image Gc with the closest color tone to the real-time image Gr from among the plurality of background images. Then, the detection unit 23d generates a difference image Gd that is the difference between the background image Gc and the real-time image Gr. Then, the detection unit 23d detects an object from the difference image Gd using a technique such as VQA (Visual Question Answering). After that, the detection unit 23d outputs the object detection result to the notification unit 23e.

[0064] When the object detected by the detection unit 23d satisfies the notification condition stored in the notification condition storage unit 22c, the notification unit 23e notifies through the notification device 30 (see FIG. 1). For example, when a person is detected in a preset restricted entry area or when an obstacle (e.g., cardboard) is detected on a passage for a forklift or the like, the notification unit 23e issues an alert.

[0065] Next, the processing procedure executed by the information processing apparatus 20 according to the embodiment will be described with reference to FIGS. 8 and 9. FIG. 8 is a flowchart showing an example of the background image determination process according to the embodiment. FIG. 9 is a flowchart showing an example of the object detection process according to the embodiment.

[0066] First, the background image determination process will be described with reference to FIG. 8. As shown in FIG. 8, first, the information processing apparatus 20 receives an image acquisition start operation by, for example, the button B1 shown in FIG. 7 (step S101).

[0067] Subsequently, the information processing apparatus 20 acquires images from the lighting device 10 at a predetermined period (step S102) and generates a histogram (step S103). Subsequently, the information processing apparatus 20 classifies the images into categories according to the similarity of the histograms (step S104). [[ID=!7]]

[0068] Subsequently, the information processing apparatus 20 determines whether or not an end condition is satisfied (step S105). For example, the information processing apparatus 20 determines that the end condition is satisfied when it receives an image acquisition stop operation from the user or when the number of candidate images classified into each category reaches a specified number.

[0069] When the information processing apparatus 20 determines that the end condition is satisfied (step S105; Yes), it proceeds to the process of step S106. When it determines that the end condition is not satisfied (step S105; No), it repeatedly executes the processes after step S102.

[0070] Subsequently, the information processing apparatus 20 displays candidate images for each category to the user (step S106), determines the background image Gc according to the user's selection operation (step S107), and ends the process.

[0071] Next, the object detection process will be described with reference to FIG. 9. As shown in FIG. 9, the information processing apparatus 20 acquires a real-time image Gr (an example of a target image) from the lighting apparatus 10 (step S201), and generates a histogram regarding the color information of the real-time image Gr (step S202).

[0072] Subsequently, the information processing apparatus 20 selects the background image Gc for the real-time image Gr according to the similarity between the histogram of the real-time image Gr and the histogram of the background image Gc (step S203).

[0073] Subsequently, the information processing apparatus 20 detects an object based on the difference between the real-time image Gr and the background image Gc (step S204). Subsequently, the information processing apparatus 20 determines whether or not the detection result of step S204 satisfies a notification condition (step S205).

[0074] When the information processing apparatus 20 determines that the notification condition is satisfied (step S205; Yes), it performs notification (step S206) and then ends the process. When it determines that the notification condition is not satisfied (step S205; No), it ends the process.

[0075] As described above, the information processing apparatus 20 according to the embodiment includes an acquisition unit 23a that acquires an image obtained by photographing a target area, a storage unit 22 that stores the image acquired by the acquisition unit 23a as a candidate image for a background image to be used for object detection, and a classification unit 23b that classifies the candidate images for each category according to the similarity of the color information of each candidate image.

[0076] Therefore, according to the information processing apparatus 20 according to the embodiment, since the background image Gc can be determined based on the classified candidate images, the setting of the background image Gc can be facilitated.

[0077] Incidentally, in the above-described embodiment, the case where an image is acquired from the camera included in the lighting device 10 has been described. However, the lighting device 10 and the camera may be separate devices. Further, an image may be acquired from a camera such as a security camera.

[0078] Although the embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

Explanation of Reference Numerals

[0079] 1 Lighting system 10 Lighting device 20 Information processing apparatus 21 Communication unit 22 Storage unit 22a Image storage unit 22b Background image storage unit 22c Notification condition storage unit 23 Control unit 23a Acquisition unit 23b Classification unit 23c Determination unit 23d detection unit 23e notification unit 30 notification device Gc background image Gd difference image Gr real-time image

Claims

1. An acquisition unit that acquires an image obtained by photographing a target area; A storage unit that stores the image acquired by the acquisition unit as a candidate image for a background image to be used for object detection; A classification unit that classifies the candidate images for each category according to the similarity of the color information of each of the candidate images; An information processing apparatus comprising the above.

2. A determination unit that determines the background image for each category from the candidate images for each category; A detection unit that detects an object existing in the target area from the difference between a target image that is the image to be the object detection target and the background image determined by the determination unit; The information processing apparatus according to claim 1, comprising the above.

3. The determination unit presents the candidate images for each category to the user, and determines the candidate image selected by the user as the background image The information processing apparatus according to claim 2.

4. The determination unit determines a plurality of the background images for each category, The detection unit detects an object using the background image whose color information is similar to that of the target image among the plurality of the background images The information processing apparatus according to claim 2.

5. An alert unit that alerts when the object is detected by the detection unit in a set area within the target area; The information processing apparatus according to claim 2, comprising the above.

6. The classification unit classifies the candidate images for each category according to the similarity of the histogram regarding the color information of the candidate images The information processing apparatus according to claim 1.

7. The acquisition unit acquires the image from a camera included in a lighting device installed in the facility The information processing apparatus according to claim 1.

8. An information processing method executed by a computer, comprising: an acquisition step of acquiring an image obtained by photographing a target area; a storage step of storing the image acquired in the acquisition step as a candidate image for a background image to be used for object detection; a classification step of classifying the candidate images for each category according to the similarity of the color information of each of the candidate images; An information processing method including the above.

Images