Learning data generation device, automatic door system, learning data generation method, trained model generation method, control program, and recording medium

Abstract

To generate efficient and highly reliable learning data.SOLUTION: A learning data generation device (100) includes: a moving image data creation unit (10) for creating moving image data in which a three-dimensional model that is a detection object is operated in a virtual space; a change data creation unit (20) for creating luminance change data or differential data in a two-dimensional image obtained by projecting the moving image data; a labeling unit (30) for attaching a label indicating that it is a detection object to a region corresponding to the detection object in the luminance change data or the differential data; and a learning data generation unit (40) for generating a set of the luminance change data or differential data, the region corresponding to the detection object, and a label attached by the labeling unit (30), as the learning data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a training data generation device that generates training data used in machine learning, a control program for the training data generation device, a recording medium on which the control program is recorded, a trained model generation method using training data generated by the training data generation device, and an automatic door system that includes a detection unit that detects moving objects using the trained model. [Background technology]

[0002] In machine learning using deep learning and the like, a large amount of training data is required for efficient learning. However, when creating training data for a trained model used in a sensor that detects moving objects, for example, the training data has traditionally been created by manually labeling moving objects captured in captured images. This has required considerable effort. Therefore, various methods for efficiently creating training data have been proposed.

[0003] Patent Document 1 describes a computer program that captures an image of a real object, derives the posture of the real object using a 3D model corresponding to the real object, and associates appearance information of the real object with a 2D model obtained from the 3D model.

[0004] Patent Document 2 describes a learning data generation device that projects a three-dimensional space in which a three-dimensional model with associated attributes and a background are modeled in a virtual space onto a two-dimensional plane, generates labels for the two-dimensional objects onto which the three-dimensional models are projected, and associates the two-dimensional images with the labels.

[0005] Patent document 3 describes a learning data generation device that generates scene data for generating learning data in which a specific object model has a specific color, sets an area for the specific object model, and associates an image for the scene data with the specific object area.

[0006] Patent Document 4 describes an object recognition device that projects a created learning image onto feature points in an eigenspace and outputs information on the feature points that are closest to the input image. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2018-51410 [Patent Document 2] International Publication No. WO2020 / 183598 [Patent Document 3] Japanese Patent Application Publication No. 2019-23858 [Patent Document 4] Japanese Patent Application Laid-Open No. 2010-211732 Summary of the Invention [Problem to be solved by the invention]

[0008] However, there is room for improvement in the above-mentioned conventional techniques, and it is possible to generate more efficient and reliable training data. One aspect of the present invention has been made in view of the above-mentioned problems, and its purpose is to realize a training data generation device that generates efficient and reliable training data. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, a training data generation device according to one embodiment of the present invention comprises: a video data generation unit that generates video data in which a three-dimensional model of a detection target moves in a virtual space; a variation data generation unit that generates brightness change data or difference data in a two-dimensional image projected from the video data; a labeling unit that assigns a label indicating that the detection target is an area in the brightness change data or difference data that corresponds to the detection target; and a training data generation unit that generates, as training data, a combination of the brightness change data or the difference data, the area corresponding to the detection target, and the label assigned by the labeling unit.

[0010] A training data generation device according to one embodiment of the present invention uses brightness change data or difference data in a two-dimensional image that outputs moving image data to generate training data for training a trained model that estimates a detection target area in a two-dimensional image and a label assigned to the detection target area.

[0011] According to the above configuration, since the created video data includes a detection target, the detection target area can be automatically identified and the detection target area can be automatically labeled. Furthermore, since brightness change data or difference data in a two-dimensional image output from the video data is used, the processing load can be reduced compared to when the video data is used as is. Therefore, learning data can be created efficiently. Furthermore, since labeling is performed using the generated video data, accurate labeling can be performed, and highly reliable learning data can be created.

[0012] Furthermore, a learning model using the learning data created by the learning data generation device can obtain estimation results by inputting brightness data or difference data, thereby reducing the processing load in the estimation process of the learning model.

[0013] In one embodiment of the learning data generation device of the present invention, the display unit that displays the two-dimensional image may include an event generation unit that generates an event at each pixel on the display unit when the brightness value of each pixel exceeds a first threshold value or falls below a second threshold value, and a position output unit that outputs the position on the display unit of the pixel at which the event generation unit generated the event, and the change data creation unit may create the brightness change data using the position output by the position output unit.

[0014] The first threshold indicates a value greater than the current luminance value, and the second threshold indicates a value less than the current luminance value. According to the above configuration, pixels whose luminance values ​​exceed the first threshold or pixels whose luminance values ​​fall below the second threshold are output together with the positions of the pixels, so that it is easy to recognize the positions where the luminance values ​​have changed beyond a predetermined range, and luminance change data can be easily created.

[0015] In the learning data generation device according to one aspect of the present invention, the moving image data generated by the moving image data generation unit may not include background data.

[0016] With this configuration, the volume of video data can be reduced because background data is not included. When using learning data for an automatic door detection sensor, it is sufficient to be able to detect passersby, who are the detection target, and background data is not necessary, so there is no problem even if background data is not included.

[0017] In the learning data generation device according to one aspect of the present invention, the detection target included in the video data generated by the video data generation unit may be a moving object.

[0018] According to the above configuration, a moving object can be detected as the detection target.

[0019] In one aspect of the learning data generation device of the present invention, the video data creation unit may generate multiple video data sets in which at least one of the direction of travel and the speed of travel of the detection object differs.

[0020] According to the above configuration, it is possible to detect moving objects that move in various directions and at various speeds as detection targets.

[0021] In the learning data generation device according to one aspect of the present invention, the video data creation unit may create a plurality of video data in different environments in which the detection target exists.

[0022] The above configuration makes it possible to detect the detection target no matter what environment the detection target is in. Here, the environment includes weather-related factors such as rain, wind, fog, snow, etc., vegetation swaying due to wind, etc., objects affected by weather, artificial lighting such as fluorescent lights, LEDs, and automobile headlights, and smoke such as cigarette smoke and steam.

[0023] In the learning data generation device according to one aspect of the present invention, the moving image data created by the moving image data creation unit may include a plurality of the detection targets.

[0024] According to the above configuration, it is possible to detect a plurality of detection targets.

[0025] In order to solve the above problem, an automatic door system according to one embodiment of the present invention includes a sensor that detects the detection target using a learning model that has been machine-learned using the learning data generated by the learning data generation device, and an automatic door that opens and closes based on the detection results of the sensor.

[0026] According to the above configuration, it is possible to improve the detection accuracy of the detection sensor for opening the automatic door. Also, it is possible to perform the detection process using a learning model with a light processing load.

[0027] In order to solve the above problem, a training data generation method according to one aspect of the present invention includes a moving image data generation step of generating moving image data in which a 3D model of a detection target moves in a virtual space, a variation data generation step of generating brightness variation data or difference data in a 2D image projected from the moving image data, a labeling step of assigning a label indicating that the detection target is located in the brightness variation data or the difference data, to a region corresponding to the detection target, and a training data generation step of generating a set of the brightness variation data or the difference data, the region corresponding to the detection target, and the label assigned in the labeling step as training data, thereby achieving the above-mentioned effects.

[0028] In order to solve the above problem, a trained model generation method according to one aspect of the present invention includes a training data generation step of generating training data using the training data generation method, and a machine learning step of performing machine learning using the training data generated in the training data generation step.

[0029] The training data generation device according to each aspect of the present invention may be realized by a computer. In this case, the control program for the training data generation device, which causes the computer to operate as each part (software element) of the training data generation device, thereby realizing the training data generation device on the computer, and the computer-readable recording medium on which the control program is recorded, also fall within the scope of the present invention. [Effects of the Invention]

[0030] According to one aspect of the present invention, automatic labeling is possible and the processing load can be reduced, thereby providing the effect of efficiently creating training data. Furthermore, since labeling is performed using generated video data, the effect of creating highly reliable training data is also provided. Furthermore, a learning model with a light processing load can be created. [Brief explanation of the drawings]

[0031] [Figure 1] 1 is a functional block diagram showing a configuration of a main part of a learning data generation device according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of a three-dimensional model used in a learning data generation device. [Figure 3] FIG. 10 is a diagram showing an example in which a three-dimensional model is placed in a virtual space and converted into a two-dimensional image. [Figure 4] FIG. 10 is a diagram showing an example of brightness change data of a two-dimensional image. [Figure 5] FIG. 10 is a diagram showing an example of setting an area of ​​a three-dimensional model in brightness change data. [Figure 6] FIG. 10 is a diagram illustrating an example of learning data. [Figure 7]FIG. 2 is a diagram illustrating a configuration of a main part of a learning device. [Figure 8] 1 is a diagram showing the main configuration of an automatic door system. [Figure 9] 10 is a flowchart showing a processing flow in the learning data generation device. [Figure 10] FIG. 10 is a diagram showing a modified example of moving image data. [Figure 11] FIG. 10 is a diagram showing a modified example of moving image data. [Figure 12] FIG. 10 is a diagram showing a modified example of moving image data. [Figure 13] FIG. 10 is a diagram showing a modified example of moving image data. DETAILED DESCRIPTION OF THE INVENTION

[0032] An embodiment of the present invention will be described in detail below. The learning data generation device 100 according to this embodiment generates learning data used for training the trained model 5. In this embodiment, an example of creating learning data used for training the trained model 5 used for detecting a detection target by the detection unit 2 provided in the automatic door system 1 will be described, but the learning data generated by the learning data generation device 100 is not limited to this. It may also be used as a learning model for sensors other than automatic doors, such as shutters.

[0033] 1 is a functional block diagram showing an example of the configuration of a training data generation device 100 according to this embodiment. As shown in FIG. 1, the training data generation device 100 includes a video data creation unit 10, a variation data creation unit 20, a labeling unit 30, a training data generation unit 40, and a display unit 50. Note that the display unit 50 may be provided outside the training data generation device 100.

[0034] The video data creation unit 10 creates video data in which a 3D model of the detection target moves in a virtual space. More specifically, the video data creation unit 10 first creates a 3D model of the detection target. The 3D model can be created using known technology, and is, for example, a moving object such as a person or a vehicle. Information representing attributes is associated with the 3D model. Examples of attributes include gender and age for a person, and type and color for a vehicle.

[0035] An example of a 3D model to be created is shown in Figure 2. Figure 2 shows an example of an adult male 201. As mentioned above, a 3D model can be created using known techniques, and is created by setting various parameters (position, rotation, scaling, dimensions, etc.).

[0036] The video data creation unit 10 then places the created 3D model in a 3D virtual space and sets the movement of the 3D model. The movement of the 3D model is set by the user. This creates video data in which the 3D model moves within the 3D virtual space. Note that the video data creation unit 10 may create multiple video data in which the 3D model moves in different directions and at different speeds. It may also create video data in which the surrounding conditions of the 3D model are different. Different surrounding conditions refer to situations in which there is nothing around the 3D model, situations in which part of the 3D model is hidden by something other than the 3D model, etc.

[0037] The video data creation unit 10 may or may not create background data that serves as the background in the three-dimensional virtual space. The background refers to the part of the virtual space other than the three-dimensional model.

[0038] The variation data creation unit 20 creates brightness variation data or difference data for a two-dimensional image onto which moving image data is projected. More specifically, the variation data creation unit 20 first projects a three-dimensional model in a three-dimensional virtual space onto a two-dimensional plane to draw a two-dimensional object. The two-dimensional object can be drawn using any known method, and any method may be used.

[0039] An example of a two-dimensional image in which a two-dimensional object is drawn on a two-dimensional plane is shown in Fig. 3. In the example of the two-dimensional image shown in Fig. 3, a two-dimensional image 211 of a created three-dimensional model of an adult male 201 is shown.

[0040] Then, the variation data creation unit 20 creates brightness variation data for a two-dimensional image 211 in which the three-dimensional model is projected onto a two-dimensional plane. FIG. 4 shows an example of brightness variation data. FIG. 4 shows a two-dimensional image created by accumulating brightness variation data for each pixel over a certain period of time. Details of the brightness variation data will be described later, but as shown in FIG. 4, the brightness variation data makes it possible to recognize the positions of pixels whose brightness values ​​exceed a first threshold value and pixels whose brightness values ​​fall below a second threshold value. Note that, instead of brightness variation data, difference data between frames may be created.

[0041] The labeling unit 30 assigns a label indicating that the region in the brightness change data or difference data corresponds to the detection target. More specifically, the labeling unit 30 calculates the region where the drawn two-dimensional object exists and labels the region using the attribute associated with the object. For example, if the two-dimensional object is a projection of a three-dimensional model with the attribute "male," the region where the two-dimensional object exists is labeled "male." The attribute to be labeled may be all or part of the attributes associated with the corresponding three-dimensional model. Furthermore, the region where the two-dimensional object exists may be a rectangular region circumscribing the two-dimensional object. FIG. 5 shows the region labeled by the labeling unit 30. In the example shown in FIG. 5, a rectangular region 231 circumscribing an adult male 221 is set as the target region to be labeled. Here, the rectangular region 231 is labeled "male."

[0042] The learning data generating unit 40 generates, as learning data, a set of brightness change data or difference data, an area indicating the detection target, and a label assigned by the labeling unit. An example of learning data is shown in Fig. 6. As shown in Fig. 6, learning data 400 is a set of brightness change data or difference data, a detection target region, and a label, and serves as learning data for a learning model in which the brightness change data or difference data is input data and the detection target region and the label are output data.

[0043] The display unit 50 is a display device that displays a two-dimensional image. The display unit 50 includes an event generation unit 51 and a position output unit 52. Here, the display unit 50 is described as being included in the training data generation device 100, but as described above, the display unit 50 may be located outside the training data generation device 100.

[0044] The event generation unit 51 generates an event when the brightness value of each pixel of the display unit 50 exceeds a first threshold value or falls below a second threshold value. The first threshold value and the second threshold value are set as percentages of a reference value. For example, the first threshold value is set to +20% of the reference value, and the second threshold value is set to -15% of the reference value. The reference value is the current brightness value, and when an event occurs, the brightness value after the event becomes the reference value.

[0045] As described above, the display unit 50 that displays a two-dimensional image includes an event generation unit 51 that generates an event at each pixel on the display unit 50 when the brightness value of that pixel exceeds a first threshold value or falls below a second threshold value, and a position output unit 52 that outputs the position on the display unit 50 of the pixel at which the event generation unit 51 generated the event.

[0046] The position output unit 52 outputs the position on the display unit 50 of the pixel at which the event generation unit 51 generated an event. The position output unit 52 may output the time at which the event occurred along with the position. The variation data creation unit 20 creates brightness variation data using the position output by the position output unit 52.

[0047] As described above, the training data generation device 100 according to this embodiment comprises a video data generation unit 10 that generates video data in which a three-dimensional model of a detection target moves in a virtual space; a variation data generation unit 20 that generates brightness change data or difference data in a two-dimensional image projected from the video data; a labeling unit 30 that assigns a label indicating that the detection target is an area in the brightness change data or difference data that corresponds to the detection target; and a training data generation unit 40 that generates, as training data, a combination of the brightness change data or the difference data, the area indicating the detection target, and the label assigned by the labeling unit.

[0048] [Configuration of learning device] Next, the learning device 200 will be described with reference to Fig. 7. As shown in Fig. 7, the learning device 200 includes a learning unit 210. The learning unit 210 performs machine learning using the learning data 400 generated by the learning data generation device 100, and generates a trained model 5. The trained model 5 is a trained model that receives brightness change data or difference data as input, and outputs a detection target region and a label.

[0049] In other words, the trained model generation method by the learning device 200 includes a machine learning step of performing machine learning using the training data 400 generated by the training data generation device 100 in the training data generation step.

[0050] [Configuration of automatic door system] Next, the automatic door system 1 will be described with reference to Fig. 8. The automatic door system 1 shown in Fig. 8 uses a trained model 5 trained using training data 400 created by a training data generation device 100 for the detection process of a detection target by the detection unit 2. Note that the operation of the automatic door system 1 can be achieved using known technology, and therefore a detailed description thereof will be omitted here.

[0051] As shown in FIG. 8, the automatic door system 1 includes a detection unit (sensor) 2, a drive unit 3, an automatic door 4, and a trained model 5.

[0052] The detection unit 2 detects the presence of a person in a predetermined area, and when a person is present, transmits a drive instruction to the drive unit 3. As described above, in this embodiment, the detection unit 2 uses the trained model 5 to perform processing to detect whether or not a person is present in the predetermined area.

[0053] The driving unit 3 operates to open the automatic door 4 in response to an instruction from the detection unit 2.

[0054] The automatic door 4 is an automatic door that is opened and closed by a drive unit 3.

[0055] Here, the trained model 5 is described as being included in the automatic door system 1, but the trained model 5 may not be included in the automatic door system 1, and the detection unit 2 may access an external trained model 5 to obtain estimation results using the trained model 5.

[0056] [Processing flow in the learning data generation device] Fig. 9 is a flowchart showing the flow of processing in the learning data generation device 100. As shown in Fig. 9, in the learning data generation device 100, first, the moving image data creation unit 10 creates a three-dimensional model (S101).

[0057] Next, the moving image data creation unit 10 uses the created 3D model to create moving image data in which the 3D model of the detection target moves in a virtual space (S102, moving image data creation step). After that, the variation data creation unit 20 projects the 3D model in the 3D virtual space onto a 2D plane to draw a 2D object and create a 2D image (S103). Then, brightness variation data or difference data for the 2D image onto which the moving image data is projected is created (S104, variation data creation step).

[0058] Thereafter, the labeling unit 30 assigns a label based on the attribute to a region in the brightness change data or difference data corresponding to the detection target (S105, labeling step).Then, the learning data generating unit 40 generates, as learning data, a set of the brightness change data or difference data, the region indicating the detection target, and the label assigned by the labeling unit 30 (S106, learning data generating step).

[0059] The above is the flow of the process for generating learning data in the learning data generation device 100.

[0060] [Modification] Next, modified examples of video data used to create learning data 400 will be described with reference to Figs. 10 to 13. In the above-described example, the video data was generated by arranging a created 3D model in a virtual space. There may be multiple 3D models, as shown in Fig. 10. In the example shown in Fig. 10, three adult males, designated by 202 to 204, are shown as 3D models arranged in the virtual space. The 3D models are not limited to adult males, but may also be females or children.

[0061] When the labeling unit 30 performs labeling, rectangular areas 231 to 234 circumscribing the adult males 202 to 204, respectively, are set as target areas for labeling.

[0062] In this way, the moving image data created by the moving image data creating unit 10 may include a plurality of three-dimensional models.

[0063] Furthermore, as shown in Fig. 11, the three-dimensional model may not be a person but may be a vehicle or the like. In the example shown in Fig. 11, the three-dimensional model is a forklift 205. In this case, the labeling unit 30 sets a rectangular area 235 circumscribing the forklift 205 as the target area to be labeled. A trained model trained using training data using a forklift or the like as a three-dimensional model can be used for an open / close sensor for a shutter or the like through which a forklift passes.

[0064] 12 and 13, various weather conditions may be set for the video data. FIG. 12 shows an example in which a 3D model 206 is placed in the rain. FIG. 13 shows an example in which a 3D model 207 is placed in the wind. In this way, by placing the 3D models in various weather conditions, it becomes possible to recognize the area in which the 3D models exist under various weather conditions.

[0065] In this way, the video data creation unit 10 may create multiple video data sets in which the 3D model exists under different weather conditions. The video data creation unit 10 may also create video data sets in which the 3D model exists under various environments, not limited to weather conditions. Here, the term "environment" includes not only naturally occurring environments such as the weather conditions described above, but also artificially generated environments. For example, it includes fluorescent lights, LEDs, automobile headlights, cigarette smoke, steam, and other fumes. It also includes natural light and artificial light that transmits, reflects, refracts, and scatters.

[0066] In this way, the environment includes both naturally occurring and artificially occurring things. In other words, the environment can be defined as anything that affects a 3D model, such as (1) solids such as snow and vegetation, (2) liquids such as rain, and (3) gases such as wind, fog, and smoke.

[0067] [Software implementation example] The functions of the learning data generation device 100 (hereinafter referred to as the "device") can be realized by a program for causing a computer to function as the device, and a program for causing a computer to function as each control block of the device (in particular, the video data creation unit 10, the variation data creation unit 20, the labeling unit 30, and the learning data generation unit 40).

[0068] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0069] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0070] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits functioning as the control blocks are formed are also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0071] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0072] 1 Automatic door system 2. Detection unit (sensor) 3 Drive unit 4 Automatic doors 5. Pre-trained model 10 Video data creation unit 20 Change data creation section 30 Labeling Department 40 Learning data generation unit 50 Display 51 Event Generation Section 52 Position output section 100 Learning data generation device 200 Learning Device 210 Learning Department 400 training data

Claims

1. a moving image data creating unit that creates moving image data in which a three-dimensional model of the detection target moves in a virtual space; a variation data generating unit that generates brightness variation data or difference data between frames obtained by comparing brightness values ​​with a threshold value in a two-dimensional image projected from the moving image data; a labeling unit that assigns a label indicating that the detection target is a region in the brightness change data or the difference data that corresponds to the detection target; a learning data generation unit that generates, as learning data, a set of the brightness change data or the difference data, an area corresponding to the detection object, and a label assigned by the labeling unit.

2. the display unit that displays the two-dimensional image includes an event generation unit that generates an event in each pixel on the display unit when a luminance value of the pixel exceeds a first threshold value or falls below a second threshold value, and a position output unit that outputs a position on the display unit of the pixel at which the event generation unit has generated the event; The learning data generating device according to claim 1 , wherein the variation data generating unit generates the luminance variation data using the positions output by the position output unit.

3. 3. The learning data generating device according to claim 1, wherein the video data generated by the video data generating unit does not include background data.

4. 4. The learning data generation device according to claim 1, wherein the detection target included in the video data generated by the video data generation unit is a moving object.

5. The learning data generation device according to claim 4 , wherein the video data generation unit generates a plurality of video data sets in which at least one of the traveling direction and traveling speed of the detection target differs.

6. The learning data generating device according to claim 1 , wherein the video data generating unit generates a plurality of video data sets in different environments in which the detection target exists.

7. 7. The learning data generation device according to claim 1, wherein the moving image data generated by the moving image data generation unit includes a plurality of the detection targets.

8. a sensor that detects the detection target using a learning model that is machine-learned using the learning data generated by the learning data generation device according to any one of claims 1 to 7; and an automatic door that opens and closes based on the detection results of the sensor.

9. a moving image data creation step of creating moving image data in which the three-dimensional model of the detection target moves in a virtual space; a variation data creation step of creating brightness variation data or difference data between frames obtained by comparing brightness values ​​with a threshold value in a two-dimensional image projected from the moving image data; a labeling step of assigning a label indicating that the detection target is a region in the brightness change data or the difference data corresponding to the detection target; a learning data generation step of generating, as learning data, a set of the brightness change data or the difference data, an area corresponding to the detection target, and a label assigned in the labeling step.

10. a learning data generating step of generating learning data by the learning data generating method according to claim 9; a machine learning step of performing machine learning using the learning data generated in the learning data generation step.

11. A control program for causing a computer to function as the learning data generation device described in claim 1, the control program causing a computer to function as the video data creation unit, the change data creation unit, the labeling unit, and the learning data generation unit.

12. A computer-readable recording medium on which the control program according to claim 11 is recorded.

13. A moving image data creation unit that creates moving image data in which a three-dimensional model of a detection target moves in a virtual space; a variation data generating unit that generates brightness variation data or difference data in a two-dimensional image projected from the moving image data; a labeling unit that assigns a label indicating that the detection target is a region in the brightness change data or the difference data that corresponds to the detection target; a learning data generation unit that generates, as learning data, a set of the luminance change data or the difference data, an area corresponding to the detection target, and a label assigned by the labeling unit; the display unit that displays the two-dimensional image includes an event generation unit that generates an event in each pixel on the display unit when a luminance value of the pixel exceeds a first threshold value or falls below a second threshold value, and a position output unit that outputs a position on the display unit of the pixel at which the event generation unit has generated the event; The variation data creation unit creates the luminance variation data using the position output by the position output unit.

Images