Information processing device, system and method

Abstract

Identify areas where changes are occurring in the real space being sensed. [Solution] The information processing device includes an area specifying means for specifying an area in real space corresponding to a detection signal detected by a sensor capable of detecting a change in luminance of each pixel of incident light as a signal. The area specifying means specifies an area in real space corresponding to the detection signal based on a correspondence between pixel positions and areas in real space. The information processing device includes an event specifying means for specifying an event that has occurred in real space in accordance with the area in real space specified by the area specifying means.

Description

[Technical field]

[0001] The present invention relates to an information processing device, a system, and a method. [Background technology]

[0002] There are so-called event sensors, which are sensors capable of detecting a change in the brightness of light incident on each pixel as a signal. Various systems using these event sensors have been considered. For example, a system has been disclosed that generates an event image by collecting event signals that detect droplets ejected from a dispenser in a predetermined integration time unit, and controls the ejection of droplets by the dispenser based on the generated event image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2023-080101 Summary of the Invention [Problem to be solved by the invention]

[0004] The objects of the present invention can be exemplified as follows. A first object of the present invention is to identify an area in which a change is occurring in the real space that is the subject of sensing. A second object of the present invention is to make it possible to exclude each area of ​​an event sensor, each area being made up of a plurality of pixels, from being a target of a predetermined process. A third object of the present invention is to identify an event that occurs in the real space that is the target of sensing. A fourth object of the present invention is to determine whether or not the state of the real space that is the subject of sensing is normal. A fifth object of the present invention is to determine whether or not an object exists, or to identify the position where an object exists, even if the real space being sensed is in a state where it is difficult for a human to see. A sixth object of the present invention is to specify, in the real space of the sensing target, the position where an object exists after the object has moved, or the area where the object exists. [Means for solving the problem]

[0005] The object of the present invention is to [1] An information processing device comprising: an area specifying means for specifying an area in real space corresponding to a detection signal detected by a sensor capable of detecting a change in luminance of incident light for each pixel as a signal; [2] The information processing device according to [1], wherein the area specifying means specifies an area in real space corresponding to the detection signal based on a correspondence relationship between a pixel position and an area in real space; [3] The information processing device according to [1] or [2], further comprising an event identification means for identifying an event occurring in the real space according to the area of ​​the real space identified by the area identification means; [4] The information processing device according to any one of [1] to [3] above, further comprising: information output means for outputting change information indicating that a state change has occurred in a real-space region identified by the region identification means when a plurality of pixels correspond to one real-space region and a total amount of detection signals corresponding to one real-space region exceeds a threshold value; or information storage means for storing the change information; [5] The information processing device according to [4], wherein the threshold value differs for each area in the real space; [6] The information processing device according to [4] or [5], wherein the threshold varies depending on time in real space; [7] A method executed in an information processing device, comprising: a region identifying step of identifying a region in real space corresponding to a detection signal detected by a sensor capable of detecting a change in luminance of incident light for each pixel as a signal; [8] An information processing device comprising: a determination means for determining whether or not a total amount of signals corresponding to one area exceeds a threshold when a plurality of pixels of a sensor capable of detecting a change in luminance of incident light for each pixel as a signal is divided into a plurality of areas according to the positions of the pixels, the threshold being different for each area; [9] The information processing device according to [8], further comprising a threshold specifying means for specifying a threshold for each region based on a detection signal detected by the sensor in a state where no change occurs in the real space of the object to be detected by the sensor, and the determination means makes a determination based on the threshold specified by the threshold specifying means;

[10] A method executed in an information processing device, comprising a determination step of determining whether a total amount of a signal corresponding to one area exceeds a threshold when a plurality of pixels of a sensor capable of detecting a change in luminance of incident light for each pixel is divided into a plurality of areas according to the positions of the pixels, the method comprising: a step of determining whether a total amount of a signal corresponding to one area exceeds a threshold, the threshold being different for each area;

[11] An information processing device including an event identification means for identifying an event that has occurred in real space based on a detection signal that detects a change in luminance of each pixel of incident light as a signal using a prediction model trained by machine learning with an input data being a detection signal detected by a sensor capable of detecting a change in luminance of each pixel of incident light as a signal and an output data being an event that has occurred in the real space of a target to be detected by the sensor;

[12] A method executed in an information processing device, comprising an event identification step of using a detection signal detected by a sensor capable of detecting a change in luminance of each pixel of incident light as a signal as input data, and an event occurring in the real space of a target to be detected by the sensor as output data, using a machine-learned prediction model to identify an event occurring in the real space based on the detection signal detected by the sensor as a change in luminance of each pixel of incident light;

[13] An information processing device including a determination means for determining whether or not the state of the real space of the object to be detected by the sensor is normal based on the detection signal detected by the sensor capable of detecting a change in luminance of each pixel of incident light as a signal, using a machine-learned prediction model as output data that is information regarding whether or not the state of the real space of the object to be detected by the sensor is normal;

[14] The information processing device according to

[13] , further comprising: an output means for outputting information indicating that the state of the real space is abnormal when the determination means determines that the state of the real space is abnormal; or an output control means for controlling another device to output information indicating that the state of the real space is abnormal;

[15] A method comprising a determination step of determining whether or not the state of the real space of the object to be detected by the sensor is normal based on the detection signal detected by the sensor capable of detecting the change in luminance of each pixel of incident light as a signal, using a machine-learned prediction model that uses information regarding whether or not the state of the real space of the object to be detected by the sensor is normal as output data;

[16] A system comprising: a detection means for detecting a detection signal using a sensor capable of detecting a change in luminance of each pixel of incident light as a signal; and a vibration generating means for generating vibrations in at least a part of an object existing in a real space to be detected by the sensor, or in the sensor;

[17] A system comprising: a detection means for detecting a detection signal using a sensor capable of detecting a change in luminance of each pixel of incident light as a signal; and a light irradiation means for irradiating light onto at least a part of a real space to be detected by the sensor;

[18] The system according to

[16] or

[17] , further comprising: a position identifying means for identifying a position of a pixel corresponding to an object existing in real space from a position of a pixel corresponding to a detection signal detected by the detection means due to a change in luminance caused by vibration generated by the vibration generating means, or a detection signal detected by the detection means due to a change in luminance caused by light irradiation by the light irradiating means;

[19] The system according to

[18] , further comprising: a display unit that displays an object on a display screen in a manner that allows the presence of the object to be recognized at a pixel position corresponding to the object when the vibration ceases to be detected and the detection signal is no longer detected, or when the light irradiation is terminated and the detection signal is no longer detected; or a display control unit that controls the display screen of another device to display the object on the display screen in a manner that allows the presence of the object to be recognized;

[20] The system according to

[18] or

[19] , further comprising a registration means for registering the presence of an object at a pixel position corresponding to the object when the detection signal is no longer detected because the vibration has ceased or when the light irradiation has ended;

[21] The system according to any one of

[16] to

[20] , further comprising an object identification means for identifying attributes of an object present in the real space to be detected by the sensor based on the detection signal detected by the detection means, using a machine-learned prediction model that uses as input data a detection signal detected by a sensor capable of detecting a change in luminance of each pixel of the incident light as a signal and an attribute of the object as output data;

[22] A method comprising: a detection step of detecting a detection signal by a sensor capable of detecting a change in luminance of each pixel of incident light as a signal; and a vibration generation step of generating vibrations in at least a part of an object existing in a real space to be detected by the sensor, or in the sensor;

[23] A method comprising: a detection step of detecting a detection signal by a sensor capable of detecting a change in luminance of incident light for each pixel as a signal; and a light irradiation step of irradiating light onto at least a part of a real space to be detected by the sensor;

[24] An information processing device comprising: a position specifying means for specifying, when a detection signal that was detected by a sensor capable of detecting a change in luminance of each pixel of incident light as a signal, the position specifying means specifying a position of a pixel corresponding to a detection signal that was detected immediately before the detection signal became undetectable;

[25] The information processing device according to

[24] , further comprising: a display means for displaying on a display screen in a manner that enables the presence of an object at the pixel position identified by the position identification means to be recognized; or a display control means for controlling to display on a display screen of another device in a manner that enables the presence of an object at the pixel position identified by the position identification means to be recognized;

[26] The information processing device according to

[24] or

[25] , further comprising a registration means for registering the presence of an object at the position of the identified pixel;

[27] The information processing device according to

[26] , wherein, when a detection signal is detected again at a pixel position corresponding to an object registered by the registration means and then the detection signal is no longer detected again, the information processing device registers that the object registered by the registration means has moved to a pixel position corresponding to a detection signal detected immediately before the detection of the detection signal was no longer detected again;

[28] A method comprising a position specifying step of specifying, when a detection signal that was being detected by a sensor capable of detecting a change in luminance of each pixel of incident light as a signal, a position specifying step of specifying a position of a pixel corresponding to a detection signal that was detected immediately before the detection signal became undetectable; This can be solved by: Effect of the Invention

[0006] The effects of the present invention can be exemplified as follows. A first effect of the present invention is that it is possible to identify an area in which a change is occurring in the real space that is the target of sensing. A second effect of the present invention is that each area of ​​an event sensor, each area being made up of a plurality of pixels, can be excluded from the target of a predetermined process. The third effect of the present invention is that it is possible to identify an event that occurs in the real space that is the target of sensing. A fourth effect of the present invention is that it is possible to determine whether the state of the real space that is the object of sensing is normal or not. A fifth object of the present invention is to determine whether or not an object exists, or to identify the position where an object exists, even if the real space being sensed is in a state where it is difficult for a human to see. A sixth object of the present invention is to be able to specify the position where an object exists after the object has moved, or the area where the object exists, in the real space of the sensing target. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram showing a configuration of a system according to an embodiment of the present invention; [Diagram 2] 1 is a diagram showing a configuration of an information processing device according to an embodiment of the present invention; [Diagram 3] FIG. 11 is a flowchart showing an event identification process according to an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating a relationship between regions and pixels according to the embodiment of the present invention. [Diagram 5] FIG. 13 is a diagram showing a table showing the correspondence between areas and events according to the embodiment of the present invention. [Figure 6] FIG. 4 is a flowchart showing a determination process according to an embodiment of the present invention. [Figure 7] FIG. 11 is a flowchart showing an event identification process according to an embodiment of the present invention. [Figure 8] FIG. 4 is a flowchart showing a determination process according to an embodiment of the present invention. [Figure 9] 1 is a diagram showing a configuration of a system according to an embodiment of the present invention; [Figure 10] FIG. 4 is a flowchart showing an object detection process according to the embodiment of the present invention. [Figure 11] 1 is a diagram showing a configuration of a system according to an embodiment of the present invention; [Figure 12] FIG. 4 is a flowchart showing an object detection process according to the embodiment of the present invention. [Figure 13] FIG. 11 is a flowchart showing a movement detection process according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments as long as it is not contrary to the spirit of the present invention. Furthermore, the description of the effects is one aspect of the effects of the embodiments of the present invention, and is not limited to those described here.

[0009] (First embodiment) The first embodiment relates to an information processing device that, based on a signal detected by a sensor (i.e., an event sensor) capable of detecting changes in the brightness of incident light for each pixel as a signal, identifies an area in real space corresponding to the signal, or identifies an event that has occurred in real space.

[0010] Fig. 1 is a diagram showing the configuration of a system according to an embodiment of the present invention. The system 1 in Fig. 1 includes an event sensor 2 and an information processing device 3. An arbitrary object 5 may exist in a real space 4 that is the target of sensing by the event sensor 2.

[0011] The event sensor 2 is also called an EVS (event-based vision sensor). The event sensor 2 converts incident light into an electric signal in a light receiving section of the sensor, and outputs the converted electric signal as an event signal. The event sensor 2 includes a camera, and each pixel of the camera includes a light receiving section that receives incident light and a detection section that detects a change in luminance. When a change in luminance of the light incident on each pixel exceeds a set threshold, the event sensor 2 detects it as an event signal, and outputs the position coordinate of the pixel where the event signal occurred, the time when the event signal occurred, and the polarity (whether it is a light-up signal or a dark-down signal). The position coordinate of the pixel where the event signal occurred can be expressed, for example, in an xy coordinate system aligned with the up, down, left, and right directions of the event sensor 2. Also, on the display screen of the display section 15 of the information processing device 3 described later, the fact that the event signal was detected can be displayed in correspondence with the position coordinate in a manner that allows the user to know which pixel of the event sensor 2 detected the event signal.

[0012] Usually, the event sensor 2 is installed without changing its position or orientation. The camera of the event sensor 2 faces the same direction at the same position even over time, and the real space 4 that is the target of sensing is also fixed.

[0013] When an arbitrary object 5 moves in the real space 4, the movement of the object 5 causes a change in the brightness of the light incident on the event sensor 2, and an event signal is detected by the event sensor 2. When the object 5 is stationary, the event signal is not detected. The event sensor 2 can detect the event signal even in a state where it is difficult for a person to visually recognize the presence or movement of an object due to darkness such as at night or backlighting, and can detect a change in the situation such as a change in the object 5 in the real space 4.

[0014] The information processing device 3 executes various information processing operations based on the event data output from the event sensor 2. Fig. 2 is a diagram showing the configuration of an information processing device according to an embodiment of the present invention. The information processing device 3 includes a control unit 11, a RAM 12, a storage unit 13, an input unit 14, a display unit 15, and a communication interface 16, which are all connected to each other via a bus.

[0015] The control unit 11 is composed of a CPU and a ROM. The control unit 11 executes programs stored in the storage unit 13 and controls the information processing device 3. The RAM 12 is a work area for the control unit 11. The storage unit 13 is a storage medium for saving programs and data. The control unit 11 performs arithmetic processing based on the event signal received from the event sensor 2, the programs and data read from the RAM 12, and / or the data inputted via the input unit 14.

[0016] The display unit 15 has a display screen. The control unit 11 outputs a video signal for displaying an image on the display screen according to the result of the arithmetic processing. Here, the display screen of the display unit 15 may be a touch panel equipped with a touch sensor. In this case, the touch panel functions as the input unit 14. Note that the output of the result of the arithmetic processing from the information processing device 3 is not limited to display on the display screen of the display unit 15, but may also be audio output by an audio output unit (not shown) or information transmission to another device via the communication interface 16.

[0017] The communication interface 16 can be connected to a communication network wirelessly or by wire, and the information processing device 3 can transmit and receive data to and from other devices via the communication network. The other devices are not particularly limited as long as they are capable of receiving information and outputting the received information, and may, for example, be equipped with a control unit, a RAM, a storage unit, an input unit, an audio output unit, a communication interface, and the like, similar to the information processing device 3.

[0018] The real space 4 may be either indoors or outdoors, and is not particularly limited. The real space 4 can be appropriately selected depending on the purpose.

[0019] The object 5 may be any of a person, an animal, and a non-living tangible object, and is not particularly limited. The non-living tangible object may be an object having a function of moving, such as a vehicle such as an automobile, or an object not having a function of moving, such as furniture, and is not particularly limited.

[0020] Next, a description will be given of the event identification process. The event identification process is executed by the information processing device 3. Fig. 3 is a diagram showing a flowchart of the event identification process according to the embodiment of the present invention.

[0021] The information processing device 3 receives an event signal from the event sensor 2 (step S11). Based on the position of the pixel corresponding to the received event signal, a real space area is identified (step S12). The correspondence between the pixel position and the real space area is determined in advance, and multiple pixels correspond to one real space area. The real space area is identified according to this correspondence. Detection of an event signal in any of the multiple pixels corresponding to one area means that some kind of change is occurring in that area. If the event signal continues to be detected in the pixel corresponding to one area, it means that changes are continuing to occur in that area.

[0022] Next, an event that has occurred in the real space 4 is identified according to the identified real space area (step S13). Here, identifying an event is a concept that includes estimating that an event will occur. For each event, a condition is set in advance as to which area is identified in step S12 as a case where a certain event has occurred, and step S13 is executed according to this condition. FIG. 5 is a diagram showing a table showing the correspondence between areas and events according to an embodiment of the present invention. In step S13, an event can be identified by referring to the table shown in FIG. Note that the condition as to which area is identified as a case where a certain event has occurred also varies depending on the layout of the object 5 in the real space 4 and the position and orientation of the event sensor 2, and therefore needs to be appropriately set according to the layout of the object 5 and the position and orientation of the event sensor 2.

[0023] In step S13, for example, if a change occurs in area A (i.e., area A is identified), it can be identified that an event α has occurred, and if a change occurs in area B, it can be identified that an event β has occurred. Also, if a change occurs in area B within a predetermined time after a change occurs in area A, it can be identified that an event γ has occurred. That is, in addition to identifying an event according to the area in which the change occurred identified in step S12, if changes occur in multiple areas over time, it can be identified according to the order in which the change occurred, the time from when a change occurs in one area or when a change ends until a change occurs in another area, or the time until a change ends in another area. Also, it can be identified according to the length of time from when a change occurs in one area to when it ends. In this way, an event can be identified according to the identified area and the time element related to the change.

[0024] For example, assume that the real space to be sensed is one in which a patient is hospitalized and sleeping in bed at night. Unlike monitoring using a normal camera, when an event sensor is used, it is possible to detect changes in the room even when the lights are turned off at night and it is nearly dark, and since the patient's condition is not imaged in detail, it also leads to protecting the patient's privacy. For example, if there is a change in the area "bed" and there is no change in other areas after that, it can be determined that the patient sleeping in the bed has turned over in his / her sleep. Also, if there is a change in the area "bed", then there is a change in the area "floor", and there is no change in other areas after that, it can be determined that the patient sleeping in the bed has fallen onto the floor. Also, if there is a change in the area "bed", then there is a change in the area "floor", and there is still a change in the area "door", it can be determined that the patient sleeping in the bed has opened the door and moved out of the room.

[0025] Note that the area of ​​real space associated with multiple pixels does not have to coincide with an object that exists when a straight line is extended from each pixel on the visual axis in real space. For example, an area in which a change in luminance may occur when a patient moves on the bed can be set as the area "bed." In this way, an area can be set based on the expected range of movement or movement of a person or object.

[0026] Next, an output is made that the event identified in step S13 has occurred (step S14). The output in step S13 may be a display on the display unit 15 of the information processing device 3, an audio output, or transmission to another device. When the identified event is transmitted to another device, the other device can output the identified event by displaying it on a display unit or outputting it as audio. In the other embodiments below, the "output" in the information processing device 3 and the other device is similar.

[0027] In the real space 4, the change information is stored as a history indicating that an event has occurred, in association with the time at which the event signal was detected (step S15).

[0028] The processes of steps S16 to S17 may be executed in parallel with the processes of steps S13 to S15, or only one of them may be executed.

[0029] If the total amount of event signals corresponding to one real space region per time exceeds a threshold, change information indicating that a state change has occurred in the real space region identified in step S12 is output (step S16). The change information is stored in association with the time when the event signal was detected as history indicating that a change has occurred in the real space 4 (step S17). The event identification process ends with steps S11 to S15, or S11, S12, S16 and S17.

[0030] Also, different thresholds can be used for each region. For example, a high threshold can be set for the region "bed" since sleeping patients often move their bodies at night, whereas a low threshold can be set for the region "floor" since changes occur less frequently than in the region "bed."

[0031] Furthermore, even for the same region, different thresholds can be used depending on the time in real space. "Using different thresholds depending on the time" is a concept that includes not only using different thresholds depending on the time of day, but also using different thresholds depending on the time of day, day, week, month, season, etc. For example, since the amount of movement of a sleeping patient differs between late night hours and early morning hours, the threshold can be set higher during late night hours than in the morning hours.

[0032] It is also possible to use different thresholds for each region, and to use different thresholds for the same region depending on time.

[0033] The information processing device 3 according to the first embodiment can be used in medical institutions, nursing homes, and homes for monitoring and protecting hospitalized patients and the elderly, watching over people who need care, etc. Even if the real space 4 that is the subject of sensing is in a situation where visibility is difficult, such as at night, it is possible to identify actions and behaviors.

[0034] (Second embodiment) The second embodiment relates to an information processing device that, when multiple pixels of an event sensor are divided into multiple regions according to the positions of the pixels, determines whether or not the total amount of signals corresponding to one region exceeds a threshold. The configuration of the system in Fig. 1 and the configuration of the information processing device in Fig. 2 described above are also applied to the second embodiment.

[0035] The determination process will be described below. The determination process is executed by the control unit 11 of the information processing device 3. Fig. 6 is a diagram showing a flowchart of the determination process according to the embodiment of the present invention.

[0036] In the second embodiment, a plurality of regions (a plurality of groups) are set according to the positions of a plurality of pixels that receive light by the event sensor. Note that the "region" in the second embodiment is a different concept from the "region in real space" in the first embodiment, but may or may not have a corresponding relationship with the region in real space in the first embodiment.

[0037] The information processing device 3 first sets a threshold value (step S21). The threshold value differs for each region. For example, since there are regions with a lot of noise and regions with little noise for the event signal detected by the event sensor 2, it is preferable to set a threshold value for each region. As a method for setting the threshold value, a threshold value can be set for each region based on the detected event signal in a state where no change occurs in the real space. For example, an average amount of event signals per frame for each region can be obtained from the total amount of event signals detected for each region per predetermined time period for each region, and a threshold value per frame for each region can be set based on this average amount of event signals. The threshold value is preferably a value larger than the average amount of event signals, and can be, for example, a value obtained by multiplying the average amount by a coefficient larger than 1. It is preferable that the event signal for setting the threshold value is detected immediately after the event sensor 2 starts operating in the real space 4 that is the object of sensing.

[0038] When the threshold is set, the event sensor 2 starts sensing the real space 4. For example, when a change occurs such as the movement of an object 5 existing in the real space 4, an event signal is detected (step S22). Based on the detected event signal, it is determined whether or not the total amount of signals corresponding to each region exceeds the threshold (step S23). For the region that exceeds the threshold, a predetermined process is executed (step S24). The predetermined process is not particularly limited, but for example, a predetermined calculation or output process is executed using the event signal corresponding to this region or information indicating that this region has exceeded the threshold. As the predetermined process, for example, a process of displaying on the display screen of the display unit 15 in a manner that makes it clear that a state change has occurred in the region that exceeded the threshold can be exemplified. On the other hand, the predetermined process is not executed for the region that does not exceed the threshold. By executing steps S21 to S24, the determination process ends.

[0039] The determination process explained in the second embodiment can also be executed in other embodiments (for example, the first embodiment, and the third to seventh embodiments).

[0040] As described above, by setting a threshold, it is possible to exclude noise in the event signal and execute a predetermined process, but instead of excluding noise, it is also possible to exclude areas of changes that have occurred in the real space 4 that are larger than a threshold and execute a predetermined process, or to exclude areas that are smaller than the threshold and execute a predetermined process. For example, in a case where it is desired to exclude detection of an event signal based on the breathing of a person sleeping on the head and detect only an event signal based on a person turning over in bed, for an area related to a bed, a threshold can be set that allows detection of only an event signal based on the breathing of a person sleeping on the head, and detection of an event signal that does not meet the threshold can be excluded from the target of the predetermined process.

[0041] (Third embodiment) The third embodiment relates to an information processing device that uses an event signal detected by an event sensor as input data and an event occurring in the real space of a detection target as output data to identify an event occurring in the real space based on the event signal detected by the event sensor in the real space that is a sensing target, using a machine-learned prediction model. The configuration of the system in Fig. 1 and the configuration of the information processing device in Fig. 2 described above are also applied to the third embodiment.

[0042] The event identification process will be described below. The event identification process is executed by the control unit 11 of the information processing device 3. Fig. 7 is a diagram showing a flowchart of the event identification process according to the embodiment of the present invention.

[0043] First, an event that has occurred is identified based on an event signal detected at a predetermined time (step S31). The length of the predetermined time can be appropriately designed depending on the real space 4 that is the target of sensing and the event to be identified.

[0044] The occurrence of an event in step S31 is identified using a prediction model that is machine-learned based on the following input data and output data. Specifically, when an event occurs in a real space, information on the occurrence of the event is output data, and while the event is occurring, an event signal detected by sensing the real space in which the event is occurring with an event sensor can be input data.

[0045] It is preferable that the predictive model used in step S31 uses as input data an event signal detected by an event sensor installed in substantially the same position and in the same orientation as the event identification process in a real space that is substantially the same as the real space 4 that is the subject of the event identification process (i.e., including not only the same real space, but also a real space that is in a different location but has the same configuration with the same area and the same arrangement of objects).

[0046] In step S31, using this prediction model, an event that has occurred in the real space that is the sensing target can be identified based on the event signal detected by the event sensor 2.

[0047] The machine learning algorithm is not particularly limited, and a known algorithm can be used, but it is preferable to use deep learning using a multilayer neural network. The multilayer neural network has an input layer, an output layer, and multiple intermediate layers. Weights are set on the edges connecting the nodes of each layer. Weights corresponding to each input to the node are set on the edges, and the edges are multiplied by the weights corresponding to each input to the node, and the values ​​obtained by multiplying these weights and the bias are added. The value obtained by the addition is nonlinearly transformed using an activation function to calculate an activation value. The calculated activation value becomes the input value passed to the node of the next layer. The number of intermediate layers can be designed as appropriate.

[0048] In step S31, the events identified are not particularly limited as long as they are state changes in the real space 4, such as the movement of a person or animal, or the movement of another object, which may cause a change in the luminance of light incident on the event sensor 2. If the real space 4 that is the target of sensing is a hospital room in a medical institution, events such as "the patient turned over in bed," "the patient fell off the bed onto the floor," and "the patient got off the bed and moved to another room" can be identified.

[0049] When an event is specified in step S31, the information processing device 3 outputs the specified event in a form that can be understood by a person (step S32).

[0050] Next, the information processing device 3 associates the identified event with the time when the event occurred (for example, the time when detection of an event signal corresponding to the event started) or the duration of the event occurring (for example, the time when detection of an event signal corresponding to the event started and the time when detection of the event signal ended) and stores them in the storage unit 13 as an event occurrence history (step S33).

[0051] The event identification process ends after steps S31 to S33. The process of steps S31 to S33 is repeatedly executed at predetermined time intervals.

[0052] The information processing device 3 of the third embodiment, like the first embodiment, can be used in medical institutions, nursing homes, and homes for purposes such as monitoring and protecting hospitalized patients and elderly people, and watching over people who need care. Even if the real space 4 that is the subject of sensing is in a situation where visibility is difficult, such as at night, actions and behaviors can be identified.

[0053] (Fourth embodiment) The fourth embodiment relates to an information processing device that uses an event signal detected by an event sensor as input data and information regarding whether the state of the real space that is the target of sensing by the event sensor is normal or not, using a machine-learned predictive model that uses information regarding whether the state of the real space that is the target of sensing is normal or not, based on the event signal detected by an event sensor in the real space that is the target of sensing.

[0054] Here, information regarding whether or not something is normal is a concept that includes not only information that means "normal" or information that means "not normal", but also information that means "abnormal" or information that means "not abnormal".

[0055] The above-mentioned system configuration in FIG. 1 and the configuration of the information processing device in FIG. 2 are also applied to the fourth embodiment.

[0056] The determination process will be described below. The determination process is executed by the control unit 11 of the information processing device 3. Fig. 8 is a diagram showing a flowchart of the event identification process according to the embodiment of the present invention.

[0057] First, based on an event signal detected at a predetermined time, it is determined whether the state of the real space is normal or not (step S41). The length of the predetermined time can be appropriately designed depending on the real space 4 that is the object of sensing and the event to be identified.

[0058] In step S41, the determination of whether the state of the real space is normal or not is made by using a prediction model machine-learned based on the following input data and output data. Specifically, when the real space is in a normal state, an event signal detected by sensing the real space with an event sensor can be used as input data, and information indicating that the real space is normal can be used as output data. When the real space is in an abnormal state, an event signal detected by sensing the real space with an event sensor can be used as input data, and information indicating that the real space is abnormal (or not normal) can be used as output data. Furthermore, a prediction model machine-learned can be used based on input data and output data corresponding to the cases when the real space is in a normal state and when the real space is in an abnormal state.

[0059] It is preferable that the predictive model used in step S41 uses as input data an event signal detected by an event sensor installed in substantially the same position and in the same orientation as in the determination process in a real space that is substantially the same as the real space 4 that is the subject of the determination process (i.e., including not only the same real space, but also a real space that is in a different location but has the same configuration with the same area and the same arrangement of objects).

[0060] In the output data of the prediction model, the method of determining whether the real space is "normal" or "abnormal" is not particularly limited, but may be determined based on the result of a person visually judging whether the real space is normal or not. In the output data of the prediction model, the method of determining whether the real space is "abnormal" or "not abnormal" is not particularly limited, but may be determined based on the result of a person visually judging whether the real space is abnormal or not. In this way, by determining whether the output data is "normal" or "not normal" (or whether the output data is "abnormal" or "not abnormal") based on the result of a person's visual judgment, the judgment in step S41 can also be made in accordance with the judgment of the person's visual judgment. It is preferable that the determination of whether the state of the real space, which is the output data of the prediction model, is normal or not is made according to a predetermined condition, a predetermined criterion, and / or a predetermined rule.

[0061] For example, when the flow of people moving in a given facility (e.g., inside a train station) is the target of sensing by an event sensor, a state in which most people walking in the station are moving in the same direction as the flow of people moving without resisting the flow of people moving can be determined as a normal state, and a state in which some people are moving against the flow of people moving can be determined as an abnormal state. By using a prediction model that specifies output data based on such criteria, it becomes possible to determine that a state in which some people are moving against the flow of people moving in the real space that is the target of sensing is abnormal.

[0062] In addition, in the case where the flow of vehicles moving on a road is the subject of sensing by an event sensor, a state in which the vehicles traveling on the road are moving in a traveling direction determined for each road at a predetermined speed or less can be regarded as a normal state, and a state in which some of the vehicles are moving in the opposite direction to the traveling direction or moving faster than the predetermined speed can be regarded as an abnormal state. The criteria for whether something is normal or abnormal are not particularly limited, and can be designed as appropriate depending on the real space to be sensed and the type of abnormality to be detected.

[0063] In step S41, using such a prediction model, it is possible to determine whether or not the real space that is the sensing target is normal based on the event signal detected by the event sensor 2 in the real space.

[0064] The machine learning algorithm is not particularly limited and any known algorithm can be used, but it is preferable to use deep learning using a multi-layer neural network. The multi-layer neural network can have the same configuration as that of the third embodiment.

[0065] If it is determined in step S41 that the state of the real space 4 is normal (YES in step S42), the determination process ends. On the other hand, if it is determined in step S41 that the state of the real space 4 is abnormal (NO in step S42), information indicating that the state of the real space 4 is not normal or is abnormal is output (step S43).

[0066] Next, the information processing device 3 associates the time when the abnormality occurred (for example, the time when detection of an event signal corresponding to the occurrence of the abnormality started) or the duration of the abnormality (for example, the time when detection of an event signal corresponding to the occurrence of the abnormality started and the time when detection of the event signal ended) and stores the abnormality occurrence history in the storage unit 13 (step S44).

[0067] The determination process is completed after steps S41 to S44. The determination process of steps S41 to S44 is repeatedly executed at predetermined time intervals.

[0068] The information processing device 3 of the fourth embodiment can be used, for example, to determine whether the flow of people, vehicles, and other moving objects is normal on a road where many people and vehicles are moving, or in a facility where many people are moving.

[0069] (Fifth embodiment) The fifth embodiment relates to a system that detects an event signal by an event sensor and detects at least a part of an object existing in a real space to be detected by the event sensor, or an event signal that can be detected by generating a vibration in the event sensor. The configuration of the information processing device in Fig. 2 described above is also applied to the fifth embodiment. In the fifth embodiment, the configuration of the system shown in Fig. 9 can be applied instead of that in Fig. 1.

[0070] Fig. 9 is a diagram showing a configuration of a system according to an embodiment of the present invention. The system 1 in Fig. 9 includes an event sensor 2 and an information processing device 3. The system 1 also includes a vibration generating device 6 for generating vibrations in the event sensor 2 and / or an object 5. Any object 5 may be present in a real space 4 that is the target of sensing by the event sensor 2.

[0071] Usually, when an arbitrary object 5 moves in the real space 4, the luminance of the light incident on the event sensor 2 changes due to the movement of the object 5, and the event sensor 2 detects an event signal. When the object 5 is stationary, the event signal is not detected. Therefore, in a state where it is difficult for a person to visually recognize the presence or movement of an object due to darkness or backlight such as at night, if the object 5 is stationary, the presence of the object 5 cannot be detected. However, by applying a vibration to the event sensor 2 and / or the object 5 temporarily (for example, for a short period of time such as 0.1 seconds) to the extent that the state of the real space 4 does not change significantly (for example, to the extent that the position and orientation of the event sensor 2 and the object 5 do not change), using the vibration generating device 6, it becomes possible to detect an event signal even if the object 5 is stationary. Note that the length of time for which the vibration is generated and the magnitude of the vibration are not particularly limited and can be appropriately set according to the object 5 assumed to exist in the real space 4.

[0072] The real space 4 may be either indoors or outdoors, and is not particularly limited. The real space 4 can be appropriately selected depending on the purpose. The object 5 may be either a person, an animal, or a tangible object other than a living thing, and is not particularly limited. The tangible object other than a living thing may be an object that has a function of moving, such as a vehicle such as an automobile, or may be an object that does not have a function of moving, such as furniture, and is not particularly limited.

[0073] The object detection process will be described below. The object detection process is mainly executed by the control unit 11 of the information processing device 3 and the vibration generating device 6. Fig. 10 is a diagram showing a flowchart of the object detection process according to the embodiment of the present invention.

[0074] First, the information processing device 3 transmits a vibration generation request to the vibration generator 6 to generate vibration (step S51). The vibration generation request is transmitted at predetermined time intervals. When the vibration generator 6 receives the vibration generation request (step S52), the vibration generator 6 generates a slight vibration for a short period of time (step S53). The generated vibration is also transmitted to the object 5 or the event sensor 2 present in the real space 4. As the vibration is transmitted to the object 5 or the event sensor 2 in this way, the light reflected by or transmitted through the object 5 is received by the event sensor 2, and an event signal due to a change in the luminance of the incident light can be detected.

[0075] When the luminance of the light incident on the event sensor 2 changes due to the vibration generated by the vibration generating device 6, an event signal is detected (step S54). Next, the attribute (what type) of the object 5 corresponding to the detected event signal is identified (step S55). The identification of the attribute of the object 5 in step S55 can utilize a machine-learned prediction model. The prediction model can use a machine-learned prediction model with an event signal detected when light reflected and / or transmitted from an object is incident on a sensor capable of detecting the change in luminance of the incident light for each pixel as an event signal as input data and the attribute of the object as output data. Here, the attribute of the object is not particularly limited as long as it can identify, for example, the name, use, material, property, etc. of the object.

[0076] The machine learning algorithm is not particularly limited and any known algorithm can be used, but it is preferable to use deep learning using a multi-layer neural network. The multi-layer neural network can have the same configuration as that of the third embodiment.

[0077] Next, the position of the pixel corresponding to the detected event signal is identified, and the position of the pixel corresponding to the object 5 (pixel position coordinates) is identified from the identified multiple pixel positions (step S56). The method of identifying the position of the pixel corresponding to the object 5 in step S56 is not particularly limited, but since event signals are detected in multiple pixels in multiple frames, the position of the pixel corresponding to the object 5 can be identified by averaging the position coordinates of these pixels.

[0078] Next, the presence of the object 5 at the identified position is registered in the RAM 12 or the storage unit 13 (step S57). At this time, along with the presence of the object 5, the attributes of the object 5 identified in step S55 may also be registered.

[0079] Next, the position of the pixel of the identified object 5 is output (step S58). When the position of the pixel of the object 5 is displayed on the display unit 15, it can also be displayed in a manner that makes it possible to grasp that the position corresponds to the object 5 on a display screen that displays a color different from the background in the pixel in which the event signal is detected. After steps S51 to S58, the object detection process ends.

[0080] (Sixth embodiment) The sixth embodiment relates to a system that irradiates light onto at least a part of a real space that is a sensing target and detects an event signal by an event sensor. The configuration of the information processing device in FIG. 2 described above is also applied to the sixth embodiment. In the sixth embodiment, the system configuration shown in FIG. 11 can be applied instead of the system configuration in FIG. 1.

[0081] Fig. 11 is a diagram showing a configuration of a system according to an embodiment of the present invention. The system 1 in Fig. 11 includes an event sensor 2 and an information processing device 3. The system 1 also includes an illumination device 7 for illuminating at least a part of a real space 4 with light. An arbitrary object 5 may be present in the real space 4 that is the target of sensing by the event sensor 2.

[0082] Usually, when an arbitrary object 5 moves in the real space 4, the luminance of the light incident on the event sensor 2 changes due to the movement of the object 5, and the event sensor 2 detects an event signal. When the object 5 is in a stationary state, the event signal is not detected. Therefore, in a state where it is difficult for a person to visually recognize the presence or movement of an object due to darkness or backlight such as at night, if the object 5 is stationary, the presence of the object 5 cannot be detected. However, by temporarily (for example, for a short time such as 0.1 seconds) illuminating at least a part of the real space 4 with the illumination device 7, it becomes possible for the event sensor 2 to detect an event signal even if the object 5 is stationary. The length of the illumination time by the illumination device 7 and the illuminance of the light to be irradiated are not particularly limited. Furthermore, the illumination of light by the illumination device 7 may be direct illumination of the object 5, or may be indirect illumination of the object 5 by illuminating a part of the real space.

[0083] The real space 4 may be either indoors or outdoors, and is not particularly limited. The real space 4 can be appropriately selected depending on the purpose. The object 5 may be either a person, an animal, or a tangible object other than a living thing, and is not particularly limited. The tangible object other than a living thing may be an object that has a function of moving, such as a vehicle such as an automobile, or may be an object that does not have a function of moving, such as furniture, and is not particularly limited.

[0084] The object detection process will be described below. The object detection process is mainly executed by the control unit 11 and the projection device 7 of the information processing device 3. Fig. 12 is a diagram showing a flowchart of the object detection process according to the embodiment of the present invention.

[0085] First, the information processing device 3 transmits an illumination request to the illumination device 7 to illuminate at least a part of the real space 4 (step S61). The illumination request is transmitted at predetermined time intervals. When the illumination device 7 receives the illumination request (step S62), the illumination device 7 temporarily illuminates the real space 4 with light (step S63). As a result, the light reflected by or transmitted through the object 5 is received by the event sensor 2, and an event signal due to a change in the luminance of the incident light can be detected.

[0086] When a change occurs in the luminance of the light incident on the event sensor 2, an event signal is detected (step S64). Next, the attribute (what type) of the object 5 corresponding to the detected event signal is identified (step S65). The identification of the attribute of the object 5 in step S65 can utilize a machine-learned prediction model. The prediction model can use a machine-learned prediction model that uses an event signal detected when light reflected and / or transmitted from an object is incident on a sensor capable of detecting a change in luminance of the incident light for each pixel as an event signal as input data and the attribute of the object as output data. Here, the attribute of the object is not particularly limited as long as it can identify, for example, the name, use, material, properties, etc. of the object.

[0087] The machine learning algorithm is not particularly limited and any known algorithm can be used, but it is preferable to use deep learning using a multi-layer neural network. For details of the multi-layer neural network, the contents described in the third embodiment can be applied.

[0088] Next, the position of the pixel corresponding to the detected event signal is identified, and the position of the pixel corresponding to the object 5 (pixel position coordinates) is identified from the identified multiple pixel positions (step S66). The method of identifying the position of the pixel corresponding to the object 5 in step S66 is not particularly limited, but since event signals are detected in multiple pixels in multiple frames, the position of the pixel corresponding to the object 5 can be identified by averaging the position coordinates of these pixels.

[0089] Next, the presence of the object 5 at the identified position is registered in the RAM 12 or the storage unit 13 (step S67). At this time, along with the presence of the object 5, the attributes of the object 5 identified in step S65 may also be registered.

[0090] Next, the position of the pixel of the identified object 5 is output (step S68). When the position of the pixel of the object 5 is displayed on the display unit 15, it can also be displayed in a manner that makes it possible to grasp that the position corresponds to the object 5 on a display screen that displays a color different from the background in the pixel in which the event signal is detected. After steps S61 to S68, the object detection process ends.

[0091] (Seventh embodiment) The seventh embodiment relates to an information processing device that, when an event signal is detected by a sensor capable of detecting a change in luminance of each pixel of incident light as an event signal, identifies the position of a pixel corresponding to an event signal detected immediately before the event signal is no longer detected. The configuration of the system in Fig. 1 and the configuration of the information processing device in Fig. 2 described above are also applied to the third embodiment.

[0092] The movement detection process will be described below. The movement detection process is mainly executed by the control unit 11 of the information processing device 3. Fig. 13 is a diagram showing a flowchart of the movement detection process according to the embodiment of the present invention.

[0093] When an object 5 existing in the real space 4 moves or behaves due to some cause, the event sensor 2 starts detecting an event signal (step S71). Then, when the object 5 moves or behaves, the detection of the event signal ends (step S72).

[0094] Next, the position of the pixel corresponding to the event signal detected immediately before the event signal was no longer detected (the event signal detected in the frame or frames immediately before the event signal was no longer detected) is identified, and the position of the pixel corresponding to the object 5 is identified from the identified multiple pixel positions (step S73). There are no particular limitations on the method of identifying the position of the pixel corresponding to the object 5 in step S73 (position coordinates on the screen), but since event signals are detected in multiple pixels in multiple frames, the position coordinates of these pixels can be averaged to identify the position of the pixel corresponding to the object 5.

[0095] Next, the presence of the object 5 at the identified position is registered in the RAM 12 or the storage unit 13 (step S74). Next, the identified position of the object 5 is output (step S75). When the position of the object 5 is displayed on the display unit 15, it can also be displayed in a manner that makes it possible to grasp that the position corresponds to the object 5 on a display screen that displays a color different from the background in the pixel where the event signal is detected. Note that when the position etc. of the object 5 are transmitted to another device, the position etc. of the object 5 are also output in the other device.

[0096] When the movement of the object 5 is resumed, in step S75, detection of an event signal is started again at the pixel position corresponding to the registered object (step S76). In this way, if the pixel position of the event signal detected just before the end of detection of the first event signal in step S72 and the pixel position of the event signal just after detection of the second event signal is started in step S76 (the event signal detected in one frame just after detection is started or in the immediately preceding few frames), match or are within a predetermined range, the first event signal and the second event signal can be treated as originating from the same object 5.

[0097] Then, when the movement of the object 5 is completed, the detection of the second event signal ends (step S77). Next, the position of the pixel corresponding to the event signal detected just before the second event signal was no longer detected, that is, the position of the pixel corresponding to the object 5, is identified (step S78).

[0098] Next, the fact that the object 5 has moved from the position registered in step S74 to the position of the identified pixel, and the position coordinates of the pixel corresponding to the object 5 after the movement are registered in the RAM 12 or the storage unit 13 (step S79). Next, the position of the identified object 5 is output (step S80). After steps S71 to S80, the movement detection process ends.

[0099] In the seventh embodiment, when a moving object 5 stops moving and then starts moving again, if the position determined based on the event signal detected just before the detection of the first event signal ends and the position determined based on the event signal when the detection of the second event signal starts are the same or within a specified range, the first event signal and the second event signal can be treated as being caused by the same object 5.

[0100] According to the present invention, based on a signal detected by an event sensor, the area of ​​real space corresponding to the signal can be identified, making it possible to identify the area in which a change is occurring in the real space being sensed. Furthermore, based on the area of ​​real space identified as where a change is occurring, it is possible to identify an event that has occurred in the real space. When a plurality of pixels correspond to one real space region, and the total amount of signals corresponding to one real space region exceeds a threshold, change information indicating that a state change has occurred in the identified real space region can be output or the change information can be stored. In this way, it is possible to prevent outputting information indicating that a state change has occurred or to prevent storing information indicating that a state change has occurred even in cases where noise has occurred. In this case, by setting different threshold values ​​for different areas of the real space, it is possible to set a high threshold value in areas where a large amount of noise occurs and a low threshold value in areas where a small amount of noise occurs. Furthermore, in this case, by varying the threshold according to the time in real space, since the environment such as the brightness of the space varies depending on the time, the threshold can be made higher during time periods when there is a lot of noise, and lowered during time periods when there is little noise.

[0101] According to the present invention, when a plurality of pixels of an event sensor are divided into a plurality of regions according to the positions of the pixels, by determining whether or not the total amount of signals corresponding to one region exceeds a threshold, it is possible to exclude signals corresponding to at least a part of the regions from being subjected to a predetermined process according to the signals detected for each region. By specifying a threshold for each region based on signals detected in a state where no change occurs in the real space that is the target of sensing, it is possible to appropriately remove noise for each region.

[0102] According to the present invention, a machine-learned predictive model is used that uses a signal detected by an event sensor as input data and an event that occurs in the real space that is the target of sensing by the event sensor as output data, and based on a signal detected as a change in the luminance of each pixel of light incident on the event sensor in the real space that is the target of sensing, an event that occurs in the real space can be identified.

[0103] According to the present invention, a machine-learned predictive model is used that uses a signal detected by an event sensor as input data and information regarding whether the state of the real space being detected by the event sensor is normal as output data, and it is possible to determine whether the state of the real space being sensed is normal based on a signal detected as a change in the luminance of each pixel of light incident on the event sensor in the real space being sensed. In addition, if it is determined that the state of the real space being sensed is not normal, information indicating that the state of the real space is not normal can be output, or another device can be controlled to output information indicating that the state of the real space is not normal.

[0104] According to the present invention, by generating vibrations in at least a portion of an object present in the real space to be detected by an event sensor, or in the event sensor and detecting a signal with the event sensor, it is possible to determine whether or not an object exists, or to identify the location where the object exists, even if the real space being sensed is in a state where it is difficult for a person to see the object. In addition, by irradiating light onto at least a portion of the real space to be detected by the event sensor and detecting a signal with the event sensor, it is possible to determine whether or not an object exists, or to identify the location where the object exists, even if the real space being sensed is in a state where it is difficult for a person to see the object. In this case, when a signal is no longer detected because the vibration subsides, or when a signal is no longer detected because the light irradiation has stopped, the signal can be displayed on the display screen in a manner that makes it possible to grasp the presence of an object at the pixel position where the signal was previously detected, or the signal can be displayed on the display screen of another device in a manner that makes it possible to grasp the presence of an object. In addition, when a signal is no longer detected because the vibration subsides, or when a signal is no longer detected because the light irradiation has ended, it is possible to register the presence of an object at the pixel position where a signal had been detected until then. Furthermore, when light reflected and / or transmitted from an object is incident on an event sensor, the detected signal is used as input data, and the attributes of the object are used as output data, allowing the attributes of the object in the real space being sensed to be identified using a machine-learned predictive model.

[0105] According to the present invention, when a signal is detected by an event sensor and then is no longer detected, by identifying the position of the pixel corresponding to the signal detected just before the signal was no longer detected, it is possible to identify the position or area in which the object is located after the object has stopped moving in the real space of the sensing target. In this case, the signal can be displayed on a display screen in a manner that makes it possible to recognize the presence of an object at the position of the pixel where the signal is detected, or the signal can be displayed on a display screen of another device in a manner that makes it possible to recognize the presence of an object. Furthermore, it is possible to register the presence of an object at the position of the pixel where the signal is detected. Furthermore, if a signal is detected again at the position of a pixel corresponding to a registered object or at a pixel surrounding that pixel, and then the signal is no longer detected again, it is possible to detect that the object registered by the registration means has moved to the position of the pixel corresponding to the signal detected just before the signal was no longer detected again. [Explanation of symbols]

[0106] 1 System 2 Event Sensor 3. Information processing equipment 4. Real Space 5 objects 6. Vibration Generator 7 Irradiation device 11 Control section 12 RAM 13 Storage Department 14 Input section 15 Display section 16 Communication Interface

Claims

1. An event sensor capable of detecting a change in luminance of each pixel of incident light as a signal, the event sensor being installed without changing its position and orientation, and a region specifying means for specifying a region in real space corresponding to the detection signal in accordance with the correspondence between the pixel position and the region in real space, based on the pixel position of the detection signal detected by the event sensor. Equipped with A plurality of pixels correspond to one real space region, an information output means for outputting change information indicating that a state change has occurred in the real-space area specified by the area specifying means when the total amount of detection signals corresponding to one real-space area exceeds a threshold value, or an information storage means for storing the change information; Equipped with An information processing device in which the threshold varies for each region of real space.

2. An event sensor capable of detecting a change in luminance of incident light for each pixel as a signal, the event sensor being installed without changing its position or orientation, and comprising area identification means for identifying an area in real space corresponding to said detection signal based on the pixel position of the detection signal detected by the event sensor in accordance with the correspondence between the pixel position and the area in real space. Equipped with A plurality of pixels correspond to one real space region, an information output means for outputting change information indicating that a state change has occurred in the real-space area specified by the area specifying means when the total amount of detection signals corresponding to one real-space area exceeds a threshold value, or an information storage means for storing the change information; Equipped with An information processing device in which a threshold varies depending on time in real space.

3. A method executed in an information processing device, comprising: an area specifying step of specifying an area in real space corresponding to a detection signal based on a pixel position of the detection signal detected by an event sensor capable of detecting a change in luminance of each pixel of incident light as a signal, the event sensor being installed without changing its position and orientation, in accordance with a correspondence relationship between the pixel position and the area in real space; having A plurality of pixels correspond to one real space region, an information output step of outputting change information indicating that a state change has occurred in the real-space area specified by the area specifying means when the total amount of detection signals corresponding to one real-space area exceeds a threshold value, or an information storage step of storing the change information; having A method in which the threshold is different for each region of real space.

4. A method executed in an information processing device, comprising: an area specifying step of specifying an area in real space corresponding to a detection signal based on a pixel position of the detection signal detected by an event sensor capable of detecting a change in luminance of each pixel of incident light as a signal, the event sensor being installed without changing its position and orientation, in accordance with a correspondence relationship between the pixel position and the area in real space; having A plurality of pixels correspond to one real space region, an information output step of outputting change information indicating that a state change has occurred in the real-space area specified by the area specifying means when the total amount of detection signals corresponding to one real-space area exceeds a threshold value, or an information storage step of storing the change information; having A method in which the threshold varies depending on time in real space.

5. A determination means for determining whether or not a total amount of a signal corresponding to one area exceeds a threshold when a plurality of pixels of an event sensor capable of detecting a change in luminance of each pixel of incident light as a signal is divided into a plurality of areas according to the positions of the pixels. Equipped with An information processing device in which the threshold varies depending on the region.

6. A threshold value specifying means for specifying a threshold value for each region based on a detection signal detected by the event sensor in a state where no change occurs in the real space of the object to be detected by the event sensor. Equipped with The information processing apparatus according to claim 5 , wherein the determining means makes the determination based on the threshold value specified by the threshold value specifying means.

7. A method executed in an information processing device, comprising: A determination step for determining whether or not a total amount of signals corresponding to one area exceeds a threshold value when a plurality of pixels of an event sensor capable of detecting a change in luminance of each pixel of incident light as a signal is divided into a plurality of areas according to the positions of the pixels. having A method in which the threshold varies from region to region.

Images