Detection system, detection method, program, and detection module

Abstract

To acquire a virtual volume from a distance image obtained by imaging, and use this virtual volume to detect a target object and detect conditions such as abnormalities.SOLUTION: A detection system includes: an imaging unit (detection module 11, imaging unit 12) configured to acquire in advance a first distance image (Vv1) representing a background with respect to a target object (4) to be detected and acquire a second distance image (Vv2) including at least the background and the target object; and a processing unit (13, processing device 16) configured to calculate a first virtual volume representing the background from the first distance image, calculate a second virtual volume representing the target object from the second distance image, and compare the first virtual volume with the second virtual volume to detect the target object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a detection technology used to detect the state of an object to be detected, for example, a human. [Background technology]

[0002] A ToF camera (Time-of-Flight Camera) is a camera that can measure three-dimensional information (distance image) from an object by shining light on it and using the arrival time of the reflected light.

[0003] Regarding this ToF detection technology, it is known that a difference image is obtained from a captured image using background subtraction processing, the head is estimated from a human object contained in this difference image, the distance between the head and the floor of the target space is calculated to determine the person's posture, and the person's behavior is detected from the posture and object position information (for example, Patent Document 1).

[0004] Regarding detection of an abnormality in an object, it is known to measure the time that the object remains stationary, and if this time exceeds a threshold, to determine that an abnormality has occurred (for example, Patent Document 2).

[0005] Regarding abnormality monitoring, it is known to monitor the change over time in the distance of a measurement point in any one area in a distance image, and recognize an abnormality if this change over time exceeds a certain range (for example, Patent Document 3).

[0006] Regarding detection of a moving object, it is known to calculate a movement vector and a volume of an object in a detection space from a distance image, and to detect the detection target based on the movement vector and the volume (for example, Patent Document 4). [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2015-130014 A [Patent Document 2] JP 2008-052631 A [Patent Document 3] JP 2019-124659 A [Patent Document 4] JP 2022-051172 A Summary of the Invention [Problem to be solved by the invention]

[0008] For example, when the object to be detected for behavior or other status is a human, there are issues such as attribute information such as portraits, information regarding privacy, etc., which should be given equal or higher priority than obtaining status information such as anomaly detection. Even if anomaly detection is possible with images taken with a general camera, it is not possible to protect personal information such as privacy.

[0009] In contrast, distance images obtained by ToF cameras have the advantage of preventing the disclosure of privacy and personal information, even if the subject is a human.

[0010] The inventors of the present disclosure have discovered that a virtual volume can be obtained from pixels of a distance image that represents the distance between an object and a sensor, and the state of the object can be detected from changes in that volume.

[0011] Therefore, in consideration of the above problems and findings, the object of the present disclosure is to obtain a virtual volume from a distance image obtained by imaging, and to use this virtual volume to detect objects and conditions such as abnormalities. [Means for solving the problem]

[0012] In order to achieve the above-mentioned object, according to one aspect of the detection system of the present disclosure, the system includes an imaging unit that acquires a first distance image representing a background in advance and acquires a second distance image including at least the background and the object, and a processing unit that calculates a first virtual volume representing the background from the first distance image, calculates a second virtual volume representing the object from the second distance image, and detects the object by comparing the first virtual volume with the second virtual volume.

[0013] In order to achieve the above-mentioned object, according to one aspect of the detection system of the present disclosure, the system includes an imaging unit that acquires in advance a first distance image representing the background along with an object other than the object to be detected, and acquires a second distance image including the background, the object, and the object, and a processing unit that calculates a first virtual volume representing the object and the background from the first distance image, calculates a second virtual volume representing the object and the object from the second distance image, and detects the object by comparing the first virtual volume with the second virtual volume.

[0014] In this detection system, the processing unit may calculate a distance and / or a virtual area of ​​the object from an imaging unit, and detect a state of the object using the distance and / or the virtual area from the imaging unit.

[0015] This detection system may include an information presentation unit that presents any one or more of the first distance image, the second distance image, the first virtual volume image, the second virtual volume image, the maximum height, the virtual area, and status information representing the status of the object.

[0016] In order to achieve the above-mentioned object, according to one aspect of the detection method disclosed herein, the method includes a step in which an imaging unit acquires in advance a first distance image representing a background for an object to be detected and acquires a second distance image including at least the background and the object, and a step in which a processing unit calculates a first virtual volume representing the background from the first distance image and calculates a second virtual volume representing the object from the second distance image, and detects the object by comparing the first virtual volume with the second virtual volume.

[0017] In order to achieve the above-mentioned object, according to one aspect of the program of the present disclosure, a program for being executed by a computer causes the computer to execute functions of: acquiring in advance a first distance image representing a background for an object to be detected; acquiring a second distance image including at least the background and the object; calculating a first virtual volume representing the background from the first distance image; calculating a second virtual volume representing the object from the second distance image; detecting the object by comparing the first virtual volume with the second virtual volume; and calculating a maximum height or virtual area from the object.

[0018] In order to achieve the above-mentioned object, according to one aspect of the detection module of the present disclosure, the detection module includes an imaging unit that acquires in advance a first distance image representing a background for an object to be detected and acquires a second distance image including at least the background and the object, and a processing unit that calculates a first virtual volume representing the background from the first distance image, calculates a second virtual volume representing the object from the second distance image, and detects the object by comparing the first virtual volume with the second virtual volume. Effect of the Invention

[0019] According to the present disclosure, any of the following effects can be obtained. (1) A virtual volume representing the object is obtained using the pixels contained in the distance image, and the object can be detected using changes in this virtual volume, making it possible to detect abnormalities and other conditions in the object with high accuracy and quickly.

[0020] (2) Since the object is identified from the distance image, if the object is, for example, a human, it is possible to omit information other than attribute information such as gender and the object's state, thereby reducing the amount of information used in the detection process, thereby reducing the load on information processing and speeding up processing.

[0021] (3) After identifying the object, it is possible to detect the condition of the object by comparing frames of range images to indicate whether the object is normal or abnormal. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a diagram showing a detection system according to a first embodiment. [Diagram 2] 2A is a diagram showing a real image representing a state X, and FIG. 2B is a diagram showing an example of a composite image representing the state X. [Diagram 3] 3A is a diagram showing a real image representing state Y, and FIG. 3B is a diagram showing an example of a composite image representing state Y. FIG. [Figure 4] FIG. 4 is a flowchart illustrating a processing procedure of the detection system according to the first embodiment. [Diagram 5] FIG. 5 is a diagram illustrating a detection system according to the second embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the detection information database. [Figure 7] FIG. 7 is a flowchart illustrating a processing procedure of the detection system according to the second embodiment. [Figure 8] FIG. 8 is a diagram showing examples of real images in states A, B, and C. [Figure 9] FIG. 9 is a diagram showing a state detection table relating to state A, state B, and state C. As shown in FIG. [Figure 10] A in Figure 10 is a diagram showing an example of a background distance image GdA relating to the third embodiment, B in Figure 10 is a diagram showing an example of a background-object distance image GdB relating to the third embodiment, and C in Figure 10 is a diagram showing an example of a background-object-target distance image GdC relating to the third embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of the detection information database according to the third embodiment. [Figure 12] FIG. 12 is a flowchart showing a processing procedure of the detection system according to the third embodiment. [Figure 13]FIG. 13A is a flowchart showing the procedure for acquiring background difference information, FIG. 13B is a flowchart showing the procedure for acquiring virtual volume difference information, and FIG. 13C is a flowchart showing the processing procedure for detecting the presence of an object. [Figure 14] FIG. 14A is a flowchart showing a processing procedure for detecting a state of an object, and FIG. 14B is a flowchart showing a processing procedure for detecting an abnormality in an object. [Figure 15] FIG. 15 is a diagram illustrating a detection module according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] First Embodiment Fig. 1 shows a detection system 2 and a detection target according to a first embodiment. The configuration shown in Fig. 1 is an example, and the present disclosure is not limited to such a configuration.

[0024] The detection system 2 acquires a background distance image Gd1 (hereinafter simply referred to as "background image Gd1") and a composite distance image Gd2 (hereinafter simply referred to as "composite image Gd2"), and detects the state of the object 4 using these Gd1 and Gd2. That is, the detection system 2 detects a change in state of the object 4 to be detected from a plurality of frames representing the image. The detection of the object 4 includes recognition of the change in state of the object 4 or grasping of the change in state of the object 4, and either of these may be used for the state detection. In the detection system 2, the background image Gd1 is an example of the first distance image of the present disclosure, and the composite image Gd2 is an example of the second distance image of the present disclosure. In the present disclosure, the acquisition of the first distance image may be a process of acquiring a first distance image representing the background 6 in advance, reacquiring the first distance image a certain time after the acquisition, and updating the previous first distance image with the reacquired first distance image.

[0025] The background image Gd1 is an image that represents the distance between the object 4 and the background 6. The object 4 is an object whose state changes, such as a human or a robot. If the object 4 whose state is to be detected is, for example, a human, the behavior of the object 4, such as the head 4a, torso 4b, or limbs 4c including the hands and feet, appears in the composite image Gd2. The background 6 is the location where the object 4 exists, such as a bathroom or a room, or the surface of the water; in other words, it is the area where the object 4 stays and its state is detected.

[0026] The composite image Gd2 is an image that includes a background 6 and an object 8 other than the target 4, and indicates the distance between them. The object 8 is assumed to be a moving or stationary object other than the target 4 that exists in the status detection area.

[0027] <Detection System 2> As shown in FIG. 1, the detection system 2 includes a detection module 11 and a processing unit 13. The detection module 11 is an example of an imaging section of the present disclosure. The detection module 11 irradiates the object 4 with intermittently emitted light Li, receives reflected light Lf from the object 4 that has received the light Li, and generates a distance image Gd in time series. In this way, the detection module 11 acquires the distance image Gd, which indicates the distance between the object 4 and the imaging section 12 (FIG. 5), in frame units in time series.

[0028] The processing unit 13 is an example of a processing unit of the present disclosure. In this embodiment, the processing unit 13 is, for example, a personal computer that executes an OS (Operating System) and a detection program of the present disclosure, executes information processing required for detecting the state of the target 4, and detects the state of the target 4.

[0029] <Detection of object 4's status> In this detection system 2, the state of the object 4 is detected using the distance image Gd, and therefore, unlike a normal optical camera, the object 4 cannot be visually recognized from the distance image Gd. Therefore, the object 4 is displayed as a real image with reference to state X (Fig. 2) and state Y (Fig. 3), and the relationship with the distance image Gd is clearly shown.

[0030] 2A shows a subject 4, a background 6 and an object 8 in state X. In this case, the subject 4 represents a human being in a standing position.

[0031] 2B shows a composite image Gd2X acquired by the detection module 11 from above the object 4 in state X. Since the composite image Gd2X in frame 15-1 contains the object 4, the background 6, and the object 8, the distance image GdX of the object 4 can be extracted by removing the background image Gd1X containing the background 6 and the object 8 from the composite image Gd2X.

[0032] Therefore, the maximum height of the object 4 is assumed to be HmaxX. This maximum height HmaxX is an example of the distance in the present disclosure, and in this embodiment is distance information representing the distance between the object 4 and the imaging unit 12. In other words, since the maximum height HmaxX of the object 4 represents the minimum distance between the object 4 and the imaging unit 12, the distance between the object 4 and the imaging unit 12 or the maximum height HmaxX of the object 4 can be represented using this distance information.

[0033] The virtual volume of the object 4 is denoted by VvX. The virtual volume VvX represents the virtual volume of the object 4 in the present disclosure. This virtual volume VvX can be expressed by Equation 1 using a distance image GdX and a maximum height HmaxX.

[0034] VvX=GdX·HmaxX (Formula 1)

[0035] In this state X, even if the object 8 moves as indicated by the dashed line, a distance image GdX of the object 4 can be extracted by removing the background image Gd1X, which includes the background 6 and the object 8, from the composite image Gd2X. The object 4 can be detected from this distance image GdX. This distance image GdX represents the virtual area VsX of the object 4.

[0036] The virtual volume Vvx may be calculated using the sum of the heights, and can be expressed by Equation 2. Vvx=ΣGdx (Formula 2)

[0037] In Equation 2, ΣGdx represents the sum of the height information of the object 4.

[0038] 3A shows a subject 4, a background 6 and an object 8 that have changed from state X to state Y. In this case, the subject 4 shows a human being in a supine position.

[0039] 3B shows a composite image Gd2Y acquired by the detection module 11 from above the object 4 in state Y. Since the composite image Gd2Y in frame 15-2 includes the object 4, a distance image GdY of the object 4 in state Y can be similarly extracted by removing the background image Gd1Y from the composite image Gd2Y. This distance image GdY represents the virtual area VsY of the object 4.

[0040] Therefore, if the maximum height of the object 4 is HmaxY and the virtual volume of the object 4 is VvY, then this virtual volume Vvy can be expressed by the following equation 3.

[0041] VvY = VsY HmaxY (Equation 3)

[0042] In this way, when the object 4 transitions from state X to state Y, the maximum height of the object 4 changes from HmaxX to HmaxY, and its virtual area changes from VsX to VsY. Therefore, by comparing the maximum heights HmaxX and HmaxY and the virtual areas VsX and VsY, it can be recognized that the object 4 has changed from state X to state Y.

[0043] <Processing procedure for detecting the state of object 4> This process shows a process for detecting a state using a distance image Gd acquired by imaging with the detection module 11.

[0044] 4 shows an example of a processing procedure for detecting the state of the object 4. This processing procedure includes imaging (S101), calculation of virtual volumes VvX1, VvX2, VvY1, and VvY2 (S102), calculation of a virtual volume difference ΔVv (S103), detection of the object 4 (S104), calculation of a maximum height HmaxX and a maximum height HmaxY of the object 4 (S105), calculation of virtual areas VsX and VsY of the object 4 (S106), and detection of the state of the object 4 (S107).

[0045] The detection module 11 captures an image (S101), and by this capture, a background image Gd1 and a compound image Gd2 in states X and Y are obtained.

[0046] The processing unit 13 uses the composite image Gd2X in state X to calculate virtual volumes VvX1, VvX2 including the object 4 in state X through information processing by the processor 26 (FIG. 5) (S102). The virtual volumes VvX1, VvX2 are virtual volumes between different frames. A volume difference ΔVv between these virtual volumes VvX1, VvX2 is calculated (S103) to detect the presence of the object 4 in the background 6. Even if the object 8 on the background 6 moves, the object 4 of a specific volume can be detected without being affected by this movement, and the presence of the object 4 can be known.

[0047] The processing unit 13 calculates the maximum height HmaxX and maximum height HmaxY of the detected object 4 from, for example, the distance image GdX in state X and the distance image GdY in state Y (S105), and calculates the virtual areas VsX and VsY of the object 4 (S106).

[0048] Then, the processing unit 13 detects the state of the object 4 by comparing the maximum heights HmaxX and HmaxY and comparing the virtual areas VsX and VsY (S107).

[0049] <Advantages of the First Embodiment> According to the first embodiment, any one of the following effects can be obtained. (1) A virtual volume Vv representing the object 4 can be obtained using the pixels gi contained in the distance image Gd, and the object 4 can be detected using changes in this virtual volume Vv, making it possible to detect conditions such as abnormalities in the object 4 with high accuracy and quickly.

[0050] (2) The object 4 can be identified from the distance image Gd. If the object 4 is, for example, a human, attribute information such as gender and other information other than the state of the object can be omitted. This reduces the amount of information used in the detection process, reduces the load on information processing, and speeds up processing.

[0051] (3) After identifying the object 4, it is possible to accurately detect whether the object is in an abnormal or normal state by comparing frames of the distance image Gd.

[0052] Second Embodiment Fig. 5 shows a detection system 2 and a detection target according to a second embodiment. The configuration shown in Fig. 5 is an example, and the present disclosure is not limited to such a configuration. In Fig. 5, the same parts as in Fig. 1 are denoted by the same reference numerals.

[0053] This detection system 2 includes a light emitting unit 10, an imaging unit 12, a control unit 14, a processing device 16, etc. The light emitting unit 10 receives a drive output from a light emitting drive unit 18 under the control of the control unit 14 to emit light intermittently, and irradiates the light Li onto the object 4. Reflected light Lf is obtained from the object 4 that has received this light Li. The time from the point in time when the light Li is emitted to the point in time when the reflected light Lf is received represents the distance.

[0054] The imaging unit 12 is an example of an imaging unit of the present disclosure. The imaging unit 12 includes a light receiving unit 20 and a distance image generating unit 22. The light receiving unit 20 receives reflected light Lf from the object 4 in time series in synchronization with the light emission of the light emitting unit 10 under the control of the control unit 14, and outputs a light receiving signal. The distance image generating unit 22 receives the light receiving signal from the light receiving unit 20 and generates a distance image Gd in time series. Thus, the distance image Gd representing the distance between the object 4 and the imaging unit 12 is obtained in frame units in time series.

[0055] The control unit 14 is, for example, a computer, and executes an imaging program to control the light emission of the light emitting unit 10 and the imaging of the imaging unit 12. The light emitting unit 10, the imaging unit 12, the control unit 14, and the light emission driving unit 18 are an example of the detection module 11 of the present disclosure, and can be configured, for example, as a discrete element in one package, such as a one-chip IC. This detection module 11, for example, constitutes the above-mentioned ToF camera.

[0056] The processing device 16 is an example of the processing unit 13 of the present disclosure. In this embodiment, the processing device 16 is, for example, a personal computer equipped with a communication function, and includes a processor 26, a storage unit 28, an input / output unit (I / O) 30, an information presentation unit 32, a communication unit 34, and the like.

[0057] The processor 26 executes the OS and the detection program of the present disclosure stored in the storage unit 28 , and executes information processing required for detecting the state of the target object 4 .

[0058] The memory unit 28 stores an OS, a detection program, and detection information databases (DB) 36-1 (FIG. 6), 36-2 (FIG. 11) used for information processing required for status detection. The memory unit 28 includes memory elements such as a ROM (Read-Only Memory) and a RAM (Random-Access Memory). The input / output unit 30 inputs and outputs information under the control of the processor 26.

[0059] An operation input unit (not shown) is connected to the input / output unit 30 in addition to the information presentation unit 32. The input / output unit 30 receives operation input information by user operation or the like, and obtains output information based on information processing by the processor 26.

[0060] The information presenting unit 32 is an example of an information presenting unit in the present disclosure, and is configured, for example, by an LCD (Liquid Crystal Display) or the like. Under the control of the processor 26, the information presenting unit 32 presents presentation information including any one or more of the distance image Gd, the virtual volume Vv, the maximum height Hmax, the virtual area Vs, and the status information Sx representing the status of the object 4. The operation input unit may be, for example, a touch panel provided on the LCD screen of the information presenting unit 32.

[0061] The communication unit 34 is controlled by the processor 26 to be connected to information devices such as a communication terminal (not shown) via a public line or the like via wired or wireless connection, and can present status information of the object 4, etc. to the communication terminal.

[0062] <Control by control unit 14> The control by the control unit 14 includes processes such as a) emission control of light Li, b) reception control of reflected light Lf, c) generation processing of distance image Gd, and d) transmission control of distance image Gd.

[0063] a) Light emission control of light Li The control unit 14 controls the light emission of the light-emitting unit 10 to generate reflected light Lf from the object 4. To cause the light-emitting unit 10 to emit light intermittently, a drive signal is provided to the light-emitting unit 10 from the light-emitting drive unit 18 under the control of the control unit 14. This causes the light-emitting unit 10 to emit intermittent light Li, which is irradiated onto the object 4.

[0064] b) Light reception control of reflected light Lf In order to receive the reflected light Lf from the object 4 that has received the light Li, the control unit 14 controls the light receiving unit 20. As a result, the reflected light Lf from the object 4 is received by the light receiving unit 20. In response to this reception, a light receiving signal is generated from the light receiving unit 20 and provided to the distance image generating unit 22.

[0065] c) Generation process of distance image Gd Distance image generator 22 generates a distance image Gd using the light reception signals under the control of controller 14. This distance image Gd is composed of pixels gi having different light reception distances depending on the unevenness of object 4 and the distance.

[0066] d) Transmission control of distance image Gd The control unit 14 receives the distance image Gd from the distance image generating unit 22, and sends this distance image Gd to the processing device 16 on a frame-by-frame basis.

[0067] <Information processing by the processing device 16> The information processing of the processing device 16 includes processes such as e) acquisition of distance image Gd, f) calculation of virtual volume Vv, g) detection of object 4, h) calculation of maximum height Hmax and virtual area Vs, i) detection of the status of object 4, j) detection of abnormalities in object 4, k) calculation of distance image Gd, virtual volume Vv, maximum height Hmax and virtual area Vs, presentation of status information Sx, and l) generation and updating of DB36-1.

[0068] e) Acquisition of distance image Gd The processing device 16 acquires a distance image Gd in time series under the control of the processor 26. The distance image Gd is acquired on a frame-by-frame basis. This distance image Gd includes a background image Gd1 and a composite image Gd2. The background image Gd1 and the composite image Gd2 have already been described, so a detailed description thereof will be omitted.

[0069] f) Calculation of virtual volume Vv The processing device 16 calculates a first virtual volume Vv1 representing the background 6 from the background image Gd1, and calculates a second virtual volume Vv2 representing the object 4 from the composite image Gd2.

[0070] If pixel gi contained in the background image Gd1 is g1, pixel gi contained in the composite image Gd2 is g2, and the conversion coefficient for converting pixel gi to volume is η, the first virtual volume Vv1 and the second virtual volume Vv2 can be expressed by Equation 4 and Equation 5.

[0071] Vv1=η g1 (Eq. 4) Vv2=η·g2 (Formula 5)

[0072] g) Detection of Object 4 The processing device 16 detects the object 4 by comparing the first virtual volume Vv1 with the second virtual volume Vv2. That is, if the virtual volume of the object 4 is Vvx, it can be expressed by Equation 6. Vvx = Vv2 - Vv1 = η (g2 - g1) =η Δg (Eq. 6) In Equation 3, Δg is the number of pixels representing the object 4 (g2−g1).

[0073] h) Calculation of maximum height Hmax and virtual area Vs The processing device 16 can express the virtual area Vs of the object 4 by the following equation 7, where Hmax is the maximum height of the object 4 obtained from the distance image Gd, and Vs is the virtual area. Vs=Vvx÷Hmax=η·Δg÷Hmax (Formula 7)

[0074] Furthermore, the processing device 16 can determine the maximum height Hmax of the object 4 from the background 6 in which the object 4 exists, using the pixels gi.

[0075] i) Detection of the state of object 4 The processing device 16 detects a change in state of the object 4 from the maximum height Hmax or the virtual area Vs. The processing device 16 sets a threshold value Hth for the maximum height Hmax and a threshold value Vsth for the virtual area Vs, and detects whether the height H is greater than or equal to the threshold value Hth and whether the virtual area Vs is greater than or equal to the threshold value Vsth.

[0076] j) Abnormality detection of object 4 The processing device 16 detects an abnormality when the change in the object 4 obtained by comparing the distance images between two or more frames is less than a threshold value.

[0077] k) Presentation of distance image Vd, virtual volume Vv, virtual area Vs, and status information Sx Under the control of the processor 26, the processing device 16 presents the background image Gd1, the composite image Gd2, the virtual volumes Vv1 and Vv2, the maximum height Hmax, the calculated virtual area Vs, and the status information Sx on the information presentation unit 32. According to this presented information, the presence and status of the object 4, and judgment information indicating whether the status is normal or abnormal, can be visually confirmed.

[0078] For this information presentation, the processing device 16 can wirelessly connect the communication unit 34 to the corresponding communication terminal under the control of the processor 26, and present the same information as the information presentation unit 32 to the corresponding communication terminal.

[0079] l) DB36-1 information processing and data update The processing device 16 generates and updates the DB 36-1 stored in the storage unit 28 under the control of the processor 26.

[0080] <db36-1> This DB36-1 is an example of a database in the present disclosure. Control information and detection information for detecting the state of the object 4 are stored in this DB36-1.

[0081] 6 shows an example of the DB 36-1. The DB 36-1 includes a distance image section 38, a virtual volume section 40, a virtual area section 42, a maximum height section 44, an object section 46, a presentation information section 48, and a history information section 50.

[0082] A background image section 38-1 and a compound image section 38-2 are set in the distance image section 38. A background image Gd1, which is a distance image of the background image, is stored in the background image section 38-1. A compound image Gd2, which is a distance image of the compound image, is stored in the compound image section 38-2.

[0083] A first virtual volume portion 40-1 and a second virtual volume portion 40-2 are set in the virtual volume portion 40. A first virtual volume Vv1 is stored in the first virtual volume portion 40-1. A second virtual volume Vv2 is stored in the second virtual volume portion 40-2.

[0084] The virtual area section 42 has an area section 42-1 and a threshold section 42-2. The area section 42-1 stores area data representing the virtual area Vs. The threshold section 42-2 stores data representing a threshold Vsth of the virtual area Vs.

[0085] The maximum height section 44 has a height section 44-1 and a threshold section 44-2. The height section 44-1 stores length data representing the maximum height Hmax. The threshold section 44-2 stores data representing a threshold Hth for the maximum height Hmax.

[0086] The object section 46 has a detection information section 46-1 and a status detection section 46-2. The detection information section 46-1 stores detection information of the object 4. The status detection section 46-2 stores status information indicating whether the object 4 is normal or abnormal, which is obtained from the detection information.

[0087] The presentation information section 48 stores presentation information such as the distance image Gd, the virtual volume Vv, the maximum height Hmax, the virtual area Vs, detection information, and status information.

[0088] The history information section 50 stores history information that indicates the history of information detection, presentation information, and the like.

[0089] Although not shown in the figure, a date and time information section may be set in DB 36-1 to store date and time information indicating the date and time when the state of the object 4 was detected.

[0090] <Processing procedure for detecting the state of object 4> This processing procedure shows a processing procedure for detecting a state using a distance image Gd acquired by imaging with the detection module 11.

[0091] 7 shows an example of a processing procedure for detecting the state of the object 4. This processing procedure includes capturing an image of a background image Gd1 (S201), calculating a first virtual volume Vv1 (S202), capturing an image of a composite image Gd21 (S203), calculating a second virtual volume Vv2 (S204), detecting the object 4 (S205), calculating a virtual area Vs and a maximum height Hmax (S206), comparing with another frame, calculating ΔVs and ΔHmax (S207), comparing ΔHmax with ΔHth (S208), comparing ΔVs with ΔVsHth (S209), judging the virtual area Vs against a threshold value Vsth (S207), detecting a normal state of the object 4 (S210), detecting an abnormality of the object 4 (S211), presenting information (S212, S213), and the like.

[0092] The imaging unit 12 captures a background image Gd1 under the control of the control unit 14 (S201). The processing device 16 calculates a first virtual volume Vv1 (S202). The imaging unit 12 captures a compound image Gd2 under the control of the control unit 14 (S203). The processing device 16 calculates a second virtual volume Vv2 (S204). The processing device 16 acquires the background image Gd1 and the compound image Gd2 from the imaging unit 12 and stores them in DB36-1.

[0093] The processing device 16, through information processing by the processor 26, determines whether the object 4 has been detected from the second virtual volume Vv2 using the first virtual volume Vv1 and the second virtual volume Vv2 (S205).

[0094] The processing device 16 calculates the virtual area Vs and the maximum height Hmax through information processing by the processor 26 (S206), and stores the calculation results in the DB 36-1.

[0095] The processing device 16, through information processing by the processor 26, compares the maximum height Hmax with the threshold value Hth and determines which is larger (S208).

[0096] If Hmax≦Hth is satisfied (YES in S208), the processing device 16 can determine that the object 4 has been detected normally by information processing of the processor 26, and compares the virtual area Vs with a threshold value Vsth to determine which is larger (S209). If Vs≧Vsth (YES in S207), the processing device 16 determines that the object 4 has been detected normally (S210).

[0097] In S208, if Hmax≦Hth is not satisfied (NO in S206), it is determined that an abnormality has been detected in the object 4 (S211). In addition, in S209, if Vs≧Vsth is not satisfied (NO in S209), it is similarly determined that an abnormality has been detected in the object 4 (S211).

[0098] The processing device 16 executes information presentation (S212, S213) under the control of the processor 26, and in the information presentation related to S212, information such as the detection information, the distance image Gd, and normality information indicating that the object 4 is normal is presented. In addition, in the information presentation related to S213, information such as the detection information, the distance image Gd, and abnormality information indicating that the object 4 is abnormal is presented.

[0099] If there is an abnormality in the object 4, this process is terminated, and if there is a normality, the process returns from S212 to S203 and the state detection continues.

[0100] <Detection of object 4's status> The object 4 to be detected for status is, for example, a human, and the distance image Gd obtained from the detection module 11 is a collection of pixels gi that represent the distance between the light receiving unit 20 and the object 4. Therefore, in order to simulate the detection of the status of the object 4, a real image of the object 4 is illustrated as an example.

[0101] Fig. 8 shows an example of the behavior of the object 4. This behavior includes, for example, state A (A in Fig. 8), state B (B in Fig. 8), and state C (C in Fig. 8).

[0102] State A shows the object 4 standing as viewed from the light receiving unit 20 above the head.

[0103] State B shows the target object 4 transitioning from state A to a crouching state, as viewed from the light receiving unit 20 above the head.

[0104] State C shows the target object 4 in a crouching state after transitioning from state B, as viewed from the light receiving unit 20 above the head. In state C, the left arm has moved upward from state B in the figure.

[0105] In the behavior of this object 4, as a simulation of state detection, if the behavior of the object 4 stops at state B, and there is no change even after a certain period of time has passed, it is determined to be in an abnormal state. On the other hand, if the object 4 changes its behavior, such as transitioning from state B to state C, it is determined to be in a normal state.

[0106] <Background image Gd1, composite image Gd2, first virtual volume Vv1, second virtual volume Vv2, object image Gdt, and state detection information for states A, B, and C>

[0107] 9 shows an example of the detection information table 51. This detection information table 51 shows, for states A, B, and C, a background image Gd1, a composite image Gd2, a first virtual volume Vv1, a second virtual volume Vv2, an object image Gdt, and state detection information.

[0108] The background image Gd1 in frame 15-3 is common to states A, B, and C. In the compound image Gd2, Gd2A in frame 15-4 corresponds to the real image of state A shown in Fig. 8A, Gd2B in frame 15-5 corresponds to the real image of state B shown in Fig. 8B, and Gd2C in frame 15-6 corresponds to the real image of state C shown in Fig. 8C.

[0109] A first virtual volume Vv1 in frame 15-7 corresponds to Gd1 and is obtained from the background image Gd1.

[0110] In the second virtual volume Vv2, Vv2A in frame 15-8 is derived from composite image Gd2A, Vv2B in frame 15-9 is derived from Gd2B, and Vv2C in frame 15-10 is derived from Gd2C.

[0111] In the object image Gdt, GdtA in frame 15-11 is obtained from the second virtual volume Vv2A, GdtB in frame 15-12 is obtained from Vv2B, and GdtC in frame 15-13 is obtained from Vv2C.

[0112] Then, by comparing the object images GdtA, GdtB, and GdtC, it is possible to detect the behavior state of the object 4. In this case, the state detection indicates that state A indicates that there is movement of the object 4, and that there is similar movement from state A to state B, and that there is similar movement from state B to state C. Therefore, in this case, the state detection indicates that the object 4 is normal.

[0113] <Advantages of the Second Embodiment> According to the second embodiment, any one of the following effects can be obtained. (1) A second virtual volume Vv2 representing the object 4 can be obtained using pixels gi contained in the distance image Gd (background image Gd1, composite image Gd2), and the object 4 is detected using the difference of this second virtual volume Vv2. Therefore, the object 4 of a specific volume can be detected without being affected by the movement of objects 8, etc.

[0114] (2) Since the state of the object 4 is detected using the fluctuation of the second virtual volume Vv2 of the object 4, a highly confidential detection process can be realized without being affected by attribute information such as the shape or gender of the object 8.

[0115] (3) Since the state of the object 4 can be detected mainly using the pixels gi that represent the object 4, the load of information processing required for detection can be reduced, the resources required for processing can be reduced, and processing can be speeded up.

[0116] (4) The object 4 is not limited to being a stationary or moving object, and the virtual volume can be accurately calculated using the distance image. This enables highly accurate state detection, such as distance measurement and height measurement, without being affected by the time difference between pixels.

[0117] Third embodiment In the third embodiment, a distance image of only the background 6 is designated as a background distance image GdA, a distance image including the background 6 and object 8 is designated as a background-object distance image GdB, and a distance image including the object 4, background 6, and object 8 is designated as a background-object-object distance image GdC.

[0118] A of Figure 10 shows an example of a background distance image GdA captured in frame 15-14, B of Figure 10 shows an example of a background-object distance image GdB captured in frame 15-15, and C of Figure 10 shows an example of a background-object-target distance image GdC captured in frame 15-16.

[0119] <Detection system 2 according to the third embodiment> The detection system 2 according to the third embodiment has the same configuration as that shown in FIG. 5, and therefore the description thereof will be omitted.

[0120] <Control by the control unit 14 according to the third embodiment> Similarly, the control by the control unit 14 in the third embodiment includes a) emission control of light Li, b) reception control of reflected light Lf, c) generation process of distance image Gd, d) transmission control of distance image Gd, etc. These controls are similar to those in the second embodiment, and therefore will not be described here.

[0121] <Information processing by the processing device 16> The information processing of the processing device 16 includes processes such as m) acquiring distance image Gd, n) acquiring background difference information, o) acquiring virtual volume difference information, p) detecting the presence of object 4, q) detecting the status of object 4, r) detecting abnormalities in object 4, s) calculating distance image, virtual volume, virtual area, and maximum height Hmax, and presenting status information, and t) generating and updating DB36-2.

[0122] m) Acquisition of distance image Gd Processing device 16 acquires a background distance image GdA, a background-object distance image GdB, and a background-object-object distance image GdC in time series under the control of processor 26. These GdA, GdB, and GdC are executed on a frame-by-frame basis.

[0123] n) Obtaining background difference information Under the control of processor 26, processing device 16 calculates the background difference between background distance image GdA and background-object distance image GdB, and the background difference between background-object distance image GdB and background-object-object distance image GdC, and stores them in DB36-2 (Figure 11).

[0124] o) Acquisition of virtual volumetric difference information Under the control of the processor 26, the processing device 16 compares the virtual volume VvA with the virtual volume VvB, and obtains change information (virtual volume difference information) that indicates the change.

[0125] p) Detection of the presence of object 4 The processing device 16, under the control of the processor 26, detects the presence of the object 4 using the virtual volumetric difference information.

[0126] q) Detection of the state of object 4 Under the control of the processor 26, the processing device 16 calculates the maximum height Hmax and virtual area Vs of the object 4 using the background / object / object distance image GdC. The state of the object 4 is detected using the maximum height Hmax and the virtual area Vs.

[0127] r) Abnormality detection of object 4 Under the control of the processor 26, the processing device 16 compares the background-object-object distance image GdC of the previous frame with the background-object-object distance image GdC of the current frame, calculates the difference between the two, and if there is a difference, detects a change within a predetermined number of frames. If there is a change, it detects that the object 4 is normal, and if there is no change, it detects that there is an abnormality in the object 4.

[0128] s) Calculation of distance image, virtual volume, virtual area, maximum height Hmax, and display of status information Under the control of the processor 26, the processing device 16 presents the distance image Gd, the virtual area for each block, the status information, and the judgment information on the information presenting unit 32. According to this presented information, the presence and status of the object 4, and judgment information showing whether the status is normal or abnormal, can be visually confirmed.

[0129] For this information presentation, the processing device 16 can wirelessly connect the communication unit 34 to the corresponding communication terminal under the control of the processor 26, and present the same information as the information presentation unit 32 to the corresponding communication terminal.

[0130] t) Generating and updating DB36-2 The processing device 16 generates and updates the DB 36-2 stored in the storage unit 28 under the control of the processor 26.

[0131] <db36-2> In the same manner as in the second embodiment, DB 36-2 stores control information of the control unit 14 for detecting the state of the object 4, control information of the processing device 16, processing information of the distance image Gd, state detection information of the object 4, and the like.

[0132] 11 shows an example of DB36-2. This DB36-2 is an example of the database of the present disclosure. This DB36-2 includes a background difference information section 52, a virtual volume difference information section 54, an object presence detection information section 56, an object state detection information section 58, an object abnormality detection section 60, a presentation information section 62, and a history information section 64.

[0133] Background difference information section 52 stores background distance image GdA (52-1) which is background difference information.

[0134] A background / object distance image section 54-1 and a background / object virtual volume section 54-2 are set in the virtual volume difference information section 54. A background / object distance image GdB is stored in the background / object distance image section 54-1. A background / object virtual volume VvB calculated from the background / object distance image GdB is stored in the background / object virtual volume section 54-2.

[0135] The object presence detection information section 56 includes a background / object / object distance image section 56-1, a background / object / object virtual volume section 56-2, a virtual volume change information section 56-3, and a presence detection information section 56-4. The background / object / object distance image section 56-1 stores a background / object / object distance image GdC. The background / object / object virtual volume section 56-2 stores a background / object / object virtual volume VvC acquired from the background / object / object distance image GdC. The virtual volume change information section 56-3 stores change information on the virtual volume. The presence detection information section 56-4 stores presence detection information indicating the presence of an object 4 detected from a change in the virtual volume.

[0136] The object state detection information section 58 has a maximum height section 58-1, a threshold section 58-2, a virtual area section 58-3, a threshold section 58-4, and an object state change section 58-5. The maximum height section 58-1 stores the maximum height Hmax of the object 4 acquired from the background / object / object distance image GdC. The threshold section 58-2 stores a threshold Hth for the maximum height Hmax. The virtual area section 58-3 stores the virtual area Vs of the object 4 acquired from the background / object / object distance image GdC. The threshold section 58-4 stores a threshold Vsth for the virtual area Vs. The object state change section 58-5 stores change information representing a state change of the object 4 calculated from the maximum height Hmax and the virtual area Vs.

[0137] The object abnormality detection section 60 is set with a frame information section 60-1, a difference information section 60-2, a change within a specified number of frames section 60-3, and an abnormality detection information section 60-4. The frame information section 60-1 stores frame information that is the target of the background-object-object distance image GdC to be compared. The difference information section 60-2 stores difference information between frames obtained by comparing the background-object-object distance image GdC of the previous frame with the background-object-object distance image GdC of the current frame. The change within a specified number of frames section 60-3 stores change information of the background-object-object distance image GdC together with the number of frames to be compared. The abnormality detection information section 60-4 stores normality information or abnormality information of the object 4 detected from the presence or absence of a change in the background-object-object distance image GdC.

[0138] The presentation information section 62 stores presentation information such as the distance image Gd.

[0139] The history information section 64 stores history information representing histories such as detection history and state history.

[0140] <Processing procedure for detecting the state of object 4> This processing procedure is a processing procedure for state detection using three distance images: a background distance image GdA, a background-object distance image GdB, and a background-object-object distance image GdC.

[0141] 12 shows a processing procedure for detecting the state of the object 4. This processing procedure includes obtaining background difference information (S301), obtaining virtual volume difference information (S302), detecting the presence of the object 4 (S303), detecting the state of the object 4 (S304), and detecting an abnormality in the object 4 (S305).

[0142] According to this processing procedure, the processing device 16, under the control of the processor 26, acquires background difference information (S301), acquires virtual volume difference information (S302), and based on these, detects the presence of the object 4 (S303), detects the state of the object 4 (S304), detects abnormalities in the object 4 (S305), and returns to S303.

[0143] <Getting background difference information> 13A shows the procedure for acquiring background difference information. In this procedure, image capture unit 12 captures background 6 under the control of control unit 14 (S3011), and processing device 16 acquires background distance image GdA under the control of processor 26 (S3012). This background distance image GdA is stored and recorded in DB 36-2 under the control of processor 26 of processing device 16 (S3013).

[0144] <Acquisition of virtual volume difference information> 13B shows a process for acquiring virtual volumetric difference information, in which image capturing unit 12 captures an image of background 6 including object 8 under the control of control unit 14 (S3021), and processing device 16 acquires a background difference from background distance image GdA under the control of processor 26 (S3022).

[0145] The processing device 16 obtains the background-object distance image GdB from the control unit 14 (S3023), and calculates the background-object virtual volume VvB using this background-object distance image GdB (S3024). This background-object virtual volume VvB is stored and recorded in DB36-2 under the control of the processor 26 of the processing device 16 (S3025).

[0146] <Detection of presence of Object 4> 13C shows a processing procedure for detecting the presence of an object 4. In this processing procedure, under the control of control unit 14, imaging unit 12 captures images of object 4, background 6, and object 8 (S3031), and processing device 16, under the control of processor 26, obtains a background difference from background-object distance image GdB (S3032).

[0147] The processing device 16 acquires the background-object-object distance image GdC from the control unit 14 (S3033), and calculates the background-object-object virtual volume VvC using this background-object-object distance image GdC (S3034). The processing device 16 compares the background-object virtual volume VvB with the background-object-object virtual volume VvC, and calculates the virtual volume difference ΔVv therebetween (S3035). The processing device 16, under the control of the processor 26, performs a threshold determination of the virtual volume difference ΔVv (S3036).

[0148] If ΔVv≧ΔVvth (YES in S3036), the presence of the object 4 is detected (S3037). If ΔVv≧ΔVvth is not satisfied (NO in S3036), the process returns to S3031.

[0149] <Detection of object 4's status> 14A shows a processing procedure for detecting the state of the object 4. In this processing procedure, under the control of the control unit 14, the imaging unit 12 calculates the maximum height Hmax and virtual area Vs of the background-object-object distance image GdC (S3041), and performs a threshold Hth determination of the maximum height Hmax (S3042).

[0150] If Hmax≦Hth (YES in S3042), the virtual area Vs is judged against a threshold value Vsth (S3043). If Vs≧Vsth (YES in S3043), a change in the object 4 is detected (S3044).

[0151] If Hmax≦Hth is not true (NO in S3042), it is detected that the object 4 is lying down (S3045). Similarly, if Vs≧Vsth is not true (NO in S3043), it is detected that the object is not lying down (S3045), and detection of the state of the object 4 is continued.

[0152] <Abnormality detected for object 4> 14B shows a processing procedure for detecting an abnormality in the object 4. In this processing procedure, under the control of control unit 14, imaging unit 12 compares the previous frame and the current frame of background-object-object distance image GdC, calculates the distance image difference ΔX between them (S3051), and performs a threshold ΔXth determination of the distance image difference ΔX (S3052).

[0153] If ΔX≧ΔXth (YES in S3052), a change in the object 4 is detected in the current frame, and this change information is recorded (S3053), and it is determined whether there is no change in a predetermined number of frames n (S3054). If ΔX≧ΔXth is not true (NO in S3052), S3053 is skipped and the process proceeds to S3054.

[0154] If there is no change in the predetermined number of frames n (YES in S3054), an abnormality in the object 4 is detected (S3055), and this process ends. If there is a change in the predetermined number of frames n (NO in S3054), it is detected that the object 4 is normal (S3056), and this process continues.

[0155] <Advantages of the Third Embodiment> The third embodiment also provides the same effects as the second embodiment. EXAMPLES

[0156] 15 illustrates an example of a single chip detection module 11. This detection module 11 includes a processing unit 66 having the same functions as the processing device 16. In this detection module 11, the same reference numerals are used for the parts common to the detection system 2 described above, and the description thereof will be omitted.

[0157] <Effects of the embodiment> According to this embodiment, any one of the following effects can be obtained. (1) It can be widely used to detect the state of an object 4 such as a human being.

[0158] (2) The state of the object 4 can be detected without regard for privacy, such as gender, and the like, and the device can be used to detect the state of the object in a bathroom, toilet, etc.

[0159] Other Embodiments (1) In the above embodiment, a human is used as the object 4, but a moving object other than a human, such as a moving object such as an automobile or a robot, may be used as the object 4.

[0160] (2) In the above embodiment, a single detection module is illustrated. However, multiple detection modules obtained by using multiple cameras may be used in combination.

[0161] (3) Regarding detection of the state of the object 4, a detection time may be set, and whether the object 4 is normal or abnormal may be detected based on the presence or absence of behavior within this detection time.

[0162] (4) In the above embodiment, the processing device 16 may detect a change in the state of the object 4 from the difference between before and after by comparing the virtual area or virtual volume between frames.

[0163] As described above, the most preferred embodiment of the present invention has been described. The technology of the present disclosure is not limited to the above description. Various modifications and changes are possible for those skilled in the art based on the gist of the invention described in the claims or disclosed in the specification. It goes without saying that such modifications and changes are included in the scope of the present invention. [Industrial Applicability]

[0164] According to the status detection system, method, program, and detection module disclosed herein, the presence and status of an object can be detected easily and with high accuracy using a virtual volume image, maximum height, and virtual area calculated from a distance image obtained from an object such as a human. [Explanation of symbols]

[0165] 2. Detection System 4. Object 4a head 4b Torso 4c Limbs 6 Background 8 objects 10 Light emitting unit 11 Detection Module 12 Imaging unit 13 Processing section 14 Control section 15-1, 15-2, 15-3, 15-4, 15-5, 15-6, 15-7, 15-8, 15-9, 15-10, 15-11, 15-12 Frame 16 Processing equipment 18 Light Emitting Driver 20 Light receiving part 22 Distance image generation unit 26 processors 28 Memory section 30 Input / output section (I / O) 32 Information presentation section 34 Communications Department 36-1, 36-2 Detection information database (DB) 38 Distance image section 38-1 Background image section 38-2 Composite Image Section 40 Virtual volume 40-1 First virtual volume 40-2 Second virtual volume 42 Virtual Area Section 42-1 Area part 42-2 Threshold section 44 Composite length section 44-1 Length section 44-2 Threshold section 46 Object Section 46-1 Detection Information Department 46-2 Status detection section 48 Presentation Information Department 50 History Information Section 51 Detection Information Table 52 Background difference information section 52-1 Background distance image section 54 Virtual volume difference information section 54-1 Background / Object Distance Image Section 54-2 Background / Object Virtual Volume 56 Object Presence Detection Information Department 56-1 Background / Object / Target Distance Image Section 56-2 Background, Object, and Target Virtual Volume 56-3 Virtual volume change information section 56-4 Presence detection information section 58 Object Status Detection Information Department 58-1 Maximum height 58-2, 58-4 Threshold section 58-3 Virtual Area Section 58-5 Object state change section 60 Object abnormality detection unit 60-1 Frame information section 60-2 Difference information section 60-3 Changes within the specified number of frames 60-4 Anomaly detection information section 62 Presentation Information Department 64 History Information Section 66 Processing section

Claims

1. An imaging unit that pre-acquires a first distance image representing a background and acquires a second distance image including at least the background and an object; A processing unit that calculates a first virtual volume representing the background from the first distance image, calculates a second virtual volume representing the object from the second distance image, and compares the first virtual volume and the second virtual volume to detect the object; A detection system comprising the above.

2. An imaging unit that pre-acquires a first distance image representing a background together with an object other than the object to be detected, and acquires a second distance image including the background, the object, and the object; A processing unit that calculates a first virtual volume representing the object and the background from the first distance image, calculates a second virtual volume representing the object and the object from the second distance image, and compares the first virtual volume and the second virtual volume to detect the object; A detection system comprising the above.

3. The detection system according to claim 1 or claim 2, wherein the processing unit calculates the distance and / or the virtual area of the object from the imaging unit, and detects the state of the object using the distance and / or the virtual area from the imaging unit.

4. The detection system according to claim 1 or claim 2, further comprising an information presentation unit that presents any one or two or more of the first distance image, the second distance image, the first virtual volume image, the second virtual volume image, the maximum height, the virtual area, and the state information representing the state of the object.

5. A step in which the imaging unit pre-acquires a first distance image representing a background for an object to be detected and acquires a second distance image including at least the background and the object; A step in which the processing unit calculates a first virtual volume representing the background from the first distance image, calculates a second virtual volume representing the object from the second distance image, and compares the first virtual volume and the second virtual volume to detect the object; A detection method comprising the above.

6. A program for causing a computer to execute, A function of pre-acquiring a first distance image representing a background for an object to be detected; A function of acquiring a second distance image including at least the background and the object; A function of calculating a first virtual volume representing the background from the first distance image; A function of calculating a second virtual volume representing the object from the second distance image; A function of comparing the first virtual volume and the second virtual volume to detect the object and calculating the maximum height or the virtual area from the object; A program for causing the computer to execute.

7. An imaging unit that pre-acquires a first distance image representing a background for an object to be detected and acquires a second distance image including at least the background and the object; A processing unit that calculates a first virtual volume representing the background from the first distance image, calculates a second virtual volume representing the object from the second distance image, and compares the first virtual volume and the second virtual volume to detect the object; A detection module including the above.

Images