Image Processing Method

Abstract

To detect a moving object accurately from a range image.SOLUTION: An image processing apparatus 10 includes: a first processing unit 12 which executes processing to detect a background section from a range image and detect a moving body from the range image on the basis of the background section; and a second processing unit 13 which detects the moving body on the basis of changes in distance in height direction in the range image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing method and an image processing device for performing processing to detect a moving object from a distance image. [Background technology]

[0002] In recent years, the elderly population has increased, and the demand for nursing care has risen. On the other hand, nursing care requires many caregivers, which causes problems such as an increase in the caregiver's workload and labor costs. Therefore, many monitoring systems have been developed to monitor people such as care recipients using distance images captured by a distance image sensor, as described in Patent Document 1.

[0003] In the surveillance system disclosed in Patent Document 1, the position of the bed is extracted from a distance image, and the area of ​​the person is detected to determine the person's movement. Specifically, the position of the bed is obtained from the frequency distribution of the height in the longitudinal and lateral directions of the bed. Furthermore, the movement of the person is determined by detecting the position of an object relative to a predetermined height position, and the person's movement, such as lying down, sitting up, or standing up, is determined. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-030042 A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, when detecting the movement of a bed or a person from a distance image as described above, the background part may be identified using the difference between distance images captured at different times. However, even if a person actually exists in a location, if the person is not moving, the change in distance in the distance image is small and the person may be mistakenly identified as the background part. In such a case, it may be difficult to newly detect a person or to track a person that has already been detected in a location that has been mistakenly identified as the background part. In other words, a problem occurs in which it is difficult to detect a moving object more accurately from a distance image.

[0006] Therefore, an object of the present invention is to solve the above-mentioned problem that it is difficult to detect a moving object with high accuracy from a range image. [Means for solving the problem]

[0007] An image processing method according to one aspect of the present invention includes: A background portion is detected from the distance image, and a moving object is detected from the distance image based on the background portion. and performing a second process of detecting a moving object based on a change in distance in a height direction in the distance image. The structure is as follows.

[0008] In addition, in the image processing method, a first process for detecting a new moving object using a difference in distance from the background portion to the distance image and tracking the detected moving object; The second process is performed in parallel with the first process. The structure is as follows.

[0009] In the image processing method, Prior to the second process, a process of detecting a background portion in a height direction from the distance image is performed; The second process detects a moving object from the distance image based on a difference in a height direction between the moving object and a background portion in the height direction. The structure is as follows.

[0010] In the image processing method, Prior to the second process, a process is performed on the distance image to detect a substantially horizontal plane of a predetermined size as a background portion in a height direction; The second process detects a moving object from the distance image based on a difference in a height direction between the moving object and a background portion in the height direction. The structure is as follows.

[0011] In the image processing method, The second process detects a moving object at a location in the distance image where a difference in a height direction from a background portion in the height direction is equal to or greater than a predetermined value. The structure is as follows.

[0012] In the image processing method, The first process tracks a moving object detected in the second process. The structure is as follows.

[0013] Further, an image processing device according to one aspect of the present invention comprises: a first processing unit that detects a background portion from a distance image and detects a moving object from the distance image based on the background portion; a second processing unit that detects a moving object based on a change in distance in a height direction in the distance image; Equipped with The structure is as follows.

[0014] In addition, in the image processing device, the first processing unit detects a new moving object from the distance image using a difference in distance from the background portion and tracks the detected moving object; The second processing unit operates in parallel with the processing of the first processing unit. The structure is as follows.

[0015] In addition, in the image processing device, the second processing unit performs a process of detecting a background portion in a height direction from the distance image, and performs a process of detecting a moving object from the distance image based on a difference in a height direction from the background portion in the height direction. The structure is as follows.

[0016] In addition, in the image processing device, The first processing unit tracks the moving object detected by the second processing unit. The structure is as follows.

[0017] In addition, a program according to one aspect of the present invention includes: In the information processing device, a first processing unit that detects a background portion from a distance image and detects a moving object from the distance image based on the background portion; a second processing unit that detects a moving object based on a change in distance in a height direction in the distance image; To achieve this, The structure is as follows. Effect of the Invention

[0018] With the above-described configuration, the present invention can detect a moving object from a range image with higher accuracy. [Brief description of the drawings]

[0019] [Figure 1] 1 is a schematic diagram showing a configuration of an information processing system according to a first embodiment of the present invention. [Diagram 2] 2 is a block diagram showing a configuration of the monitoring server disclosed in FIG. 1. [Diagram 3] 2 is a diagram showing how a distance image is processed by the monitoring server disclosed in FIG. 1; [Figure 4] 2 is a diagram showing how a distance image is processed by the monitoring server disclosed in FIG. 1; [Diagram 5] 2 is a diagram showing how a distance image is processed by the monitoring server disclosed in FIG. 1; [Figure 6]2 is a flowchart showing a processing operation of the monitoring server disclosed in FIG. 1; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] <Embodiment 1> A first embodiment of the present invention will be described with reference to Fig. 1 to Fig. 6. Fig. 1 and Fig. 2 are diagrams showing the configuration of an information processing system. Fig. 3 to Fig. 6 are diagrams for explaining the processing operation of the information processing system. Below, the configuration and operation of the information processing system will be described with reference to each diagram.

[0021] The information processing system of the present invention is an information processing system for monitoring the behavior of a person P who is a monitoring target, such as a care recipient or a monitored person. The information processing system is configured to include a monitoring server 10, a distance image camera C, a monitoring terminal 30, and a mobile terminal 40.

[0022] The distance image camera C (imaging device) is disposed in a position capable of capturing distance images within a room R1 (first area), which is an area in which a bed B on which a person P to be monitored is lying is disposed. For example, the distance image camera C is provided on the ceiling of a room R1 in a medical facility, a care facility, a house, or the like, and is disposed in a position where most of the room R1 fits within the image. Note that the distance image camera C is not necessarily limited to being installed on the ceiling, and may be installed anywhere, such as on a wall or a stand. However, the bed B installed in the room R1 may be any type of sleeping surface, such as a futon.

[0023] The distance imaging camera C captures distance images with pixel values ​​as distance values. For example, the distance imaging camera C may be of a type that measures distance from the time it takes for an infrared laser to travel back and forth when projected onto a target, and may be any type of imaging device. The distance imaging camera C has a function of capturing distance images at regular time intervals or when an instruction to capture images is received, and transmitting the captured distance images to a monitoring server 10, which will be described later. The function of the monitoring server 10, which will be described later, may also be incorporated in the distance imaging camera C. In other words, the function of the monitoring server 10 may be realized by an information processing device incorporated in the distance imaging camera C.

[0024] The monitoring terminal 30 (monitoring device) is an information processing terminal operated by a caregiver who cares for the person P who is being cared for, or a user U who is a supervisor who monitors the person P. The monitoring terminal 30 is installed in a place (second area) away from the room R1 (first area) of the person P, such as a monitoring room R2 where the user U, such as a caregiver, is present, and can be operated by multiple users U. The monitoring terminal 30 displays a distance image on the display screen, enables input of operation instructions related to monitoring, and has a function of transmitting such operation instructions to the monitoring server 10 and receiving a notification of the status of the person P issued from the monitoring server 10 and notifying the users U.

[0025] The mobile terminal 40 (monitoring device) is an information processing terminal such as a smartphone held and operated by the user U. The mobile terminal 40 is capable of communicating with the monitoring server 10 via wireless communication, and is located in a place (second area) away from the room R1 (first area) of the person P, such as a monitoring room R2 where the user U, such as a caregiver, is present or in the rooms of other care recipients, and can be operated by multiple users U. The mobile terminal 40 has a function of displaying a distance image on a display screen, enabling input of operation instructions related to monitoring, transmitting such operation instructions to the monitoring server 10, and receiving a notification of the status of the person P issued from the monitoring server 10 and notifying the user U.

[0026] The monitoring server 10 mainly has a function of detecting a specific motion state of the person P in the room of the person P by using a distance image from the distance image camera C. The monitoring server 10 is composed of one or more information processing devices having a calculation device and a storage device. Specifically, as shown in FIG. 2, the monitoring server 10 includes an image acquisition unit 11, a motion detection unit 12, a height direction processing unit 13, and a notification unit 14, which are constructed by the calculation device executing a program. The monitoring server 10 also includes a criterion storage unit 16 formed in the storage device. Each component will be described in detail below, and the operation of the monitoring server 10 will also be described with reference to the flowchart of FIG. 6. The function of the monitoring server 10 can also be mounted on the distance image camera C. That is, the function of the monitoring server 10 may be realized by an information processing device mounted on the distance image camera C.

[0027] The image acquisition unit 11 acquires a distance image of the inside of the room R1 including the bed B, which has been photographed by the distance image camera C as described above, from the distance image camera C. The acquired distance image is then passed to the motion detection unit 12 and the height direction processing unit 13, whereby the distance image is used for processing described below. Note that the image acquisition unit 11 may always transmit the acquired distance image as is to the monitoring terminal 30 or the mobile terminal 40, and output it to be displayed on the display device of the monitoring terminal 30 or the mobile terminal 40.

[0028] The motion detection unit 12 (first processing unit) performs a process of detecting a person P in the distance image. Specifically, the motion detection unit 12 first detects, as a background portion, a portion where the difference between distance images obtained in advance and backward in time is smaller than a predetermined value (step S1 in FIG. 6). For example, in a distance image of the room R1, a portion where a fixed object such as a floor surface, a wall surface, or a bed B is reflected is detected as a background portion. At this time, the motion detection unit 12 detects a bed area, which is an area of ​​the bed B (a sleeping place), from the detected background portion. For example, the motion detection unit 12 detects a plane extending horizontally with a predetermined width at a predetermined height position from the distance image, and detects, from the plane, a plane area corresponding to the bed information stored in advance as the bed area. For example, the bed information is stored in the determination criterion storage unit 16, and the bed information includes information such as the width of the bed, the lengths of the long and short sides of the rectangular outer shape, and the height from the floor surface F. Then, the motion detection unit 12 detects, in particular, an edge portion that is an edge that forms the outer shape of the bed area. For example, the motion detection unit 12 detects, as an edge portion, a linear portion located near the outer shape of the bed B where the distance values ​​between adjacent pixels change abruptly.

[0029] Then, the motion detection unit 12 calculates the difference in distance between each pixel of the newly captured distance image and the background, and detects the presence of a moving object (moving body) in a location where the difference is greater than a predetermined value (step S2 in FIG. 6). The motion detection unit 12 further detects a person P based on the shape of the detected object, and tracks the person P in distance images captured sequentially thereafter (step S3 in FIG. 6). For example, the motion detection unit 12 detects a person P consisting of multiple body parts such as a pair of a head and a torso from the shape of the detected object (e.g., aspect ratio), the positional relationship between the objects, and the volume ratio (area ratio). In addition, for example, the motion detection unit 12 tracks parts of the person P (such as the head and torso) that can be determined to be the same by pattern matching the shapes of the body parts in the new distance image, or by using the optical flow of the feature points of the parts. At this time, the motion detection unit 12 detects an object existing near the position of the person P detected immediately before in the new distance image, and tracks the object using pattern matching or the like.

[0030] Furthermore, the motion detection unit 12 detects the motion state of the followed person P (step S4 in FIG. 6) and judges whether or not the motion state is a specific motion state (step S5 in FIG. 6). For example, the motion detection unit 12 detects the motion state of the followed person P based on the positional relationship of the body parts (head, torso, etc.) of the person P. As an example, the motion detection unit 12 detects the motion state of the person P based on the positional relationship between the height of the person P that can be detected from the head, the center of gravity position that can be detected from the torso, and the bed B in the distance image, and judges whether or not the motion state is a specific motion state set in advance. For example, when the person P is lying on the bed B (supine position), it simply recognizes that the person P is in that position (lying on the bed B) and does not judge that the motion state is a specific motion state (No in step S5 in FIG. 6). On the other hand, when the person P is located at the boundary of the bed B, when the person P is crouched away from the bed B, or when the person P is lying down away from the bed B, the motion detection unit 12 determines that the motion state of the person P is a specific motion state that may be determined as abnormal (Yes in step S5 of FIG. 6). Note that the criteria for determining that the motion state of the person P is a specific motion state based on the height, center of gravity, positional relationship with the bed B, etc., as described above, are stored in advance in the determination criteria storage unit 16.

[0031] When the motion detection unit 12 determines that the person P is in a specific motion state (Yes in step S5 in FIG. 6), the notification unit 14 notifies the user U that the person P may be in an abnormal state (step S6 in FIG. 6). For example, the notification unit 14 transmits notification information to the monitoring terminal 30 or the mobile terminal 40 to notify that the person P may be in an abnormal state. However, the specific motion state determined in the present invention is not limited to the above-mentioned states, and other motion states may be determined as the specific motion state. For example, a state in which the person P stands on the bed B (standing state) and a state in which the person P sits on the bed B (sitting up state) may be determined as the specific motion state that may be determined as abnormal.

[0032] Here, as shown in the distance image of FIG. 3, a case where a person P is lying on the floor away from the bed B will be described. First, in the distance image in which the person P is not lying on the floor, as shown in FIG. 4(1), for example, the motion detection unit 12 detects a portion of the distance D1 from the distance image camera C to the floor as a background portion. Then, as shown in FIG. 3 and FIG. 4(2), when the person P is lying on the floor, the difference between the distance D2 from the distance image camera C to the person P and the distance D1 from the distance image camera C to the floor is small, and the person P portion may be determined as the background portion as it is. Then, the motion detection unit 12 may not be able to detect the person P lying on the floor, or may not be able to track the person P that was being tracked on the floor. In addition, not only on the floor, but also on the bed B or on a wall, the motion detection unit 12 may not be able to detect or track the person P located at that portion, and may detect it as a background portion. Taking such a situation into consideration, in parallel with the detection and tracking process (first process) of person P based on the distance image by the motion detection unit 12 described above, the height direction processing unit 13 executes the second process described below.

[0033] Prior to the second process described below, the height direction processing unit 13 (second processing unit) first extracts a height direction distance value from the distance value of each pixel of the distance images before and after in time, and detects the background part in the height direction using the difference between the height direction distance values ​​(step S11 in FIG. 6). That is, as shown in FIG. 5(1), the height direction processing unit 13 detects only the height direction distance value D3 for the surface of a fixed object that forms a substantially horizontal plane of a predetermined size, such as a floor surface or a bed B, as the background part in the height direction. Thereafter, as shown in FIG. 5(2), the height direction processing unit 13 extracts the height direction distance value D4 for each pixel of the newly captured distance image, and calculates the difference from the height direction distance value D3 of the background part. Then, the height direction processing unit 13 detects the presence of an object (moving object) at a location where the difference is larger than a predetermined value (step S12 in FIG. 6). In this way, by extracting the height direction component from the distance value of the distance image and examining the change in the distance value for the background part in the height direction, if an object is located in the background part, the change will be greatly indicated. As a result, for example, even if a person P lies on the floor as shown in Fig. 4(2), it is possible to detect the presence of the person P on the floor. Then, the height direction processing unit 13 notifies the motion detection unit 12 that an object exists at the detected location.

[0034] The extraction of the height direction distance value of each pixel by the height direction processing unit 13 can be realized by calculation based on information representing the installation status of the distance image camera C, such as the installation height and the shooting angle in the horizontal or vertical direction of the distance image camera C. For example, the height direction processing unit 13 stores in advance the shooting angle in the horizontal or vertical direction corresponding to each pixel of the distance image camera C, and can calculate the height direction distance values ​​D3, D4 from the distance values ​​D1, 2 measured for each pixel based on such shooting angle.

[0035] Then, when the motion detection unit 12 receives notification of the presence of an object from the height direction processing unit 13, it performs detection processing of person P at that location, and, as described above, tracks the detected person P in the subsequent distance images (step S3 in Figure 6) and judges whether there are any abnormalities in the motion (steps S4 and S5 in Figure 6).

[0036] As described above, according to the present invention, even if a moving object such as a person P is present in an area detected as the background, the moving object is detected using the change in distance value in the height direction of the distance image, making it possible to detect the moving object more accurately from the distance image.

[0037] The above-mentioned program is stored in a storage device or a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0038] Although the present invention has been described with reference to the above-mentioned embodiment, the present invention is not limited to the above-mentioned embodiment. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0039] 10 Monitoring Server 11 Image acquisition section 12 Motion detection unit 13 Height direction processing section 14 Notification Department 16 Judgment criteria storage section 30 Monitoring terminal 40 Mobile Devices B Bed C. Range imaging camera P person U User

Claims

1. a first process is performed when a background portion is detected from a distance image and a moving object is detected from the distance image based on the background portion, the first process detects a new moving object from the distance image using a difference in distance from the background portion and tracks the detected moving object; Furthermore, in parallel with the first process, a second process is performed to detect a moving object based on a change in distance in a height direction in the range image. Prior to the second process, a process of detecting a background portion in a height direction from the distance image is performed; the second processing includes performing processing for detecting a moving object on the distance image based on a difference in a height direction distance from a background portion in the height direction detected before the second processing. Image processing methods.

2. 2. The image processing method according to claim 1, Prior to the second process, a process is performed on the distance image to detect a substantially horizontal plane of a predetermined size as a background portion in the height direction; the second processing includes performing processing for detecting a moving object on the distance image based on a difference in a height direction distance from a background portion in the height direction detected before the second processing. Image processing methods.

3. 3. The image processing method according to claim 1, further comprising: The second processing detects a moving object at a location in the distance image where a difference in a height direction distance from a background part in the height direction detected before the second processing is equal to or greater than a predetermined value. Image processing methods.

4. 4. An image processing method according to claim 1, further comprising: The first process tracks a moving object detected in the second process. Image processing methods.

5. a first processing unit that, when detecting a background portion from a distance image and detecting a moving object from the distance image based on the background portion, detects a new moving object from the distance image using a difference in distance from the background portion and tracks the detected moving object; a second processing unit that performs a second process of detecting a moving object based on a change in distance in a height direction in the distance image; Equipped with The second processing unit operates in parallel with the processing of the first processing unit, Furthermore, the second processing unit performs a process of detecting a background portion in a height direction for the distance image before the second process, and performs the second process of detecting a moving object for the distance image based on a difference in a distance in a height direction from the background portion in the height direction detected before the second process. Image processing device.

6. 6. The image processing device according to claim 5, The first processing unit tracks the moving object detected by the second processing unit. Image processing device.

7. In the information processing device, a first process is performed when a background portion is detected from a distance image and a moving object is detected from the distance image based on the background portion, the first process detects a new moving object from the distance image using a difference in distance from the background portion and tracks the detected moving object; Furthermore, in parallel with the first process, a second process is performed to detect a moving object based on a change in distance in a height direction in the range image. Prior to the second process, a process of detecting a background portion in a height direction from the distance image is performed; the second processing includes performing processing for detecting a moving object on the distance image based on a difference in a height direction distance from a background portion in the height direction detected before the second processing. A program to make it happen.

Images