Motion recognition system, motion recognition method, and program

Abstract

The present invention detects a state and a specific motion behavior of a person in consideration of protection of privacy. This motion recognition system recognizes motion of a subject and includes: a storage means that stores specific motion data indicating a specific motion including a series of predefined postures and motions of at least one body part of the subject; a detection means that continuously acquires point cloud data indicating a three-dimensional coordinate position on a surface of the subject and detects the motion of the subject from the point cloud data; a display means that continuously displays, as a moving image, the motion of the subject detected by the detection means as point cloud data; an estimation means that estimates the motion of the subject from a continuous change in the point cloud data; a reception means that receives designation of a specific motion among a plurality of motions of the subject; and a notification means that notifies that the specific motion has occurred when the motion of the subject is a motion matching the designated specific motion.

Description

Action recognition system, action recognition method, and program

[0006] ,

[0001] The present invention relates to an action recognition system, an action recognition method, and a program.

[0002] An input receiving unit that reproduces and displays a first moving image of a person performing a specific action, and receives an instruction input for instructing a position related to the specific action on the frame of the reproduced and displayed first moving image during the reproduction display; a second feature point group including one or more feature points included in the second moving image; and a determination unit that calculates the similarity between the first moving image and the second moving image based on a comparison result of feature amounts between the first feature point group including two or more feature points included in the first moving image, and determines whether the specific action is included in the second moving image based on the calculated similarity. When calculating the similarity, the determination unit assigns a greater weight to the comparison result of the feature amounts for the feature points closer to the instruction position by the instruction input among the first feature point group. An action recognition system having the determination unit is known (Patent Document 1).

[0003] A nurse call system having a nurse call sub-unit installed on a bed for a patient to call a nurse, and a nurse call main unit installed in a nurse station to respond to a call by the nurse call sub-unit. The nurse call system includes a camera that images a patient on the bed from above the bed, and a state determination unit that analyzes the captured image of the camera, detects a change in the state of the patient, and outputs a first signal when a state change occurs. The nurse call main unit has a notification unit that executes a first notification operation upon receiving the first signal. The state determination unit outputs a second signal different from the first signal when detecting a predetermined action of a person including at least one of the patient and the nurse by analyzing the captured image. The camera or the nurse call main unit executes a second notification operation upon receiving the second signal. A nurse call system is also known (Patent Document 2).

[0004] Japanese Patent Application Laid-Open No. 2022-77870, Japanese Patent Application Laid-Open No. 2023-51150

[0005] The present invention detects a person's state and specific action behavior in consideration of privacy protection.

[0006] To solve the aforementioned problems, the motion recognition system described in claim 1 is a motion recognition system for recognizing the motion of an object, comprising: storage means for storing specific motion data indicating a specific motion including a predetermined series of postures and movements of at least one body part of the object; detection means for continuously acquiring point cloud data indicating three-dimensional coordinate positions on the surface of the object and detecting the motion of the object from the point cloud data; display means for continuously displaying the motion of the object detected by the detection means as a moving image from the point cloud data; estimation means for estimating the motion of the object from the continuous changes in the point cloud data; receiving means for receiving the designation of a specific motion from among a plurality of motions of the object; and notification means for notifying that a specific motion has occurred when the motion of the object matches the designated specific motion.

[0007] The invention described in claim 2 is characterized in that, in the motion recognition system described in claim 1, the detection means projects invisible light from an array light source consisting of a plurality of light-emitting elements via a diffractive optical element and receives the reflected light reflected from the surface of the target to acquire the point cloud data.

[0008] The motion recognition system according to claim 3 is characterized in that, in the motion recognition system according to claim 2, the estimation means estimates the motion of the target using a learning model that has been trained using training data in which annotations indicating the motion of the target have been added to the acquired point cloud data.

[0009] The invention described in claim 4 is characterized in that, in the motion recognition system described in any one of claims 1 to 3, the specific motion is a body motion that includes the angle and width of the movement of the target body part.

[0010] The invention described in claim 5 is characterized in that, in the motion recognition system described in claim 1, when it is notified that the specific motion has occurred, the point cloud data acquired by the detection means is recorded.

[0011] To solve the aforementioned problems, the motion recognition method described in claim 8 is a motion recognition method for recognizing the motion of an object, comprising: a storage step of storing specific motion data that indicates a specific motion including a predetermined series of postures and movements of at least one body part of the object; a detection step of continuously acquiring point cloud data indicating three-dimensional coordinate positions on the surface of the object and detecting the motion of the object from the point cloud data; a display step of continuously displaying the motion of the object detected in the detection step as a moving image using the point cloud data; an estimation step of estimating the motion of the object from the continuous changes in the point cloud data; a reception step of receiving a designation of a specific motion from among a plurality of motions of the object; and a notification step of notifying that a specific motion has occurred when the motion of the object matches the designated specific motion.

[0012] To solve the aforementioned problems, the program described in claim 6 is characterized by causing a computer to execute: a storage step of storing specific action data that includes a predetermined series of postures and movements of at least one body part of an object; a detection step of continuously acquiring point cloud data indicating three-dimensional coordinate positions on the surface of the object and detecting the movement of the object from the point cloud data; a display step of continuously displaying the movement of the object detected in the detection step as a moving image using the point cloud data; an estimation step of estimating the movement of the object from the continuous changes in the point cloud data; a reception step of receiving a designation of the specific action from among a plurality of movements of the object; and a notification step of notifying that the specific action has occurred when the movement of the object matches the designated specific action.

[0013] According to the invention described in claim 1, a person's state and specific actions can be detected while taking into consideration the protection of privacy.

[0014] According to the invention described in claim 2, the movement of an object can be acquired as point cloud data using invisible, safe light.

[0015] According to the invention described in claim 3, the accuracy of estimating the target's operation can be improved.

[0016] According to the invention described in claim 4, the specific actions of the target can be visualized.

[0017] According to the invention described in claim 5, the occurrence of a specific action can be recorded.

[0018] According to the inventions described in claims 6 and 7, a person's state and specific actions can be detected while taking into consideration the protection of privacy.

[0019] This is a block diagram showing the functional configuration of the motion recognition system according to this embodiment. This is a diagram showing an example of a motion pattern for a specific action in this embodiment. This is a block diagram showing the functional configuration of a 3D sensor. Figure 4A is a schematic diagram showing the dot pattern irradiation of the 3D sensor, and Figure 4B is a diagram showing an example of the measurement range of the 3D sensor. This is a schematic diagram showing an example of the arrangement of 3D sensors provided in the motion recognition system. This is a flowchart showing the flow of the detection process for detecting the three-dimensional movement of an object in the motion recognition system. This is a conceptual diagram showing an example of the process of extracting point cloud data of only the patient from acquired point cloud data. This is a conceptual diagram explaining an example of the process of removing noise from point cloud data. This is a diagram explaining an example of calculating the 3D coordinate information value of a patient using trigonometry.

[0020] Next, specific examples of embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments. It should be noted that in the following explanation using the drawings, the drawings are schematic, and the ratios of the dimensions, etc., may differ from those of reality, and for ease of understanding, drawings of components other than those necessary for the explanation have been omitted as appropriate.

[0021] (1) Overall Configuration of Motion Recognition System Figure 1 is a block diagram showing the functional configuration of the motion recognition system 1 according to this embodiment, Figure 2 is a diagram showing an example of a motion pattern for a specific operation in this embodiment, Figure 3 is a block diagram showing the functional configuration of the 3D sensor 20, Figure 4A is a schematic diagram showing the dot pattern irradiation of the 3D sensor 20, and Figure 4B is a diagram showing an example of the measurement range of the 3D sensor 20. The overall configuration of the motion recognition system 1 will be described below with reference to the drawings.

[0022] The motion recognition system 1 according to this embodiment is a system that selects a pattern of specific movements of a target person and displays an alert when a specific movement occurs while the person is being imaged. As shown in Figure 1, the motion recognition system 1 includes a specific movement data storage unit 10 that stores specific movement data indicating a specific movement including a predetermined series of postures and movements of at least one body part of the target M, a three-dimensional sensor 20 as a detection means that continuously acquires point cloud data P indicating the three-dimensional coordinate position on the surface of the target M and detects the movement of the target M from the point cloud data P, and a display unit 30 as a display means that continuously displays the movement of the target M detected by the three-dimensional sensor 20 as a moving image. Furthermore, the motion recognition system 1 includes a motion estimation unit 40 as an estimation means for estimating the motion of the target M from the continuous changes in point cloud data P, an input reception unit 50 as a reception means for receiving the designation of a specific motion from among a plurality of motions of the target M, a notification unit 60 as a notification means for notifying that a specific motion has occurred when the motion of the target M matches the designated specific motion, and a recording unit 70 for recording the point cloud data P acquired by the 3D sensor 20.

[0023] In the motion recognition system 1, the processing of the specific motion data storage unit 10, display unit 30, motion estimation unit 40, input reception unit 50, notification unit 60, and recording unit 70 is implemented by a general-purpose computer in which a processor consisting of a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory) executes a predetermined program. The 3D sensor 20 can perform light emission control, light reception control, and communication control using a program processing unit implemented as a single-board computer equipped with an ARM processor, such as a Raspberry Pi (registered trademark), or a small computer device, but is not particularly limited.

[0024] (Specific Action Data Storage Unit) The specific action data storage unit 10 stores specific actions, including a predetermined series of postures and movements of body parts of the target M, as action pattern data. Examples of specific actions include, but are not limited to, lying in bed, sitting up halfway out of bed, sitting on the bed, standing up from bed, touching the arm, touching the neck, touching the shoulder, and standing, as shown in Figure 2 as an example of an action pattern.

[0025] (3D Sensor) As shown in the block diagram of Figure 3, the 3D sensor 20 comprises a projector 21 consisting of a light-emitting element that emits light, an image sensor module 22 consisting of an image sensor that receives reflected light reflected from the surface of the object to be imaged, and a control unit 23 that controls the light emission of the projector 21, the light reception of the image sensor module 22, and the processing of the received light data.

[0026] The projector 21 consists of a VCSEL (Vertical Cavity Surface Emitting Laser) 211, which is an array light source composed of multiple light-emitting elements, and a transmissive diffractive optical element (DOE) 212 that controls the laser light distribution above the VCSEL 211. The VCSEL 211 emits invisible near-infrared laser light with a wavelength of 940 nm. Therefore, imaging is possible regardless of whether it is daytime or nighttime. The diffractive optical element 212 converts the light emitted by the VCSEL 211 into a regular dot pattern distribution pattern and irradiates the target M as multiple beam-shaped ranging beams.

[0027] Figure 4A shows an example in which the diffractive optical element 212 converts the light emitted from the projector 21 into ranging beams L1 to Ln with a spread angle θ so that they are distributed to the target M as a regular dot pattern. In this embodiment, the dot pattern is irradiated with 4000 to 5000 laser beams, and as an example, as shown in Figure 4B, the measurement distance (L) is 300 mm to 2000 mm and the measurement range (R) is 285 × 285 mm to 1900 × 1900 mm.

[0028] The image sensor module 22 uses a CMOS (Complementary Metal Oxide Semiconductor) image sensor, which is a photoelectric conversion element. The image sensor module 22 receives the reflected light from the target M when invisible near-infrared laser light emitted from the projector 21 is reflected, and acquires point cloud data P.

[0029] In this embodiment, the control unit 23 is composed of a Raspberry Pi (registered trademark), which is a single-board computer equipped with an ARM processor. The control unit 23 controls the operation of the projector 21 and the image sensor module 22, and processes the point cloud data P acquired by the image sensor module 22 to generate three-dimensional coordinate information values.

[0030] The three-dimensional sensor 20 configured in this way emits invisible laser light from the projector 21 onto the target M, and while receiving the reflected light reflected from the surface of the target M with the image sensor module 22, it acquires point cloud data P and detects the three-dimensional movement of the target M.

[0031] (Display Unit) The display unit 30 can be implemented in various forms, but in this embodiment, it can be implemented as the display of a computer on which the program for the motion recognition system 1 is installed, and outputs various information. In this embodiment, the motion of the target M detected by the 3D sensor 20 is displayed in real time as point cloud data P having 3D coordinate information values. The display unit 30 may also be configured as a touch panel display and serve as the input reception unit 40.

[0032] The operation of the target M displayed on the display unit 30 is displayed as point cloud data P having three-dimensional coordinate information values. Therefore, compared to camera images (RGB color images) taken with a webcam capable of capturing both still and moving images, the risk of acquiring personally identifiable information is lower, and the display is designed with privacy protection in mind.

[0033] (Motion Estimation Unit) The motion estimation unit 40 estimates the motion of the target M from the continuous changes in the point cloud data P acquired by the 3D sensor 20. Specifically, it estimates the motion of the target M using a learning model that has been trained on point cloud data P with annotations indicating the motion of the target M as training data. It is preferable to perform annotation work in advance and create a learning database as training data. For example, continuous changes (video) in the point cloud data P include "touching the body with the hand", "difficult to determine whether or not it is touching", "the state of the body changes", "another object is visible", etc., and annotations are added to these intervals.

[0034] (Input Reception Unit) The input reception unit 50 receives the specification of a specific operation from among multiple operations of the target M. Specifically, for example, it receives input information of a specific operation pattern specified by the user of the operation recognition system 1. Input operations for input information are performed through the input device of the computer constituting the operation recognition system 1. For example, if the computer constituting the operation recognition system 1 is equipped with a touch panel display, the touch panel display functions as the display unit 30 and the input reception unit 50. The input reception unit 50 is not limited to a touch panel display, but may also be, for example, a keyboard, mouse, or mechanical switch. The input reception unit 50 may also be a microphone. Alternatively, the input reception unit 50 may be part of an independent device, such as a remote controller, or part of a mobile terminal (smartphone or tablet terminal) with a dedicated application program installed. In that case, the input reception unit 50 receives input information by receiving the input information sent from the mobile terminal via the network.

[0035] The specific action patterns specified by the user are those pre-stored in the specific action data storage unit 10. Examples include lying in bed, half-sitting up in bed, sitting on the bed, standing up from bed, touching the arm, touching the neck, touching the shoulder, and standing. In addition, the specific action patterns pre-stored in the specific action data storage unit 10 can be increased as actions that require special attention according to the situation of each individual target M, and the user can specify the increased specific action patterns via the input reception unit 50.

[0036] (Notification Unit) The notification unit 60 issues an alert indicating that a specific action has occurred if the action of the target M matches a specified action. The action of the target M that matches a specified action is an action in which the detected action of the target M is judged to have a certain degree of probability of being an action designated by the user as an action that should be detected, prompting the user to check the situation. For example, it determines whether there is a possibility that a specific action such as sitting on the bed, standing up from the bed, touching the arm, touching the neck, touching the shoulder, or standing has occurred, and if it is determined that a specific action has occurred, the notification unit 60 notifies that a specific action has occurred.

[0037] The notification unit 60 may consist of, for example, a speaker that notifies information by sound, a vibrator that notifies information by vibration, or a lamp that notifies information by lighting or flashing. Alternatively, the display unit 30 may function as the notification unit 60 by notifying information by displaying it on a display. The notification unit 60 is not limited to the examples shown herein and may consist of other mechanisms capable of notifying information, or may consist of a combination of multiple such mechanisms. Furthermore, these notification units 60 may be provided independently of the computer that constitutes the operation recognition system 1 and may be controlled by communication.

[0038] When the notification unit 60 notifies that a specific operation has occurred, it records the point cloud data P acquired by the 3D sensor 20. The recording unit 70 that records the point cloud data P can be configured with, for example, a semiconductor memory including non-volatile memory such as NVRAM (Non-Volatile RAM) or a recording medium such as an HDD (Hard Disk Drive). The point cloud data to be recorded can be shared among users by recording it as data in formats such as CSV (Comma Separated Value), JSON (JavaScript Object Notation), and XML (Extensible Markup Language).

[0039] (2) Detection process for specific movements Figure 5 is a schematic diagram showing an example of the arrangement of the three-dimensional sensor 20 provided in the motion recognition system 1, Figure 6 is a flowchart showing the flow of the detection process for detecting the three-dimensional movement of the target M in the motion recognition system 1, Figure 7 is a diagram conceptually showing an example of the process of extracting point cloud data P of only the patient from the acquired point cloud data P, and Figure 8 is a conceptual diagram explaining an example of the process of removing noise from point cloud data P. The detection process for specific movements in the motion recognition system 1 according to this embodiment will be described below with reference to the drawings.

[0040] In the motion recognition system 1 according to this embodiment, as shown in Figure 5, the three-dimensional sensor 20 is installed above the bed 80 and is configured to capture images mainly of the state and movements of patient M, which is an example of a target M using the bed 80.

[0041] First, in step S101, the 3D sensor 20 is turned on and laser light is shone from the projector 21 onto patient M (S101). The laser light is shone as dots, with thousands of laser beams aligned simultaneously. The laser light is shone at regular intervals while acquiring point cloud data P of patient M, and the laser light shone as aligned dots toward patient M is received by the image sensor module 22 and acquired as point cloud data P in real time (S102).

[0042] Next, point cloud data P representing the background other than the bed 80 and patient M is removed from the point cloud data P acquired in step S102, and point cloud data P representing only the bed 80 and patient M is extracted (S103). As an example, as shown in Figure 7A, several tens of points are extracted from the point cloud data P acquired for each frame in the direction of the X and Y axes, and the variance σ1 in the Z axis direction, which is orthogonal to the X and Y axes, is calculated. If the calculated variance σ1 in the Z axis direction is greater than a predetermined value, it can be estimated that there is movement, i.e., that patient M has been detected, and point cloud data P representing patient M can be extracted (see Figure 7B), but this method is not limited to this. In addition, the area of ​​the bed 80 is calculated from the size of the bed 80 which has been registered in advance, and point cloud data representing the bed 80 and the position of the bed 80 is extracted (S103).

[0043] In step S103, noise is removed from the point cloud data P from which the point cloud data P showing the background other than the bed 80 and the patient M has been removed, and the point cloud data P of the bed 80 and the patient M is extracted (S104). As schematically shown in FIG. 8, the variance σ2 at the neighboring points of the point cloud data that deviates from within a virtual sphere centered on an arbitrary point Pn of the acquired point cloud data P and having a predetermined threshold value as the radius is calculated. If the variance σ2 is greater than a predetermined value, it is determined as noise and removed from the point cloud data P. By repeatedly executing this process for each point of the point cloud data P, noise is removed from the point cloud data P, and the point cloud data P of the bed 80 and the patient M is extracted (S104). Further, noise may be detected by calculating a statistical outlier in which the average distance from an arbitrary point Pn of the acquired point cloud data P to the neighboring points is far from the average of the entire point cloud data, and removed as noise from the acquired point cloud data P, and this process may be additionally performed. Note that if the point cloud data P of the bed 80 and the patient M can be identified with a certain level of reliability as noise, the noise removal process in step S104 may be skipped.

[0044] Then, in step S105, the three-dimensional posture of the patient M is estimated from the point cloud data P only in the region where the bed 80 and the patient M exist (S105). Here, FIG. 9 shows an example of calculating the three-dimensional coordinate position on the surface of the patient M from the point cloud data P by the triangulation method. The distance D between an arbitrary point Pn of the point cloud data P in FIG. 9 and the three-dimensional sensor 20 is the distance A between the projector 21 and the image sensor module 22, the angle α1 formed between the distance measurement light L1 and the central axis c1 of the projector 21, and the central axis c2 of the image sensor module 22 and the reflected light R1 received by the image sensor module 22. As the angle α2 formed therebetween, Equation 1: D * tan α1 + D * tan α2 = A (Equation 1) It is expressed by, and it is possible to calculate the distance from the projector 21 to the patient M on which the distance measurement light L1 is projected, that is, the three-dimensional coordinate position on the surface of the patient M. For each point of the point cloud data P of the bed 70 and the patient M, the three-dimensional coordinate position can be calculated from the above relational expression.

[0045] The three-dimensional coordinate positions of the point cloud data P calculated in this way represent the state of patient M relative to the bed 80, and the actions of patient M can be estimated from the continuous changes in the three-dimensional coordinate positions of each point cloud data P. For example, using a learning model trained on training data annotated with "touching the body with hands" as a continuous change (video) of point cloud data P, it can be estimated that actions such as touching the arm, touching the neck, and touching the shoulder occurred in patient M.

[0046] Next, the specific motion data storage unit 10 determines whether the patient M's state and actions, as estimated by the motion estimation unit 40, are specific actions designated by the user, based on the specific motion data stored in the unit (step S106). Specifically, it determines whether the patient M is lying in bed 80 (supine or lateral position), sitting on the bed (sitting on the edge), touching their arm, getting up from bed (getting out of bed), touching their arm, touching their neck, touching their shoulder, or standing. Whether an action is a specific action is determined by whether the difference between the stored specific motion data and the three-dimensional coordinate position of the action estimated by the motion estimation unit 40 is smaller than a predetermined threshold.

[0047] In step S106, if it is determined that a specific action has occurred in patient M during imaging (step S106; Yes), the notification unit 60 issues an alert indicating that a specific action has occurred (step S107). Specifically, it displays the patient M's state and actions as point cloud data P in real time on the display, and also provides notification by displaying on the display that it is a specified action, turning on or flashing a lamp, or emitting sound through a speaker (S107).

[0048] When the notification unit 60 notifies that a specific operation has occurred, in step S108, the point cloud data P acquired by the three-dimensional sensor 20 is recorded (step S108). The point cloud data P to be recorded is recorded as data in formats such as CSV (Comma Separated Value), JSON (JavaScript Object Notation), and XML (Extensible Markup Language).

[0049] As described above, the motion recognition system 1 according to the present embodiment includes a specific motion data storage unit 10 that stores specific motion data indicating a specific motion including a series of predefined postures and motions of at least one body part of a target person, a three-dimensional sensor 20 that continuously acquires point cloud data P indicating three-dimensional coordinate positions on the surface of the target person and detects the motion of the person from the point cloud data P, a display unit 30 that continuously displays the motion of the person detected by the three-dimensional sensor 20 as the point cloud data P, a motion estimation unit 40 that estimates the motion of the target person from the continuous change of the point cloud data P, an input reception unit 50 that receives the designation of a specific motion among a plurality of motions of the target person, and a notification unit 60 that notifies that the specific motion has occurred when the motion of the target person is a motion that matches the designated specific motion, and a recording unit 70.

[0050] According to this configuration, various three-dimensional motions of the patient M can be confirmed in real time by the point cloud data P having three-dimensional coordinate positions. In particular, each point of the point cloud data P records the three-dimensional coordinate position, and it is possible to recognize the motion of a body part that is in close contact with the body and difficult to see. Further, according to this configuration, the motion of the patient M displayed on the display unit 30 is displayed as the point cloud data P having three-dimensional coordinate positions, so that the risk of acquiring information for identifying an individual is lower than that of a camera image (RGB color image) using a web camera, and the display takes privacy protection into consideration.

[0051] In this embodiment, an example of detecting the state and specific actions of patient M using the bed 70 has been described. However, the motion recognition system 1 according to this embodiment can be suitably installed in hospitals, nursing homes, elderly care facilities, and residential homes, etc., depending on the type of object for which privacy protection is required, and can recognize and analyze the state and actions of a person who needs to be monitored, for example.

[0052] 1... Motion recognition system 10... Specific motion data storage unit 20... 3D sensor 21... Projector, 211... VCSEL, 212... Diffractive optical element 22... Image sensor module 23... Processing unit 30... Display unit 50... Input reception unit 60... Notification unit 70... Recording unit 80... Bed M... Subject (patient)

Claims

1. An action recognition system for recognizing the actions of an object, comprising: storage means for storing specific action data that indicates a specific action including a predetermined series of postures and actions of at least one body part of the object; detection means for continuously acquiring point cloud data indicating three-dimensional coordinate positions on the surface of the object and detecting the actions of the object from the point cloud data; display means for continuously displaying the actions of the object detected by the detection means as a moving image from the point cloud data; estimation means for estimating the actions of the object from the continuous changes in the point cloud data; receiving means for receiving the designation of a specific action from among a plurality of actions of the object; and notification means for notifying that a specific action has occurred when the action of the object matches the designated specific action.

2. The motion recognition system according to claim 1, characterized in that the detection means projects invisible light from an array light source consisting of a plurality of light-emitting elements via a diffractive optical element and receives the reflected light reflected from the surface of the target to acquire the point cloud data.

3. The motion recognition system according to claim 2, characterized in that the estimation means estimates the motion of the target using a learning model that has been trained using training data in which annotations indicating the motion of the target have been added to the acquired point cloud data.

4. The motion recognition system according to any one of claims 1 to 3, characterized in that the specified motion is a body motion including the angle and width of the movement of the body part of the target.

5. The motion recognition system according to claim 1, characterized in that, when notifying that the specific motion has occurred, the point cloud data acquired by the detection means is recorded.

6. An action recognition method for recognizing the actions of an object, comprising: a storage step of storing specific action data that indicates a specific action including a predetermined series of postures and actions of at least one body part of the object; a detection step of continuously acquiring point cloud data indicating three-dimensional coordinate positions on the surface of the object and detecting the actions of the object from the point cloud data; a display step of continuously displaying the actions of the object detected in the detection step as a moving image using the point cloud data; an estimation step of estimating the actions of the object from the continuous changes in the point cloud data; a reception step of receiving a designation of a specific action from among a plurality of actions of the object; and a notification step of notifying that a specific action has occurred when the action of the object matches the designated specific action.

7. A program characterized by causing a computer to execute: a storage step of storing specific action data that includes a predetermined series of postures and movements of at least one body part of an object; a detection step of continuously acquiring point cloud data indicating three-dimensional coordinate positions on the surface of the object and detecting the movement of the object from the point cloud data; a display step of continuously displaying the movement of the object detected in the detection step as a moving image using the point cloud data; an estimation step of estimating the specific action of the object from the continuous changes in the point cloud data; a reception step of receiving a designation of the specific action from among a plurality of movements of the object; and a notification step of notifying that the specific action has occurred when the movement of the object matches the designated specific action.

Images