Motion recognition device, motion recognition method, and motion recognition program

Abstract

To provide a motion recognition device capable of handling gesture recognition in various situations.　A motion recognition device 10 according to the present invention includes: a setting unit that sets a detection area 200 within an imaging range 100 of a camera 10a on the basis of a user operation; a recognition unit that analyzes an image captured by the camera 10a and recognizes a prescribed motion for a person within the detection area 200; and an output control unit that, when the recognition unit recognizes the prescribed motion, performs control to output an output signal corresponding to the prescribed motion.

Description

Motion recognition device, motion recognition method, and motion recognition program

[0001] The present invention relates to an action recognition device, an action recognition method, and an action recognition program.

[0002] Gesture recognition technology that recognizes human gestures has recently been attracting attention from the perspective of improving user convenience when using various electronic devices.

[0003] In this regard, Patent Document 1 below discloses a gesture detection device that detects hand gestures made by a vehicle occupant and inputs operations to various systems within the vehicle. The gesture detection device disclosed in Patent Document 1 detects the physique of the occupant using a camera and sets a region for detecting hand gestures at a position corresponding to the physique of the occupant. With this configuration, the hand gestures of the occupant can be reliably detected regardless of the physique of the occupant.

[0004] Japanese Patent Application Laid-Open No. 2021-194982

[0005] However, the above-described gesture detection device detects hand gestures made by a vehicle occupant, and the area in which a person makes gestures is limited to the inside of the vehicle.

[0006] For example, in production sites such as factories, there are various situations in which the application of gesture recognition technology is considered effective, such as situations where a worker working on an assembly line wants to call a manager without leaving the site, or situations where a worker wants to operate a terminal but cannot touch it because their hands are dirty.

[0007] Therefore, it is desirable for production sites to introduce highly flexible gesture recognition technology that can adapt to various situations.

[0008] The present invention has been made in view of the above-mentioned problems, and therefore, an object of the present invention is to provide a gesture recognition device, a gesture recognition method, and a gesture recognition program that are capable of recognizing gestures in various situations.

[0009] The above object of the present invention can be achieved by the following means.

[0010] (1) An action recognition device having: a setting unit that sets a detection area within a camera's imaging range based on user operation; a recognition unit that analyzes the image captured by the camera and recognizes a predetermined action of a person within the detection area; and an output control unit that controls the output of an output signal corresponding to the action when the predetermined action is recognized by the recognition unit.

[0011] (2) The action recognition device according to (1), wherein the recognition unit recognizes the predetermined action by estimating a person's skeleton from the captured image.

[0012] (3) The action recognition device described in (2) above, further comprising a determination unit that, when multiple people are present within the detection area, determines one person from the multiple people to be the target of recognition of the specified action based on the distance between specified joint points.

[0013] (4) The action recognition device according to (1) or (2) above, further comprising an attribute recognition unit that recognizes attribute information of a person within the detection area.

[0014] (5) The action recognition device according to (4), wherein the output control unit performs control to output different output signals depending on the attribute information.

[0015] (6) The action recognition device described in (4) above, further comprising a determination unit that, when multiple people are present within the detection area, determines one person from the multiple people to be the target of recognition of the specified action based on attribute information of the multiple people.

[0016] (7) The action recognition device according to (4) above, wherein the attribute information includes information about the color of the person's clothing, or information about the color of a hat or helmet covering the person's head.

[0017] (8) The motion recognition device according to (1) or (2), wherein the output signal includes an email.

[0018] (9) The motion recognition device according to (1) or (2) above, wherein the output signal includes a signal for driving another device or a signal for stopping the driving of another device.

[0019] (10) The action recognition device according to (1) or (2) above, wherein the output signal includes a signal for displaying predetermined information on a display unit of a terminal device.

[0020] (11) The specified information includes a web page, and when the recognition unit recognizes another specified action, the output control unit further controls to output a signal for scrolling the web page.

[0021] (12) The action recognition device described in (10) above, wherein the specified information includes a table having a plurality of cells, and when another specified action is recognized by the recognition unit, the output control unit further controls to output a signal to move a selected cell within the table.

[0022] (13) The action recognition device according to (1) or (2), wherein the detection area includes a plurality of detection areas that are different from each other.

[0023] (14) The recognition unit recognizes the specified action for each person in the multiple detection areas, and when the specified action is recognized for each person in the multiple detection areas, the output control unit controls the output of the output signal.

[0024] (15) The action recognition device described in (1) or (2) above, wherein the specified action includes a person raising their hand, a person moving their hand, or a person's action such that both of their legs are included in the detection area.

[0025] (16) A method for recognizing an action, comprising: a step (a) of setting a detection area within an imaging range of a camera based on a user operation; a step (b) of analyzing an image captured by the camera to recognize a predetermined action of a person within the detection area; and a step (c) of controlling output of an output signal corresponding to the predetermined action when the predetermined action is recognized in the step (b).

[0026] (17) A motion recognition program that causes a computer to execute the following steps: (a) setting a detection area within a camera's imaging range based on user operation; (b) analyzing the image captured by the camera to recognize a predetermined motion of a person within the detection area; and (c) controlling the output of an output signal corresponding to the predetermined motion if the predetermined motion is recognized in (b).

[0027] According to the present invention, gesture recognition (movement recognition) can be performed in a variety of situations.

[0028] Advantages and features provided by one or more embodiments of the present invention will be more fully understood from the following detailed description and the accompanying drawings, which are for illustrative purposes only and are not intended to define limitations of the present invention.

[0023] Fig. 1 is a diagram showing a schematic configuration of a production system. Fig. 2 is a block diagram showing a schematic configuration of an action recognition device. Fig. 3 is a diagram showing the contents stored in a memory unit of the action recognition device. Fig. 4 is a flowchart showing the procedure for detection area setting processing. Fig. 5 is a flowchart showing the procedure for action recognition processing. Fig. 6 is a diagram showing an example of an image of a detection area according to the first embodiment. Fig. 7 is a diagram showing another example of an image of a detection area according to the first embodiment. Fig. 8 is a diagram showing an example of an image of a detection area according to the second embodiment. Fig. 9 is a diagram showing an example of an image of a detection area according to the third embodiment. Fig. 10 is a diagram for explaining a detection area according to the fourth embodiment. Fig. 11 is a diagram showing an example of an image of a detection area according to the fifth embodiment. Fig. 11 is a diagram showing an example of an image of a detection area according to the sixth embodiment.

[0029] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings, but the scope of the present invention is not limited to the disclosed embodiments.

[0030] First Embodiment FIG. 1 is a diagram showing a schematic configuration of a production system 1 to which an action recognition device according to a first embodiment of the present invention is applied.

[0031] 1, the production system 1 includes an action recognition device 10, multiple terminal devices 20a and 20b, and a production device 30. The action recognition device 10, the terminal devices 20a and 20b, and the production device 30 are connected to each other so that they can communicate with each other via a network 40. The action recognition device 10 is connected to a camera 10a. The camera 10a is, for example, a color camera and has a rectangular imaging range (angle of view) 100.

[0032] The action recognition device 10 and terminal devices 20a and 20b are so-called PCs (Personal Computers). The action recognition device 10 recognizes the actions of a person in a detection area 200 set within an imaging range 100 of the camera 10a. The terminal device 20a is a management terminal device used, for example, by a manager of the production system 1. The terminal device 20b is an operation terminal device used, for example, by a worker in the production system 1. The production device 30 is, for example, an injection molding device that injection molds resin parts.

[0033] The types and number of devices included in the production system 1 are not limited to the example shown in Fig. 1. For example, the production system 1 may be provided with various production devices other than the production device 30. Furthermore, for example, mobile terminal devices such as smartphones and tablet terminals may be connected to the network 40 via wireless communication.

[0034] Fig. 2 is a block diagram showing a schematic configuration of the action recognition device 10. As shown in Fig. 2, the action recognition device 10 includes a control unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an input unit 15, which are interconnected by a bus.

[0035] The control unit 11 is configured with a CPU (Central Processing Unit) and memories such as RAM (Random Access Memory) and ROM (Read Only Memory), and controls the above-mentioned units and performs various arithmetic processing according to programs.

[0036] The storage unit 12 is configured by a hard disc drive (HDD) or a solid state drive (SSD), and stores various programs and various data.

[0037] The communication unit 13 is an interface for communicating with other devices via the network 40. The communication unit 13 receives an output signal from the camera 10a.

[0038] The display unit 14 is, for example, a liquid crystal display, and displays various information.

[0039] The input unit 15 includes a keyboard, a numeric keypad, a mouse, etc., and receives input of various instructions and information.

[0040] 3 is a diagram showing the contents stored in the storage unit 12 of the action recognition device 10. The storage unit 12 of the action recognition device 10 stores action information 120 that associates a person's actions (poses) with output signals (output actions) of the action recognition device 10.

[0041] Furthermore, the storage unit 12 of the action recognition device 10 stores programs corresponding to the setting unit 121, the recognition unit 122, and the output control unit 123. The setting unit 121 sets a detection area 200 within the imaging range 100 of the camera 10a based on a user operation. The recognition unit 122 analyzes the image captured by the camera 10a and recognizes a predetermined action of a person within the detection area 200. When a predetermined action is recognized by the recognition unit 122, the output control unit 123 performs control to output an output signal corresponding to the action. The functions of each of the above units are exerted by the control unit 11 executing the corresponding program.

[0042] The action recognition device 10 may include components other than those described above, or may not include some of the components described above. For example, the action recognition device 10 may include a built-in camera.

[0043] In the production system 1 configured as described above, when a worker performs a predetermined action in the detection area 200 set within the imaging range 100 of the camera 10a, the action is recognized by the action recognition device 10. An output signal corresponding to the predetermined action is then transmitted from the action recognition device 10 to another device, and various processes and controls corresponding to the predetermined action are performed. The operation of the action recognition device 10 according to this embodiment will be described below with reference to FIGS. 4 to 7.

[0044] First, the operation of the action recognition device 10 for setting the detection area 200 will be described with reference to Fig. 4. As described above, the detection area 200 is set within the imaging range 100 of the camera 10a.

[0045] 4 is a flowchart showing the procedure of the detection area setting process executed by the action recognition device 10. The process shown in the flowchart of FIG. 4 is executed by the control unit 11 in accordance with a program stored in the storage unit 12 of the action recognition device 10.

[0046] (Step S101) First, the control unit 11 acquires image data of an image captured by the camera 10a, and causes the display unit 14 to display an image of the imaging range 100 of the camera 10a.

[0047] (Step S102) Subsequently, the control unit 11 accepts a user operation to specify the detection area 200. In this embodiment, the user of the action recognition device 10 specifies a rectangular area corresponding to the detection area 200 within the image capture range 100 via the input unit 15 while referring to the image of the image capture range 100 displayed on the display unit 14 in the processing of step S101. The control unit 11 accepts the area specification operation input via the input unit 15 by the user of the action recognition device 10.

[0048] (Step S103) Then, control unit 11 sets detection area 200 within imaging range 100 and ends the process. More specifically, control unit 11 sets the rectangular area specified within imaging range 100 in the process of step S102 as detection area 200 and ends the process.

[0049] As described above, according to the processing of the flowchart shown in Fig. 4, the detection area 200 is set within the imaging range 100 of the camera 10a based on a user operation. With this configuration, the user of the action recognition device 10 can freely set the detection area 200 for detecting the worker's actions. Furthermore, the user of the action recognition device 10 can change the detection area 200 after setting it. In other words, a highly flexible action recognition device 10 is provided that can handle action recognition in various situations in the production system 1.

[0050] Next, the operation of the action recognition device 10 that recognizes the action of a worker within the detection area 200 will be described with reference to FIG.

[0051] 5 is a flowchart showing the procedure of the action recognition process executed by the action recognition device 10. The process shown in the flowchart in FIG. 5 is executed by the control unit 11 in accordance with a program stored in the storage unit 12 of the action recognition device 10.

[0052] (Step S201) First, the control unit 11 acquires image data of the image captured by the camera 10a and performs image analysis. More specifically, the control unit 11 acquires image data of the image captured by the camera 10a and performs skeleton estimation to estimate joint points on the image data of a portion corresponding to the detection area 200. If a person is present in the detection area 200, the joint points are estimated. On the other hand, if no person is present in the detection area 200, the joint points are not estimated. Note that a known skeleton estimation technique such as OpenPose is used for skeleton estimation, and therefore a detailed description of skeleton estimation will be omitted.

[0053] (Step S202) Subsequently, the control unit 11 determines whether or not a person (worker) is present within the detection area 200. More specifically, the control unit 11 determines whether or not a joint point has been estimated by the skeleton estimation performed in the processing of step S101, thereby determining whether or not a person is present within the detection area 200.

[0054] If it is determined that no person is present in the detection area 200 (step S202: NO), the control unit 11 returns to the process of step S201. As a result, the processes of steps S201 and S202 are repeated until it is determined that a person is present in the detection area 200.

[0055] On the other hand, when it is determined that a person is present in the detection area 200 (step S202: YES), the control unit 11 proceeds to the process of step S203.

[0056] (Step S203) When it is determined that a person is present within the detection area 200 (step S202: YES), the control unit 11 acquires image data of the image captured by the camera 10a and performs image analysis. More specifically, the control unit 11 acquires image data of the image captured by the camera 10a and performs skeletal estimation to estimate joint points for the image data of a portion corresponding to the detection area 200. As a result, the joint points of the person present within the detection area 200 are estimated. Note that the control unit 11 can perform image analysis (skeletal estimation) on image data of multiple temporally consecutive captured images.

[0057] (Step S204) Subsequently, the control unit 11 determines whether or not a predetermined movement is recognized for the person in the detection area 200. More specifically, the control unit 11 determines whether or not a predetermined movement stored as movement information 120 in the storage unit 12 is recognized as a movement of the person in the detection area 200, based on the result of the skeleton estimation performed in the processing of step S203.

[0058] When it is determined that the predetermined action is not recognized for the person in the detection area 200 (step S204: NO), the control unit 11 returns to the process of step S201. As a result, the processes of steps S201 to S204 are repeated until a person is present in the detection area 200 and the predetermined action is recognized.

[0059] On the other hand, when it is determined that a predetermined motion is recognized for a person within the detection area 200 (step S204: YES), the control unit proceeds to the process of step S205.

[0060] (Step S205) When it is determined that a predetermined motion is recognized for a person in the detection area 200 (step S204: YES), the control unit 11 controls to output an output signal corresponding to the motion, and ends the process. More specifically, the control unit 11 refers to the motion information 120 stored in the storage unit 12, controls to output an output signal corresponding to the motion recognized in the process of step S204, and ends the process.

[0061] As described above, according to the processing of the flowchart shown in FIG. 5 , when a predetermined motion is recognized for a worker within the detection area 200, an output signal corresponding to the recognized motion is output to the outside of the motion recognition device 10. In this embodiment, when a motion of the worker raising his right hand is recognized, an abnormality occurrence notification email is output as the output signal. On the other hand, when a motion of the worker raising his left hand is recognized, a signal for temporarily stopping the production equipment in the production system 1 is output as the output signal. The motion recognition processing according to this embodiment will be described in more detail below with reference to FIGS. 6 and 7 .

[0062] (1) Action of a Worker Raising His Right Hand FIG. 6 is a diagram showing an example of an image of the detection area 200. As shown in FIG. 6, the image of the detection area 200 includes an image of a worker 310 raising his right hand. In FIG. 6, multiple points arranged on the worker 310 are points estimated as joint points by skeletal estimation, and indicate the positions of the right ankle, left ankle, right knee, left knee, right hip, left hip, neck, right shoulder, left shoulder, right elbow, left elbow, right wrist, left wrist, right eye, left eye, right ear, left ear, and nose, respectively. Note that in this embodiment, the detection area 200 is set to a range that includes the work area of ​​the worker 310 so that a worker working is normally present within the detection area 200.

[0063] The action recognition device 10 recognizes the action of the worker 210 raising his / her right hand from, for example, the position of the right wrist (or right elbow) relative to the line connecting both shoulders of the worker 310, or the angle (or distance) between the line connecting both shoulders and the line connecting the right shoulder and right wrist. Furthermore, in this embodiment, from the standpoint of preventing erroneous recognition, the action recognition device 10 recognizes the action of the worker 310 raising his / her right hand as a predetermined action if the state in which the worker 310 keeps raising his / her right hand continues for a predetermined time (for example, 3 seconds) or more.

[0064] When the action recognition device 10 recognizes the action of raising the right hand of the worker 310, it sends an abnormality occurrence notification email to the management terminal device 20a or the manager's mobile terminal device (not shown). With this configuration, when an abnormality occurs in the production system 1, the worker 310 can notify the manager that an abnormality has occurred in the production system 1 without leaving his / her work area.

[0065] (2) Action of worker raising left hand Fig. 7 is a diagram showing another example of the image of the detection area 200. As shown in Fig. 7, the image of the detection area 200 includes an image of a worker 310 raising his left hand.

[0066] The action recognition device 10 recognizes the action of the worker 210 raising his / her left hand from, for example, the position of the left wrist (or left elbow) relative to the line connecting the shoulders of the worker 310, or the angle (or distance) between the line connecting the shoulders and the line connecting the left shoulder and left wrist. Furthermore, in this embodiment, from the standpoint of preventing erroneous recognition, the action recognition device 10 recognizes the action of the worker 310 raising his / her left hand as a predetermined action if the state in which the worker 310 keeps raising his / her left hand continues for a predetermined time (for example, 3 seconds) or more.

[0067] When the action recognition device 10 recognizes that the worker 310 is raising his left hand, it sends a line stop signal to the management terminal device 20a and a control device (not shown) that manages the entire production system. This temporarily stops the production equipment in the production system 1. With this configuration, if any abnormality occurs in the production system 1, the worker 310 can temporarily stop the production equipment in the production system 1 without leaving his or her work area.

[0068] In the above-described embodiment, the detection area 200 is set to a range that includes the worker's work area so that the worker normally exists within the detection area 200. However, unlike this embodiment, the detection area 200 may be set to a range that does not include the worker's work area. In this case, the worker moves from the work area into the detection area 200 as necessary.

[0069] Second Embodiment Next, a second embodiment of the present invention will be described with reference to Fig. 8. In this embodiment, when a predetermined operation is recognized, information is displayed on the display unit of the terminal device 20b. Note that, except for the fact that information is displayed on the display unit of the terminal device 20b, the configuration of the production system according to this embodiment is the same as the production system according to the first embodiment, and therefore a detailed description of the production system will be omitted.

[0070] Fig. 8 is a diagram showing an example of an image of the detection area 200. As shown in Fig. 8, the image of the detection area 200 includes an image of the worker 320. In this embodiment, the detection area 200 is set to an area that does not include the work area of ​​the worker 320. Therefore, the worker 320 is not normally present within the detection area 200, and the worker 320 moves from the work area into the detection area 200 as necessary.

[0071] The action recognition device 10 performs skeletal estimation to determine whether or not a worker 320 is present in the detection area 200. If it is determined that a worker 320 is present in the detection area 200, the action recognition device 10 then determines whether or not both legs of the worker (joint points of both ankles and both knees) are recognized within the detection area 200.

[0072] When it is determined that both legs of the worker 320 are recognized within the detection area 200, the action recognition device 10 outputs a signal to the work terminal device 20b to display a predetermined webpage. The terminal device 20b, which has received the signal from the action recognition device 10, displays the predetermined webpage on the display unit of the terminal device 20b. With this configuration, the worker 320 can display the predetermined webpage on the display unit of the terminal device 20b and view the webpage without directly touching the terminal device 20b. For example, even if the worker cannot touch the terminal device 20b because his or her hands are dirty, the worker can display the webpage on the display unit of the terminal device 20b and view the webpage.

[0073] Furthermore, in this embodiment, the web page is scrolled by the action of the worker 320. Specifically, when the action recognition device 10 recognizes the action of the worker 320 moving his / her right arm (right wrist) upward, it outputs a signal to scroll the web page upward. On the other hand, when the action recognition device 10 recognizes the action of the worker 320 moving his / her right arm (right wrist) downward, it outputs a signal to scroll the web page downward. With this configuration, the worker 320 can scroll the web page displayed on the display unit of the terminal device 20b without directly touching the terminal device 20b.

[0074] (Modification) Instead of a web page, a table may be displayed on the display unit of terminal device 20b. For example, as shown in FIG. 8 , when it is determined that both legs of worker 320 are recognized within detection area 200, action recognition device 10 outputs a signal to cause terminal device 20b to display a predetermined table. Upon receiving the signal from action recognition device 10, terminal device 20b displays the predetermined table on the display unit of terminal device 20b. With this configuration, worker 320 can display the predetermined table on the display unit of terminal device 20b and view the table without directly touching terminal device 20b.

[0075] When the action recognition device 10 recognizes that the worker 320 moves his / her right arm (right wrist) upward, it outputs a signal to move the selected cell in the table upward. On the other hand, when the action recognition device 10 recognizes that the worker 320 moves his / her right arm (right wrist) downward, it outputs a signal to move the selected cell in the table downward.

[0076] Furthermore, if the action recognition device 10 recognizes that the worker 320 spreads his right arm (right wrist, right elbow) to the side, it outputs a signal to move the selected cell in the table to the right. On the other hand, if the action recognition device 10 recognizes that the worker 320 spreads his left arm (left wrist, left elbow) to the side, it outputs a signal to move the selected cell in the table to the left.

[0077] With this configuration, the worker 320 can move a selected cell in a desired direction in the table displayed on the display unit of the terminal device 20b without directly touching the terminal device 20b.

[0078] Third Embodiment Next, a third embodiment of the present invention will be described with reference to FIG. 9 . In this embodiment, a plurality of workers are present within the detection area 200. In this embodiment, the motion of the worker who is closest to the camera 10a among the plurality of workers is preferentially recognized. Note that the storage unit 12 of the motion recognition device 10 according to this embodiment stores a program for determining one person from among the plurality of people as a target for motion recognition.

[0079] 9 is a diagram showing an example of an image of the detection area 200 according to this embodiment. As shown in FIG. 9, the image of the detection area 200 includes images of three workers 331, 332, and 333.

[0080] The action recognition device 10 first calculates the distances L1, L2, and L3 from the eye position to the ankle position for each of the three workers 331, 332, and 333. Then, the action recognition device 10 determines the worker 331, who has the longest distance from the eye position to the ankle position, as the worker whose action is to be recognized. With this configuration, even when multiple workers are present within the detection area 200, the action of one worker can be appropriately recognized.

[0081] Note that various distances between joint points may be used as the distance between joint points used to determine the worker whose action is to be recognized. Furthermore, the worker whose action is to be recognized is not limited to the worker whose predetermined joint points are the longest distance apart, and for example, the worker whose predetermined joint points are the second longest distance apart may be determined as the worker whose action is to be recognized.

[0082] Fourth Embodiment Next, a fourth embodiment of the present invention will be described with reference to Fig. 10. In this embodiment, a plurality of detection areas 200 are set. As shown in Fig. 10, in this embodiment, a first detection area 200a and a second detection area 200b are set within the imaging range of the camera 10a.

[0083] The action recognition device 10 recognizes an action of worker 341 in the first detection area 200a raising his right hand and an action of worker 342 in the second detection area 200b raising his left hand. If these actions continue for a predetermined time (e.g., 3 seconds) or more, the action recognition device 10 outputs a signal to the outside to activate the power supply of the production device 30. This starts up the production device 30.

[0084] With this configuration, the worker can start up the production device 30 without directly touching the production device 30. Furthermore, since a signal is output based on the actions of the two workers 341 and 342, work can be performed with greater safety.

[0085] In the above-described embodiment, two detection areas are set within the imaging range of one camera. However, unlike the present embodiment, two cameras may be connected to the action recognition device 10, and detection areas may be set within the imaging ranges of the two cameras. Furthermore, the number of detection areas set is not limited to two, and three or more detection areas may be set within the imaging ranges of one or two or more cameras.

[0086] Fifth Embodiment Next, a fifth embodiment of the present invention will be described with reference to FIG. 11 . In this embodiment, in addition to the motion of a worker, attribute information of the worker is recognized. In this embodiment, the color of the worker's clothing is recognized as the attribute information of the worker. Note that the storage unit 12 of the motion recognition device 10 according to this embodiment stores a program for recognizing the attribute information of a person. The program for recognizing the attribute information includes a learning model capable of distinguishing the color of the clothing and hat.

[0087] 11 is a diagram showing an example of an image of the detection area 200 according to this embodiment. As shown in Fig. 11, the image of the detection area 200 includes an image of a worker 351 wearing a white coat and an image of a worker 352 wearing blue work clothes.

[0088] The action recognition device 10 first performs image analysis to determine the color of the clothing of the workers 351 and 352. When the action recognition device 10 recognizes the action of worker 351, who is wearing a white coat, raising his right hand, it outputs a signal to cause the display unit of terminal device 20b to display a web page. On the other hand, when the action recognition device 10 recognizes the action of worker 352, who is wearing blue work clothes, raising his right hand, it outputs a signal to cause the display unit of terminal device 20b to display a predetermined drawing (e.g., a drawing for the next process). This configuration allows the action of each worker to be properly recognized even when multiple workers are present within the detection area 200.

[0089] The action recognition device 10 of this embodiment can also determine whether the worker 352 is wearing a hat. If the worker 352 is not wearing a hat, the action recognition device 10 displays warning information on the display unit of the terminal device 20b to prompt the worker 352 to wear a hat.

[0090] Sixth Embodiment In this embodiment, one worker to be the subject of action recognition is selected from among a plurality of workers based on attribute information of the workers.

[0091] 12 is a diagram showing an example of an image of the detection area 200 according to this embodiment. As shown in FIG. 12, the image of the detection area 200 includes images of three workers 361, 362, and 363. The worker 361 is not wearing a hat or a helmet on his / her head. The worker 362 is wearing a blue hat. The worker 363 is wearing a white helmet.

[0092] The action recognition device 10 determines the color of the hat or helmet, and determines, for example, a worker 362 wearing a blue hat as the worker whose action is to be recognized. With this configuration, even when multiple workers are present within the detection area 200, the action of a single worker can be appropriately recognized.

[0093] The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims.

[0094] For example, in the above-described embodiment, the action of the worker raising his / her hands, the action of moving his / her hands up and down, and the action of including both legs within the detection area 200 are recognized as predetermined actions by the action recognition device 10. However, the actions recognized by the action recognition device 10 are not limited to these actions, and various actions can be applied. For example, various actions such as the action of the worker raising both hands, the action of the worker crossing his / her hands to form the letter "X," the action of the worker jumping, the action of the worker tilting his / her head, etc. can be applied.

[0095] In the above-described embodiment, the color of clothing and the color of a hat or helmet are used as attribute information of a worker. However, the attribute information of a worker is not limited to the color of clothing, etc., and, for example, the gender of the worker may be used. Alternatively, whether or not a helmet or hat is worn may be used as attribute information, regardless of the color of the helmet or hat. Various learning models may be used to recognize the attribute information.

[0096] In the above-described embodiment, the worker's movements are recognized by estimating the worker's skeleton. However, the method for recognizing the worker's movements is not limited to estimating the skeleton, and various image processing techniques may be used.

[0097] The means and methods for performing various processes in the action recognition device 10 according to the above-described embodiment can be realized by either a dedicated hardware circuit or a programmed computer. The above program may be provided by a computer-readable recording medium such as a USB (Universal Serial Bus) memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is typically transferred to and stored in a storage unit such as a HDD. The above program may also be provided as standalone application software, or may be incorporated into the software of the action recognition device 10 as a function of the device.

[0098] While embodiments of the present invention have been described and illustrated in detail, the disclosed embodiments are made for purposes of illustration and example only and are not intended to be limiting, and the scope of the present invention should be construed by the language of the appended claims.

[0099] This application is based on a Japanese patent application (Patent Application No. 2024-086892) filed on May 29, 2024, the disclosure of which is incorporated herein by reference in its entirety.

[0100] REFERENCE SIGNS LIST 1 Production system, 10 Action recognition device, 10a Camera, 11 Control unit, 12 Memory unit, 13 Communication unit, 14 Display unit, 15 Input unit, 20a, 20b Terminal device, 30 Production device, 40 Network, 100 Imaging range, 200 Detection area

Claims

1. A motion recognition device having: a setting unit that sets a detection area within a camera's imaging range based on user operation; a recognition unit that analyzes the image captured by the camera and recognizes a predetermined motion of a person within the detection area; and an output control unit that, when the predetermined motion is recognized by the recognition unit, controls the output of an output signal corresponding to the motion.

2. The action recognition device according to claim 1, wherein the recognition unit recognizes the predetermined action by estimating a person's skeleton from the captured image.

3. The action recognition device according to claim 2, further comprising a determination unit that, when multiple people are present within the detection area, determines one person from the multiple people to be the target of recognition of the specified action based on the distance between specified joint points.

4. The action recognition device according to claim 1 or 2, further comprising an attribute recognition unit that recognizes attribute information of a person within the detection area.

5. The action recognition device according to claim 4, wherein the output control unit performs control to output different output signals depending on the attribute information.

6. The action recognition device according to claim 4, further comprising a determination unit that, when multiple people are present within the detection area, determines one person from among the multiple people to be the target of recognition of the specified action based on attribute information of the multiple people.

7. The action recognition device according to claim 4, wherein the attribute information includes information about the color of clothing worn by the person, or information about the color of a hat or helmet covering the person's head.

8. The action recognition device according to claim 1 or 2, wherein the output signal includes an email.

9. The action recognition device according to claim 1 or 2, wherein the output signal includes a signal for driving another device or a signal for stopping the driving of another device.

10. The action recognition device according to claim 1 or 2, wherein the output signal includes a signal for displaying predetermined information on a display unit of a terminal device.

11. The action recognition device described in claim 10, wherein the specified information includes a web page, and when another specified action is recognized by the recognition unit, the output control unit further controls to output a signal for scrolling the web page.

12. The action recognition device according to claim 10, wherein the predetermined information includes a table having a plurality of cells, and when another predetermined action is recognized by the recognition unit, the output control unit further controls to output a signal for moving a selected cell within the table.

13. The action recognition device according to claim 1 or 2, wherein the detection area includes a plurality of detection areas that are different from each other.

14. The action recognition device described in claim 13, wherein the recognition unit recognizes the predetermined action for each person within the plurality of detection areas, and when the predetermined action is recognized for each person within the plurality of detection areas, the output control unit controls to output the output signal.

15. The action recognition device according to claim 1 or 2, wherein the predetermined action includes an action of a person raising a hand, an action of a person moving a hand, or an action of the person in such a way that both of the person's legs are included in the detection area.

16. A method for recognizing motion, comprising: a step (a) of setting a detection area within a camera's imaging range based on a user operation; a step (b) of analyzing the image captured by the camera and recognizing a predetermined motion of a person within the detection area; and a step (c) of controlling output of an output signal corresponding to the predetermined motion when the predetermined motion is recognized in step (b).

17. A motion recognition program that causes a computer to execute the following steps: (a) setting a detection area within a camera's imaging range based on user operation; (b) analyzing the image captured by the camera and recognizing a predetermined motion of a person within the detection area; and (c) controlling the output of an output signal corresponding to the predetermined motion if the predetermined motion is recognized in (b).

Images