Analytical systems, analytical programs, and analytical methods

Abstract

This system provides an analysis method that can easily improve the accuracy of analysis on subjects, regardless of their physical size. [Solution] An analysis system comprising: an image acquisition unit that acquires images of a predetermined monitoring area in which a person to be analyzed is located; a body size-related information acquisition unit that acquires body size-related information related to the physique of the person to be analyzed; an analysis target area determination unit that determines an analysis target area in the image according to the body size-related information; and an analysis unit that analyzes the person to be analyzed in the analysis target area.

Description

Technical Field

[0006] , , , , ,

[0001] The present invention relates to an analysis system, an analysis program, and an analysis method.

Background Art

[0002] In recent years, methods for analyzing human work operations by image analysis have been studied.

[0003] The following Patent Document 1 discloses the following prior art. By comparing an acquired image including a subject photographed in a laboratory with a background image by background difference, a subject area of the subject and an area of a roller held by the subject are extracted. Then, by obtaining and analyzing the relative movement of the roller with respect to each body part of the subject area by optical flow, it is determined whether or not the rolling is appropriately performed.

[0004] The following Patent Document 2 discloses the following prior art. Image information of a predetermined area is acquired as area information. Based on the area information and detection information stored in a storage unit and including motion information when a person fell down in the past, a dangerous situation in the predetermined area is detected. Then, when a dangerous situation is detected, that fact is notified.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, the prior art disclosed in Patent Document 1 detects the area of ​​a person and the area of ​​a roller by background subtraction, so the detection accuracy of the area of ​​a person may decrease due to occlusion of multiple people or reflections of objects, and the accuracy of determining whether the roller is being used may decrease. The technology disclosed in Patent Document 2 has the problem that the accuracy of detecting dangerous situations may decrease depending on the size of the person.

[0007] This invention was made to solve these problems. Specifically, it aims to provide an analysis system, an analysis program, and an analysis method that can easily improve the accuracy of analysis of an analyzer, regardless of the analyzer's physique. [Means for solving the problem]

[0008] The above-mentioned problems of the present invention are solved by the following means.

[0009] (1) An analysis system comprising: an image acquisition unit that acquires an image of a predetermined monitoring area in which a person to be analyzed is located; a body size-related information acquisition unit that acquires body size-related information related to the physique of the person to be analyzed; an analysis target area determination unit that determines an analysis target area in the image acquired by the image acquisition unit according to the body size-related information acquired by the body size-related information acquisition unit; and an analysis unit that analyzes the person to be analyzed in the analysis target area determined by the analysis target area determination unit.

[0010] (2) The analysis system described in (1) above, wherein the body size-related information acquisition unit acquires skeletal information as body size-related information based on the image acquired by the image acquisition unit.

[0011] (3) The analysis system according to (1) above, further comprising an existence determination unit that determines whether the person to be analyzed is present in the monitoring area based on the image acquired by the image acquisition unit, and the analysis unit performs an analysis on the person to be analyzed when the existence determination unit determines that the person to be analyzed is present.

[0012] (4) The analysis system according to (1) above, wherein the analysis target area determination unit determines the analysis target area for each body part of the person to be analyzed based on the body size-related information.

[0013] (5) The analysis system according to (1) above, wherein the analysis target area determination unit determines the analysis target area obtained by adjusting the size and position of the analysis target area with respect to reference values ​​for size and position of the analysis target area based on the body size-related information.

[0014] (6) The analysis system according to (2) above, wherein the analysis target area determination unit determines the analysis target area based on the average or median value of multiple frames of the skeletal information of the person to be analyzed, which is obtained as body size-related information from the image acquired by the image acquisition unit.

[0015] (7) The analysis system described in (5) above, wherein the analysis target area determination unit adjusts the reference value based on the user's instructions.

[0016] (8) The analysis system according to (1) above, further comprising an object detection unit for detecting an object to be analyzed in the image acquired by the image acquisition unit, wherein the analysis unit, when an object to be analyzed is detected within the area to be analyzed, stores the information of the area to be analyzed, the information of the object to be analyzed, and a timestamp in a storage unit in association with each other.

[0017] (9) The analysis system according to (8) above, wherein the analysis unit stores information of the object to be analyzed in the storage unit as relative coordinates with respect to the area to be analyzed.

[0018] (10) The analysis system according to (8) above, wherein the object detection unit detects a plurality of objects to be analyzed within the area to be analyzed.

[0019] (11) The analysis unit determines whether the object to be analyzed is detected within the analysis target area, and when it is determined by the analysis unit that the object to be analyzed is detected within the analysis target area, the analysis system according to (8) above further includes a notification unit that notifies a warning.

[0020] (12) The analysis unit calculates a feature quantity for analyzing the operation of the person to be analyzed for each analysis target area using the data stored in the storage unit, and analyzes the operation of the person to be analyzed based on the calculated feature quantity. The analysis system according to (8) above.

[0021] (13) The information of the object to be analyzed is the coordinates of the object to be analyzed, and the feature quantity is the area of the convex hull of the coordinates of the object to be analyzed within the analysis target area, the trajectory length of the coordinates of the object to be analyzed within the analysis target area, or the time during which the coordinates of the object to be analyzed are included within the analysis target area, and is calculated for each analysis target area. The analysis system according to (12) above.

[0022] (14) The analysis unit calculates the feature quantity by dividing the area of the convex hull of the coordinates of the object to be analyzed within the analysis target area, the trajectory length of the coordinates of the object to be analyzed within the analysis target area, or the time during which the coordinates of the object to be analyzed are included within the analysis target area by the area of the analysis target area. The analysis system according to (13) above.

[0023] (15) The analysis system according to (12) above further includes a display unit that visibly displays the feature quantity for each analysis target area.

[0024] (16) The object to be analyzed is a cleaning roller. The analysis system according to (8) above.

[0025] A analysis program for causing a computer to execute a process having: step (a) of obtaining an image of a predetermined monitoring area into which an analysis target person enters; step (b) of obtaining physical condition related information related to the physical condition of the analysis target person; step (c) of determining an analysis target area corresponding to the physical condition related information in the image obtained in step (a); and step (d) of analyzing the analysis target person in the analysis target area determined in step (c).

[0026] (18) A method executed by an analysis system, the method having: step (a) of obtaining an image of a predetermined monitoring area into which an analysis target person enters; step (b) of obtaining physical condition related information related to the physical condition of the analysis target person; step (c) of determining an analysis target area corresponding to the physical condition related information in the image obtained in step (a); and step (d) of analyzing the analysis target person in the analysis target area determined in step (c).

Advantages of the Invention

[0027] An image of a predetermined monitoring area and physical condition related information related to the physical condition of an analysis target person are obtained. In the image, an analysis target area corresponding to the physical condition related information is determined, and the analysis target person in the analysis target area is analyzed. Thereby, the analysis accuracy of the analysis target person can be easily improved regardless of the physical condition of the analysis target person.

Brief Description of the Drawings

[0028] Advantages and features provided by one or more embodiments of the present invention will be more fully understood from the following detailed description and the accompanying drawings. However, these are for illustrative purposes only and are not intended to limit the present invention. [Figure 1] It is a diagram showing a schematic configuration of an analysis system. [Figure 2] It is a block diagram showing a hardware configuration of an analysis device. [Figure 3] This is a schematic diagram showing the subjects of analysis within the monitoring area captured by the imaging device. [Figure 4] This is an explanatory diagram to show how the presence of an analysis target is determined within the monitoring area. [Figure 5] This is an explanatory diagram to show what happens when it is determined that there are no individuals to be analyzed within the monitoring area. [Figure 6] This diagram shows a bounding box surrounding a cleaning component. [Figure 7] This is an explanatory diagram showing the reference rectangles for the head area, upper body area, and lower body area. [Figure 8] This is an explanatory diagram to explain how the head area, which is the area to be analyzed, is calculated. [Figure 9] This is an explanatory diagram illustrating the method for calculating the upper body area, which is the area to be analyzed. [Figure 10] This is an explanatory diagram illustrating the method for calculating the lower body area, which is the area to be analyzed. [Figure 11] This figure shows an example of an area to be analyzed. [Figure 12] This is an explanatory diagram showing the roller data frame. [Figure 13] This is an explanatory diagram showing the convex hull of the coordinates of the cleaning rollers in a time series within the head area. [Figure 14] This is an explanatory diagram showing the trajectory length of the cleaning roller coordinates within the area being analyzed. [Figure 15] This figure shows a display screen where the features are clearly visible. [Figure 16] This is a flowchart showing the operation of the analysis system. [Figure 17] Figure 16 shows the subroutine flowchart for step S08 of the flowchart. [Figure 18] Figure 17 shows the subroutine flowchart for step S21 of the subroutine flowchart. [Figure 19] This figure shows a Roller data frame in which label transitions are recorded. [Figure 20] This figure shows the output of the Range for each label in the Raw of the Roller data frame after the label transition determination process. [Figure 21] This is an explanatory diagram showing the state in which the analyte has been detected within the analysis area. [Figure 22] This is a flowchart showing the operation of the analysis system. [Modes for carrying out the invention]

[0029] Hereinafter, an analysis system, an analysis program, and an analysis method according to embodiments of the present invention will be described with reference to the attached drawings. However, the scope of the present invention is not limited to the disclosed embodiments. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. Also, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from actual ratios.

[0030] (First Embodiment) Figure 1 is a diagram showing the schematic configuration of analysis system 1. Figure 2 is a block diagram showing the hardware configuration of analysis device 10.

[0031] The analysis system 1 includes an analyzer 10 and an imaging device 20. The analysis system 1 may also consist of the analyzer 10 alone.

[0032] The analyzer 10 and the imaging device 20 can be connected to each other in a way that allows them to communicate with one another.

[0033] The analysis device 10 includes a control unit 100, a storage unit 200, a display unit 300, an input unit 400, and a communication unit 500. These components are interconnected via a bus 600. The analysis device 10 is comprised of a computer. The analysis device 10 may be, for example, a PC (Personal Computer) or a server.

[0034] The control unit 100 is composed of a CPU (Central Processing Unit) and performs control and calculation processing on each part of the analysis system 1 according to the program. The functions of the control unit 100 will be described later.

[0035] The storage unit 200 may consist of RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The RAM temporarily stores programs and data as a working area for the control unit 100. The ROM stores various programs and data in advance. The flash memory stores various programs and data, including the operating system.

[0036] The display unit 300 is, for example, a liquid crystal display, which displays various information.

[0037] The input unit 400 is comprised of, for example, a touch panel and various keys. The input unit 400 is used for various operations and inputs.

[0038] The communication unit 500 is an interface for communicating with external devices. Communication standards such as Ethernet (registered trademark), USB, MIPI (Mobile Industry Processor Interface), IEEE 1394, Bluetooth (registered trademark), and IEEE 802.11 can be used for communication.

[0039] The imaging device 20 may be a camera that captures monochrome or color images. The imaging device 20 can generate two-dimensional images, which are still images, by decoding the captured images (moving images). This allows for a sequence of images (still images) in chronological order. The imaging device 20 is installed in a predetermined location and captures images of, for example, people and objects.

[0040] Figure 3 is a schematic diagram showing the subjects 90 being analyzed within the monitoring area 25 captured by the imaging device 20. The display unit 300 is also shown in Figure 3.

[0041] As shown in Figure 3, the imaging device 20 is fixed to a wall or the like to image a predetermined monitoring area 25. The optical axis of the imaging device 20 may be installed horizontally or slightly tilted downwards (within the range of 1 to 45 degrees). The imaging device 20 starts and stops imaging under the control of the control unit 100. The imaging device 20 outputs the captured color or black and white image. The imaging device 20 captures video, which is a video consisting of multiple time-sequential images (frames) at a frame rate of 10 to 60 fps. For the sake of simplicity, the following explanation will use the case where the imaging device 20 outputs time-series video data at 30 fps as an example.

[0042] The monitoring area 25 is, for example, an anteroom for a semiconductor manufacturing line, a precision equipment manufacturing line, or a food manufacturing line. Markers are placed on the floor of the monitoring area 25, and the person being analyzed 90 is instructed to enter the monitoring area 25 and stand in the position marked when cleaning their own body. The person being analyzed 90 may be the subject of analysis to determine whether they performed the cleaning work properly. The person being analyzed 90 is instructed to stand in front of the camera of the imaging device 20 or slightly to the side of it within the monitoring area 25. In addition, if the person being analyzed 90 is cleaning their backside, they are instructed to position themselves so that the cleaning position is facing the camera. A fixed camera is used, and since the position of the person being analyzed 90 remains the same, the angle of view and shooting distance (distance from the subject) are always constant.

[0043] The cleaning operation is the process in which the person being analyzed 90 cleans their own body using cleaning materials. Specifically, the person being analyzed 90 uses a cleaning roller 80 with an adhesive tape attached and a handle, and moves the cleaning roller 80 back and forth over their work clothes to remove foreign matter such as fibers, dust, and hair that is attached to the surface of their body. Following the manual, the person being analyzed 90 cleans their entire body from head to toe, in a sequential manner from top to bottom. The cleaning roller 80 is included in the object being analyzed 70. In the following description, the case in which the object being analyzed 70 is the cleaning roller 80 will be used as an example.

[0044] Referring again to Figure 1, the functions of the control unit 100 will be explained. By executing a program, the control unit 100 functions as an image acquisition unit 110, a skeletal information detection unit 120, an existence determination unit 130, an object detection unit 140, an analysis target area determination unit 150, an analysis unit 160, a display control unit 170, and a notification unit 180. The skeletal information detection unit 120 constitutes the body size-related information acquisition unit. The display control unit 170, together with the display unit 300, constitutes the display unit.

[0045] Alternatively, the image acquisition unit 110, presence determination unit 130, analysis target area determination unit 150, analysis unit 160, display control unit 170, and notification unit 180 may be configured by a single information processing device, while the skeletal information detection unit 120 and object detection unit 140 may be configured by other information processing devices. In this case, for example, the single information processing device may be configured as a terminal such as a PC, and the other information processing device may be configured as a server. The single information processing device and the other information processing device are connected to each other so as to be able to communicate with one another.

[0046] The image acquisition unit 110 can acquire images of the monitoring area 25 captured by the imaging device 20 by receiving them from the imaging device 20 via the communication unit 500. Alternatively, the image acquisition unit 110 may acquire images by reading images that have been previously captured by the imaging device 20 and stored in the storage unit 200.

[0047] The skeletal information detection unit 120 acquires skeletal information as body size-related information based on the image acquired by the image acquisition unit 110. Specifically, the skeletal information detection unit 120 acquires skeletal information as body size-related information by detecting it from the image. Body size-related information is information that reflects a person's body size. Hereinafter, unless otherwise specified, the case where body size-related information is skeletal information will be explained as an example. Skeletal information may be the coordinates of each joint point 121 (see Figure 4) on the image. Hereinafter, the coordinates of the joint point 121 will also be simply referred to as "joint point 121". The coordinates of the joint point 121 may be coordinates (x coordinate, y coordinate) on the image. The skeletal information detection unit 120 may detect the joint points 121 using known methods. For example, the skeletal information detection unit 120 may detect the joint points 121 using Deep Pose.

[0048] Articular point 121 may include articular points 121 of the "right eye," "left eye," "nose," "right shoulder," "left shoulder," "neck," "right elbow," "left elbow," "right wrist," "left wrist," "right hip," "left hip," "right knee," "left knee," "right ankle," and "left ankle." Hereafter, these articular points 121 will also simply be referred to as "right eye," "left eye," "nose," "right shoulder," "left shoulder," "neck," "right elbow," "left elbow," "right wrist," "left wrist," "right hip," "left hip," "right knee," "left knee," "right ankle," and "left ankle."

[0049] The presence determination unit 130 determines whether the person being analyzed 90 is present in the monitoring area 25 based on the image acquired by the image acquisition unit 110. The presence determination unit 130 can determine whether the person being analyzed 90 is present in the monitoring area 25 by checking whether the ankle joint point 121a is present within the monitoring area 25 in the image.

[0050] Figure 4 is an explanatory diagram illustrating the case in which it is determined that the subject of analysis 90 is present in the monitoring area 25. Figure 5 is an explanatory diagram illustrating the case in which it is determined that the subject of analysis 90 is not present in the monitoring area 25.

[0051] As shown in Figure 4, if the joint points 121a of the right and left ankles are present within the monitoring area 25 in the image, the presence determination unit 130 determines that the person being analyzed 90 is present in the monitoring area 25. As shown in Figure 5, if the joint points 121a of the right and left ankles are not present within the monitoring area 25 in the image, the presence determination unit 130 determines that the person being analyzed 90 is not present in the monitoring area 25.

[0052] The object detection unit 140 detects the cleaning roller 80, which is the object to be analyzed, in the image acquired by the image acquisition unit 110. The object detection unit 140 can detect the cleaning roller 80 using known methods. For example, the object detection unit 140 can detect the cleaning roller 80 using YOLO (You Only Look Once).

[0053] Figure 6 shows a bounding box 81 surrounding the cleaning roller 80.

[0054] The cleaning roller 80 may be detected as the coordinates of the center of gravity of the bounding box 81 surrounding the cleaning roller 80. The cleaning roller 80 may also be detected as the coordinates of each end of the cleaning roller 80.

[0055] The analysis target area determination unit 150 determines the analysis target area 190 (see Figure 8) in the image acquired by the image acquisition unit 110, according to the skeletal information detection unit 120 which has acquired the skeletal information related to body size. The analysis target area determination unit 150 may also determine the analysis target area 190 if the presence determination unit 130 determines that the person to be analyzed 90 is present in the monitoring area 25.

[0056] The analysis area determination unit 150 can determine the analysis area 190 for each body part of the subject 90 based on skeletal information. The analysis area determination unit 150 may also determine the analysis area 190 for each of three body parts: "head," "upper body," and "lower body."

[0057] The analysis target area determination unit 150 may determine the analysis target area 190 by adjusting the size and position relative to reference values ​​for the size and position of the analysis target area 190 based on the skeletal information. The analysis target area determination unit 150 may also determine the analysis target area 190 by adjusting the size, position, and angle relative to reference values ​​for the size, position, and angle of the analysis target area 190 based on the skeletal information.

[0058] The reference values ​​can be set by pre-storing them in the storage unit 200 as reference values ​​for size and position, or as reference values ​​for size, position, and angle.

[0059] The analysis target area determination unit 150 can determine the analysis target area 190, which consists of the head area 191, the upper body area 192, and the lower body area 193.

[0060] Figure 7 is an explanatory diagram showing the reference rectangles 190S for the head area 191, the upper body area 192, and the lower body area 193. The size, position, and angle of each reference rectangle 190S constitute the reference values ​​for the size, position, and angle of each area 190 under analysis. The size of the reference rectangle 190S may be indicated by the reference rectangle width and reference rectangle height. The reference rectangle 190S may be pre-set based, for example, on reference joint points 121S, which are the joint points when a person with a standard build is standing in a standard posture with their arms down and feet together. A person with a standard build may be a person with the median height in any group. A person with a standard build may be a virtual person with average height generated by a generative AI. The reference joint points 121S and the reference rectangle 190S may be data detected from an image of the person with the standard build.

[0061] As shown in Figure 7, the reference rectangle 191S of the head area 191 may be a rectangle enclosing the reference joint points 121S of the "right eye," "left eye," "nose," "right shoulder," "left shoulder," and "neck." The reference rectangle 192S of the upper body area 192 may be a rectangle enclosing the reference joint points 121S of the "right shoulder," "left shoulder," "neck," "right elbow," "left elbow," "right wrist," "left wrist," "right hip," and "left hip." The reference rectangle 193S of the lower body area 193 may be a rectangle enclosing the reference joint points 121S of the "right hip," "left hip," "right knee," "left knee," "right ankle," and "left ankle."

[0062] Figure 8 is an explanatory diagram illustrating the calculation method for the head area 191, which is the analysis target area 190.

[0063] The analysis target area determination unit 150 can determine the head area 191, which is the analysis target area 190, based on the reference rectangle 190S, as follows.

[0064] (1) The position of the reference rectangle 191S is adjusted so that the position of the center of the reference rectangle 191S coincides with the position of the joint point 121 of the "nose" detected by the skeletal information detection unit 120.

[0065] (2) The rectangular width of the head area 191, which is the analysis target area 190, is calculated by adjusting the reference rectangular width of the reference rectangular 191S whose position has been adjusted using the following formula.

[0066] The rectangular width of head area 191 = (shoulder width of subject 90 + eye width of subject 90) / (reference shoulder width + reference eye width) × reference rectangular width Here, the shoulder width of subject 90 is the distance between joint point 121 of the "right shoulder" and joint point 121 of the "left shoulder". The eye width of subject 90 is the distance between joint point 121 of the "right eye" and joint point 121 of the "left eye". Standard shoulder width is the distance between reference joint point 121S of the "right shoulder" and reference joint point 121S of the "left shoulder" for a person with a standard build. Reference eye width is the distance between reference joint point 121S of the "right eye" and reference joint point 121S of the "left eye" for a person with a standard build. The shoulder width of subject 90 is shown by a thick dashed line in Figure 8. The eye width of subject 90 is shown by a thick solid line in Figure 8.

[0067] (3) The height of the head area 191, which is the analysis target area 190, is calculated by adjusting the height of the reference rectangle 191S, whose position has been adjusted, using the following formula.

[0068] Rectangle height of head area 191 = (y-coordinate of "shoulder" of subject 90 - y-coordinate of "eye" of subject 90) / (y-coordinate of reference shoulder - y-coordinate of reference eye) × reference rectangle height Here, the y-coordinate of the standard shoulder is the average of the y-coordinates of the reference joint point 121S of the "right shoulder" and the "left shoulder" of a person with a standard build. The y-coordinate of the reference eye is the average of the y-coordinates of the reference joint point 121S of the "right eye" and the "left eye" of a person with a standard build.

[0069] (4) The tilt of the reference rectangle 190S is adjusted so that it is the average value of the tilt of the "eyes" and the tilt of the "shoulders" detected by the skeletal information detection unit 120.

[0070] Here, the tilt of the "eyes" is the slope of the line connecting the "right eye" and the "left eye." The tilt of the "shoulders" is the slope of the line connecting the "right shoulder" and the "left shoulder."

[0071] Note that the adjustment in (4) above is optional and does not need to be included in the calculation of head area 191.

[0072] Figure 9 is an explanatory diagram illustrating the calculation method for the upper body area 192, which is the area 190 to be analyzed.

[0073] The analysis target area determination unit 150 can determine the upper body area 192, which is the analysis target area 190, based on the reference rectangle 190S, as follows.

[0074] (1) The position of the reference rectangle 192S is adjusted so that the position of the center of the reference rectangle 192S coincides with the position of the center coordinate 122 of the joint point 121 of the upper body, which is detected by the skeletal information detection unit 120.

[0075] Here, the center coordinates 122 of the joint point 121 of the upper body are the coordinates of the geometric centers of the "right shoulder," "left shoulder," "neck," "right elbow," "left elbow," "right wrist," "left wrist," "right hip," and "left hip" that are included in the joint point 121 of the upper body.

[0076] (2) The rectangular width of the upper body area 192, which is the area to be analyzed 190, is calculated by adjusting the reference rectangular width of the reference rectangular 192S whose position has been adjusted using the following formula.

[0077] Upper body area 192 rectangle width = (Shoulder width of 90 subjects + Waist width of 90 subjects) / (Standard shoulder width + Standard waist width) × Standard rectangle width Here, the waist width of subject 90 is the distance between joint point 121 on the "right hip" and joint point 121 on the "left hip". The standard waist width is the distance between the standard joint point 121S on the "right hip" and the standard joint point 121S on the "left hip" of a person with a standard body type. The shoulder width of subject 90 is shown by a thick dashed line in Figure 9. The waist width of subject 90 is shown by a thick dotted line in Figure 9.

[0078] (3) The height of the upper body area 192, which is the area to be analyzed 190, is calculated by adjusting the height of the reference rectangle 192S, whose position has been adjusted, using the following formula.

[0079] The height of the rectangle in upper body area 192 = (y-coordinate of the waist of subject 90 - y-coordinate of the shoulder of subject 90) / (y-coordinate of the reference waist - y-coordinate of the reference shoulder) × reference rectangle height Here, the y-coordinate of the reference hip is the average of the y-coordinate of the reference joint point 121S of the "right hip" and the y-coordinate of the reference joint point 121S of the "left hip" for a person with a standard body type.

[0080] (4) The tilt of the reference rectangle 190S is adjusted so that it is the average value of the tilt of the "shoulder" and the tilt of the "waist" detected by the skeletal information detection unit 120.

[0081] Here, the "hip" tilt is the slope of the straight line connecting the "right hip" and the "left hip". Note that the adjustment in (4) above is optional and does not need to be included in the calculation of the upper body area 192.

[0082] Figure 10 is an explanatory diagram illustrating the calculation method for the lower body area 193, which is the area 190 to be analyzed.

[0083] The analysis target area determination unit 150 can determine the lower body area 193, which is the analysis target area 190, based on the reference rectangle 193S as follows.

[0084] (1) The position of the reference rectangle 193S is adjusted so that the position of the center of the reference rectangle 193S coincides with the position of the center coordinate 123 of the joint point 121 of the lower body, which is detected by the skeletal information detection unit 120.

[0085] Here, the center coordinates 123 of the joint point 121 of the lower body are the coordinates of the geometric centers of the "right hip," "left hip," "right knee," "left knee," "right ankle," and "left ankle" included in the joint point 121 of the lower body.

[0086] (2) The rectangular width of the lower body area 193, which is the area to be analyzed, is calculated by adjusting the reference rectangular width of the reference rectangular 193S whose position has been adjusted using the following formula.

[0087] The rectangle width for the lower body area 193 = (shoulder width of subject 90 + waist width of subject 90) / (standard shoulder width + standard waist width) × standard rectangle width (3) The height of the lower body area 193, which is the analysis target area 190, is calculated by adjusting the reference rectangle height of the reference rectangle 193S whose position has been adjusted using the following formula.

[0088] The height of the rectangle in lower body area 193 = (y-coordinate of the "waist" of subject 90 - y-coordinate of the "shoulder" of subject 90) / (y-coordinate of the reference waist - y-coordinate of the reference shoulder) × reference rectangle height (4) The tilt of the reference rectangle 193S is adjusted so that it is the average value of the tilt of the "shoulder" and the tilt of the "waist" detected by the skeletal information detection unit 120.

[0089] Note that the adjustment in (4) above is optional and does not need to be included in the calculation of the lower body area 193.

[0090] Figure 11 shows an example of the 190 areas to be analyzed.

[0091] In the example shown in Figure 11, the head area 191, upper body area 192, and lower body area 193 are determined by adjusting the position, width, height, and tilt of each reference rectangle 190S.

[0092] The analysis target area determination unit 150 may determine the analysis target area 190 based on the average or median of multiple frames of skeletal information of the subject 90, which is obtained as body size-related information from the images acquired by the image acquisition unit 110. For example, the analysis target area determination unit 150 may determine the analysis target area 190 calculated by the above calculation method using the average value of each joint point 121 over 5 to 10 frames.

[0093] The analysis area determination unit 150 may adjust the reference values ​​based on user instructions. For example, the analysis area determination unit 150 may set reference values ​​entered by the user in the input unit 400. In this case, the input screen of the input unit 400 may display default reference values ​​that can be changed by the user, or input fields where default reference values ​​have not been entered may be displayed so that the user can enter reference values. In addition, the input screen of the input unit 400 may display the reference rectangles 190S for the head area 191, upper body area 192, and lower body area 193 along with the image of the person being analyzed 90, so that the user can change their size (see Figure 7). In this case, the size and position of each reference rectangle 190S after the size has been changed by the user, or the size, position, and angle, may be set as reference values.

[0094] The analysis unit 160 analyzes the 90 individuals to be analyzed in the analysis area 190 determined by the analysis area determination unit 150. The analysis performed by the analysis unit 160 is described in detail below.

[0095] The analysis unit 160 detects the cleaning roller 80, which is the object of analysis 70, within the analysis area 190. The analysis unit 160 distinguishes which of the analysis area 190—head area 191, upper body area 192, or lower body area 193—the cleaning roller 80 was detected in. If the center of gravity of the bounding box 81 surrounding the cleaning roller 80 is included in any of the head area 191, upper body area 192, or lower body area 193, the analysis unit 160 determines that the analysis area 190 containing that center is the analysis area 190 where the cleaning roller 80 exists. If the analysis unit 160 detects the cleaning roller 80 within the analysis area 190, it associates the information of the analysis area 190, the information of the cleaning roller 80, and the timestamp, and stores them in the storage unit 200 as a roller data frame 161. Specifically, the analysis unit 160 stores the information of the cleaning roller 80, which is the coordinates of the cleaning roller 80, with a timestamp and a label indicating which analysis target area 190 the cleaning roller 80 is located in, as a roller data frame 161 in the storage unit 200. Here, the distinction of which analysis target area 190 the cleaning roller 80 is located in refers to whether the analysis target area 190 where the cleaning roller 80 is located is the head area 191, the upper body area 192, or the lower body area 193. The coordinates of the cleaning roller 80 may be the coordinates of the center of gravity of the bounding box 81 surrounding the cleaning roller 80. The coordinates of the cleaning roller 80 may also be the coordinates of an image, or they may be relative coordinates of the analysis target area 190 where the cleaning roller 80 is located.

[0096] Figure 12 is an explanatory diagram showing the roller data frame 161.

[0097] As shown in Figure 12, the roller data frame 161 associates information about the area under analysis 190, information about the cleaning roller 80, and a timestamp. Specifically, the head area 191, upper body area 192, and lower body area 193 within the area under analysis 190 where the cleaning roller 80 is located (labels), the coordinates of the cleaning roller 80, and the timestamp (time) are associated. For example, the association of this information indicates that the cleaning roller 80 was present at coordinates (362.3047, 64.25781) within the head area 191 at 0.033333 seconds. Also, for example, the association of this information indicates that the cleaning roller 80 was present at coordinates (448.4375, 225.5859) within the overlapping region of the head area 191 and upper body area 192 at 27.2 seconds. If the cleaning roller 80 is not detected in any of the head area 191, upper body area 192, or lower body area 193, the label is stored in the roller data frame 161, associated with a timestamp, for example, as "NaN".

[0098] As described later, the analysis unit 160 performs label transition determination processing on the roller data frame 161. This ensures that if, in the roller data frame 161, two analysis target areas 190 are associated with the coordinates of one cleaning roller 80, only one of the analysis target areas 190 is associated.

[0099] The analysis unit 160 uses the data from the roller data frame 161 to calculate feature quantities for analyzing the actions of the person being analyzed 90 for each analysis area 190. Hereinafter, the feature quantities for analyzing the actions of the person being analyzed 90 will also be simply referred to as "feature quantities". Based on the calculated feature quantities, the analysis unit 160 analyzes the actions of the person being analyzed 90. The feature quantities are the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the analysis area 190, the trajectory length of the coordinates of the cleaning roller 80 within the analysis area 190, or the time during which the coordinates of the cleaning roller 80 are contained within the analysis area 190. Feature quantities are calculated for each analysis area 190.

[0100] Figure 13 is an explanatory diagram showing the convex hull of the time-series coordinates of the cleaning roller 80 located within the head area 191. In Figure 13, the time-series coordinates of the cleaning roller 80 located within the head area 191 are shown as black dots, and the convex hull of these coordinates is shown as a thick dashed line.

[0101] The analysis unit 160 evaluates the cleaning work for each area 190 of the analysis target, based on the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the analysis target area 190, which is a feature quantity. The area of ​​the convex hull of the coordinates of the cleaning roller 80 within the analysis target area 190 can be considered as the cleaning range of the cleaning work using the cleaning roller 80. The analysis unit 160 may also evaluate the cleaning work for each area 190 of the analysis target, based on the value obtained by dividing the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the analysis target area 190 by the area of ​​each analysis target area 190. Hereinafter, the value obtained by dividing the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the analysis target area 190 by the area of ​​each analysis target area 190 will also be called the first feature quantity. Specifically, for example, the analysis unit 160 evaluates the cleaning work for the head area 191 based on the value obtained by dividing the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the head area 191 by the area of ​​the head area 191. This reduces the difference in evaluation based on the physical size of the 90 subjects analyzed.

[0102] The analysis unit 160 evaluates the cleaning work for each area 190 based on the characteristic feature, which is the trajectory length of the coordinates of the cleaning roller 80 within the area 190 to be analyzed.

[0103] Figure 14 is an explanatory diagram showing the trajectory length of the coordinates of the cleaning roller 80 within the analysis area 190.

[0104] In Figure 14, the coordinates of the cleaning roller 80 within the analysis area 190 are shown as black dots over time. 't' represents an arbitrary time, and t+1 represents the time when the next frame after the frame captured at time t was captured. The sum of the lengths of the lines connecting the coordinates of the black dots in chronological order represents the trajectory length of the cleaning roller 80 within the analysis area 190.

[0105] The trajectory length of the coordinates of the cleaning roller 80 within the analysis area 190 indicates the distance the cleaning roller 80 moved within the analysis area 190 during cleaning work using the cleaning roller 80. The analysis unit 160 may evaluate the cleaning work for each analysis area 190 based on the value obtained by dividing the trajectory length of the coordinates of the cleaning roller 80 within the analysis area 190, which is a feature quantity, by the area of ​​the analysis area 190. Hereinafter, the value obtained by dividing the trajectory length of the coordinates of the cleaning roller 80 within the analysis area 190 by the area of ​​the analysis area 190 will also be called the second feature quantity. Specifically, for example, the analysis unit 160 evaluates the cleaning work related to the head area 191 based on the second feature quantity obtained by dividing the trajectory length of the coordinates of the cleaning roller 80 within the head area 191 by the area of ​​the head area 191. This reduces the evaluation difference due to the body size of the person being analyzed 90. Furthermore, the analysis unit 160 may evaluate the cleaning work for each analysis area 190 based on the value obtained by dividing the trajectory length of the coordinates of the cleaning roller 80 within the analysis area 190, which is a feature quantity, by the square root of the area of ​​the analysis area 190. Specifically, the analysis unit 160 may evaluate the cleaning work for the head area 191 based on the value obtained by dividing the trajectory length of the coordinates of the cleaning roller 80 within the head area 191 by the square root of the area of ​​the head area 191.

[0106] The analysis unit 160 may use a feature obtained by multiplying the trajectory length of the coordinates of the cleaning roller 80 within the analysis area 190 by the width of the bounding box 81 of the cleaning roller 80, and evaluate the cleaning work for each analysis area 190 based on this feature. This allows for an appropriate evaluation of the cleaning work, taking into account the cleaning width of the cleaning roller 80.

[0107] The analysis unit 160 evaluates the cleaning work for each analysis area 190 based on the time during which the coordinates of the cleaning roller 80 are contained within the analysis area 190, which is a feature quantity.

[0108] The time during which the coordinates of the cleaning roller 80 are contained within the analysis area 190 reflects the time during which the cleaning roller 80 moved within the analysis area 190 during cleaning work using the cleaning roller 80. The analysis unit 160 may evaluate the cleaning work for each analysis area 190 based on the value obtained by dividing the time during which the coordinates of the cleaning roller 80 are contained within the analysis area 190, which is a feature quantity, by the area of ​​the analysis area 190. Hereinafter, the value obtained by dividing the time during which the coordinates of the cleaning roller 80 are contained within the analysis area 190 by the area of ​​the analysis area 190 will also be called the third feature quantity. Specifically, for example, the analysis unit 160 evaluates the cleaning work related to the head area 191 based on the value obtained by dividing the time during which the coordinates of the cleaning roller 80 are contained within the head area 191 by the area of ​​the head area 191. This reduces the evaluation difference due to the size of the body of the person being analyzed 90. Furthermore, the analysis unit 160 may evaluate the cleaning work for each analysis area 190 based on the value obtained by dividing the time during which the coordinates of the cleaning roller 80 are contained within the analysis area 190, which is a feature quantity, by the square root of the area of ​​the analysis area 190. Specifically, for example, the analysis unit 160 evaluates the cleaning work for the head area 191 based on the value obtained by dividing the time during which the coordinates of the cleaning roller 80 are contained within the head area 191 by the square root of the area of ​​the head area 191.

[0109] The display control unit 170 displays the feature quantities in a visually identifiable manner for each analysis target area 190.

[0110] Figure 15 shows a display screen in which the feature quantities are displayed in a visually identifiable manner. The display screen is shown on the display unit 300 and can be viewed by the person being analyzed 90, allowing the person being analyzed 90 to confirm the cleaning status (see Figure 3).

[0111] In the example in Figure 15, the convex hull of the coordinates of the cleaning roller 80 within the head area 191 is displayed on the screen as a thick dashed line. This makes the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the head area 191, which is a feature, visible. The convex hull of the coordinates of the cleaning roller 80 within the upper body area 192 is shown on the screen as a thick dashed line. This makes the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the upper body area 192, which is a feature, visible. Also, the convex hull of the coordinates of the cleaning roller 80 within the lower body area 193 is shown on the screen as a thick dashed line. This makes the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the lower body area 193, which is a feature, visible. In the example in Figure 15, the feature is displayed visible by superimposing it with the joint point 121. The feature may also be displayed visible by superimposing it with the corresponding analysis target area 190.

[0112] In the example in Figure 15, the time during which the coordinates of the cleaning roller 80 are contained within each analysis area 190, which is a feature, is further displayed on the screen for visual confirmation. It is shown that the time during which the coordinates of the cleaning roller 80 are contained within the head area 191 is 12 seconds. It is shown that the time during which the coordinates of the cleaning roller 80 are contained within the upper body area 192 is 8 seconds. It is also shown that the time during which the coordinates of the cleaning roller 80 are contained within the lower body area 193 is 8 seconds.

[0113] In the example shown in Figure 15, the evaluation results of the cleaning work for each analysis area 190 by the analysis unit 160 are displayed. The analysis unit 160 may evaluate that the cleaning work meets the predetermined standards in any of the following cases, for example:

[0114] (1) The first feature, the second feature, and the third feature are each above a predetermined threshold.

[0115] (2) Of the first feature, second feature, and third feature, one or two of the pre-set features are each above a predetermined threshold.

[0116] In the example in Figure 15, for the head area 191, since the first and third features are above a predetermined threshold, a circle is displayed as the evaluation result, indicating that the cleaning work meets the predetermined standard. As described above, the first feature is the value obtained by dividing the area of ​​the convex hull of the coordinates of the cleaning roller 80 within the analysis area 190 by the area of ​​the analysis area 190. The third feature is the value obtained by dividing the time that the coordinates of the cleaning roller 80 are contained within the analysis area 190 by the area of ​​the analysis area 190. For the upper body area 192 and the lower body area 193, since either the first or third feature is below a predetermined threshold, a triangle is displayed as the evaluation result, indicating that the cleaning work does not meet the predetermined standard. If the coordinates of the cleaning roller 80 are not contained within any of the analysis area 190, a cross may be displayed as the evaluation result, indicating that no cleaning work has been performed.

[0117] If the notification unit 180 detects that the cleaning work has been completed by the person being analyzed 90 while the cleaning work does not meet the predetermined standards, it may notify that the cleaning work does not meet the predetermined standards. For example, the notification unit 180 may notify that the cleaning work does not meet the predetermined standards by sounding a buzzer (not shown).

[0118] The operation of analysis system 1 will be explained.

[0119] Figure 16 is a flowchart of the operation of the analysis system 1. Figure 17 is a subroutine flowchart of step S08 of the flowchart shown in Figure 16. Figure 18 is a subroutine flowchart of step S21 of the subroutine flowchart shown in Figure 17. These flowcharts can be executed by the control unit 100 according to the program.

[0120] The control unit 100 acquires images captured by the imaging device 20 (S01). The control unit 100 may acquire images on a frame-by-frame basis.

[0121] The control unit 100 detects skeletal information based on the acquired image (S02). That is, the control unit 100 detects joint points 121 based on the image.

[0122] The control unit 100 detects the cleaning roller 80 based on the acquired image (S03).

[0123] The control unit 100 determines whether the person to be analyzed 90 is present in the monitoring area 25 (S04). If the control unit 100 determines that the person to be analyzed 90 is not present in the monitoring area 25 (S04: NO), it repeats step S04.

[0124] When the control unit 100 determines that the person to be analyzed 90 is present in the monitoring area 25 (S04: YES), the control unit 100 determines the area to be analyzed 190 (S05). Specifically, the control unit 100 determines the head area 191, the upper body area 192, and the lower body area 193.

[0125] The control unit 100 determines whether the person being analyzed 90 has started cleaning (S06). The control unit 100 determines the start and end of cleaning by determining, based on the joint point 121, that the person being analyzed 90 has made a specific gesture corresponding to each. Specific gestures can be stored in advance in the storage unit 200. A specific gesture corresponding to the start of cleaning may include the posture of holding the cleaning roller 80 above the head, as shown in Figure 8. A specific gesture corresponding to the end of cleaning may include the posture of the person being analyzed 90 returning the cleaning roller 80 to its storage case.

[0126] If the control unit 100 determines that the person being analyzed 90 has not started the cleaning work (S06: NO), it repeats step S06.

[0127] When the control unit 100 determines that the person being analyzed 90 has started cleaning work (S06:YES), it stores the information of the area to be analyzed 190, the coordinates of the cleaning roller 80, and the timestamp in the storage unit 200 in association with each other (S07). Specifically, the control unit 100 stores the coordinates of the cleaning roller 80, which is the information of the cleaning roller 80, along with a timestamp and a label indicating whether the cleaning roller 80 is located in the area to be analyzed 190, as a roller data frame 161 in the storage unit 200.

[0128] The control unit 100 analyzes the actions of the person being analyzed 90 (S08). As shown in Figure 17, the control unit 100 analyzes the actions of the person being analyzed 90 as follows.

[0129] (Step S21) The control unit 100 performs a label transition determination process. As shown in Figure 18, the control unit 100 performs the label transition determination process as follows.

[0130] (Step S31) The control unit 100 acquires the roller data frame 161 by reading it from the storage unit 200 (S31). The data length n of the roller data frame 161 corresponds to the number of frames in the roller data frame 161. For example, the data length n is 2300.

[0131] (Steps S32-S36) Steps S32 to S36 are a loop process. The initial value i=0 and the end value is n-1. The data with the initial value i=0 is the data first stored in the roller data frame 161. The data with the end value i=n-1 is the data last stored in the roller data frame 161.

[0132] In step S33, if the label of data i in the acquired roller data frame 161 is a duplicate label indicating that it is within the overlapping range of the analysis areas 190, i.e., "Head and Upper" or "Upper and Lower", or if it is NaN (null), the following steps are skipped. Then, in step S36, the label for the next data i+1 is acquired, and the processing from step S32 is executed. In all other cases, i.e., if the label is one of "Head", "Upper", or "Lower", the processing proceeds to step S34.

[0133] In step S34, if the same label appears N times or more consecutively (YES), a state transition is detected, and the process proceeds to step S35. Here, the same label refers to any one of the labels "Head," "Upper," and "Lower," excluding duplicate labels and NaN labels, appearing N times or more consecutively. N is a pre-set integer value, which can be any value from 2 to 10. For example, N=3. On the other hand, if the previous label (i-1) and the current label are different, or if the number of consecutive identical labels is less than N, the next step S35 is skipped.

[0134] Step S35 records the transition of labels. Figure 19 shows a Roller data frame 161 in which the label transitions have been recorded. In Range 0-2, the same label, "Head," appears three times consecutively (area enclosed by a dashed rectangle). Therefore, going back to Range 0, the start of the transition to the label "Head" is recorded. Similarly, in Range 200-202, the same label, "Upper," appears three times consecutively. Therefore, the start of the transition to the label "Upper" is recorded in Range 200. In addition, the previous Range 199 records the end of the transition of the label whose transition start was recorded immediately before it. For example, in the example in Figure 19, the end of the transition of the label "Head," whose transition start was recorded in the immediately preceding Range 0, is recorded in Range 199. Through similar processing, the label "Head," which marks the start of the transition, is recorded in Range 500, and the label "Upper," which marks the end of the transition, is recorded in the previous Range 499.

[0135] Figure 20 shows the output of the Range for each label in the Raw of the roller data frame 161 after the label transition determination process. For example, Ranges 0-199 and 500-602 are labeled "Head", Ranges 200-400, 603-999, and 2000-2299 are labeled "Upper", and Range 1000-1999 is labeled "Lower".

[0136] (Step S22) As shown in Figure 17, the control unit 100 calculates the feature quantities (S22).

[0137] (Step S23) The control unit 100 analyzes the actions of the person being analyzed 90 based on the feature quantities. Specifically, the control unit 100 evaluates the cleaning work of the person being analyzed 90 for each area being analyzed 190 based on the feature quantities.

[0138] Returning to Figure 16, we will continue the explanation.

[0139] The control unit 100 displays the analysis results for the subject 90 (S09). Specifically, the control unit 100 displays the feature quantities in a visually identifiable manner for each analysis area 190. The control unit 100 also displays the evaluation results of the cleaning work performed by the subject 90. The evaluation results of the cleaning work may include the evaluation results of the cleaning work for each analysis area 190 and the overall evaluation results of the cleaning work for all analysis areas 190.

[0140] (Second Embodiment) A second embodiment will now be described. This embodiment differs from the first embodiment in that the upper body area 192 is determined as the range that the person being analyzed 90 can reach by extending their arms, and a warning is issued when a pre-registered object to be analyzed 70 is detected within the upper body area 192. In the following description of the second embodiment, explanations that overlap with the description of the first embodiment will be omitted or simplified. In this embodiment, the control unit 100 does not necessarily have to function as the presence determination unit 130 and the display control unit 170.

[0141] The object detection unit 140 detects pre-registered objects 70 for analysis in the image acquired by the image acquisition unit 110. The object detection unit 140 may detect multiple pre-registered objects 70 for analysis in the image acquired by the image acquisition unit 110. The pre-registered objects 70 for analysis include objects that are dangerous to touch by hand and objects that could cause a person to trip. Examples of dangerous objects that are dangerous to touch by hand include knives, hot cooking utensils, and hot cooking equipment. Examples of objects that could cause a person to trip include steps on the floor and cleaning tools placed on the floor.

[0142] The analysis area determination unit 150 determines the analysis area 190 based on the joint points 121 acquired as body size-related information by the skeletal information detection unit 120 in the image acquired by the image acquisition unit 110. Specifically, the analysis area determination unit 150 determines the upper body area 192 as the range that the person being analyzed 90 can reach by extending their arms. The upper body area 192 can be determined as the smallest rectangle enclosing a circle centered on the "right shoulder" with a radius equal to the sum of the distance between the "right shoulder" and the "right elbow" and the distance between the "right elbow" and the "right wrist", and a circle centered on the "left shoulder" with a radius equal to the sum of the distance between the "left shoulder" and the "left elbow" and the distance between the "left elbow" and the "left wrist". The analysis area determination unit 150 also determines the lower body area 193 as the range that the person being analyzed 90 can reach by extending their legs. The lower body area 193 can be determined as the smallest rectangle enclosing a circle centered at the "right hip" with a radius equal to the sum of the distance between the "right hip" and the "right knee" and the distance between the "right knee" and the "right ankle," and a circle centered at the "left hip" with a radius equal to the sum of the distance between the "left hip" and the "left knee" and the distance between the "left knee" and the "left ankle."

[0143] The analysis unit 160 determines whether the object to be analyzed 70 has been detected within the analysis area 190. As described above, the object to be analyzed 70 is registered in advance. The object to be analyzed 70 may be registered for each analysis area 190. For example, for the upper body area 192, one or more objects that are dangerous if touched by a person's hand are registered. For the lower body area 193, one or more objects that could cause a person to trip if touched by a person's hand are registered.

[0144] Figure 21 is an explanatory diagram showing the state in which the analyte 70 is detected within the analysis area 190.

[0145] As shown in Figure 21, the bounding box of the object to be analyzed 70 overlaps with the upper body area 192. In this case, it is determined that the object to be analyzed 70 was detected within the upper body area 192.

[0146] The analysis unit 160 can calculate the frequency with which the analyte 70 was present within the analysis area 190 and display it on the display unit 300. This allows the manager to understand situations in the work environment of the person being analyzed 90 that could lead to an accident, and to consider improving the environment.

[0147] The notification unit 180 issues a warning if the analysis unit 160 determines that the object to be analyzed 70 has been detected within the analysis area 190. For example, the notification unit 180 may sound a buzzer (not shown) to notify that the person being analyzed 90 may be injured or fall due to movement. The notification unit 180 may also identify the person being analyzed 90 by facial recognition based on the image and notify the device carried by the identified person being analyzed 90 that there is a possibility of injury or fall due to movement.

[0148] The operation of analysis system 1 will be explained.

[0149] Figure 22 is a flowchart showing the operation of the analysis system 1. This flowchart can be executed by the control unit 100 according to a program.

[0150] The control unit 100 acquires the image captured by the imaging device 20 (S41). The control unit 100 may acquire the image frame by frame.

[0151] The control unit 100 detects skeletal information based on the acquired image (S42). That is, the control unit 100 detects joint points 121 based on the image.

[0152] The control unit 100 detects a pre-set object to be analyzed 70 based on the acquired image (S43).

[0153] The control unit 100 determines the analysis area 190 (S44). Specifically, based on the joint points 121, the control unit 100 determines the upper body area 192 as the range that the person being analyzed 90 can reach by extending their arms. Based on the joint points 121, the control unit 100 determines the lower body area 193 as the range that the person being analyzed 90 can reach by extending their legs.

[0154] The control unit 100 determines whether the analyte 70 is present in the analysis area 190 (S45). If the control unit 100 determines that the analyte 70 is not present in the analysis area 190 (S45: NO), it returns to step S41 and continues processing.

[0155] If the control unit 100 determines that the analyte 70 is present within the analysis area 190 (S45: YES), it issues a warning (S46).

[0156] The control unit 100 calculates the frequency with which the analyte 70 was present within the analysis area 190 and displays it on the display unit 300 (S47).

[0157] The embodiment provides the following effects.

[0158] Images of a designated monitoring area are captured, along with body-related information concerning the physique of the person being analyzed. Based on the image, an analysis area is determined according to the body-related information, and the person being analyzed within that area is analyzed. This makes it easy to improve the accuracy of the analysis of the person being analyzed, regardless of their physique.

[0159] Furthermore, skeletal information is acquired as body-related information based on the image. This makes it easier to further improve the accuracy of the analysis of the subject, regardless of their body size.

[0160] Furthermore, based on the image, the system determines whether the person to be analyzed is present in the monitoring area, and if so, analyzes the person. This reduces the computational load required for analyzing the person.

[0161] Furthermore, based on body size-related information, the analysis area for each body part of the subject is determined. This allows for a more effective improvement in the accuracy of the analysis of the subject.

[0162] Furthermore, based on body size-related information, the size and position of the analysis target area are determined by adjusting them against standard values ​​for size and position of the analysis target area. This makes it possible to improve the accuracy of the analysis of the analysis target more effectively and easily.

[0163] Furthermore, the analysis area is determined based on the average or median of the skeletal information of the subject obtained as body-related information from the image, across multiple frames. This allows for easy and stable improvement of the analysis accuracy of the subject.

[0164] Furthermore, the reference values ​​are adjusted based on user instructions. This allows for easy and flexible improvement of the accuracy of the analysis of the subjects being analyzed.

[0165] Furthermore, the system detects objects to be analyzed in images, and if an object is detected within the analysis area, it stores the information of the analysis area, the object, and a timestamp in association with each other. This further improves the accuracy of the analysis of the subject.

[0166] Furthermore, information about the object being analyzed is stored in the memory unit as relative coordinates to the area being analyzed. This allows for flexible improvement of the accuracy of the analysis of the object being analyzed.

[0167] Furthermore, it can detect multiple analytes within the analysis area. This ensures the safety of the person being analyzed, even when multiple analytes are present.

[0168] Furthermore, the system determines whether the analyte has been detected within the analysis area, and if it is determined that the analyte has been detected within the analysis area, it issues a warning. This makes it easier to ensure the safety of the person being analyzed when the analyte is present.

[0169] Furthermore, using the stored data described above, feature quantities for analyzing the subject's movements are calculated for each analysis area, and the subject's movements are analyzed based on these feature quantities. This allows for a more appropriate improvement in the accuracy of the subject's analysis.

[0170] Furthermore, the information of the object being analyzed is defined as its coordinates. The feature quantities are then calculated for each area being analyzed, and are defined as the area of ​​the convex hull of the coordinates of the object being analyzed within the analysis area, the length of the trajectory of the coordinates of the object being analyzed within the analysis area, or the time during which the coordinates of the object being analyzed are contained within the analysis area. This allows for a more appropriate and effective improvement in the accuracy of the analysis of the object being analyzed.

[0171] Furthermore, the analysis area is determined based on skeletal information. Then, features are calculated by dividing the area of ​​the analysis area by the area of ​​the analysis area, the area of ​​the convex hull of the coordinates of the analyte within the analysis area, the trajectory length of the coordinates of the analyte within the analysis area, or the time the coordinates of the analyte are contained within the analysis area. This effectively reduces the impact of differences in the analyte's physique on the analysis accuracy.

[0172] Furthermore, the features are displayed in a visually identifiable manner for each area being analyzed. This allows for a visual recognition of the degree of completion and scope of work performed by the analyzed individuals in each area being analyzed.

[0173] Furthermore, the object to be analyzed will be a cleaning roller. This will improve the accuracy of the analysis of the degree of cleaning work performed by the person being analyzed.

[0174] The present invention is not limited to the embodiments described above.

[0175] For example, in this embodiment, skeletal information is acquired as body size-related information. However, the input unit 400 may receive information related to the body size of the person being analyzed, such as weight and height, and based on this information, an analysis area 190 corresponding to each person being analyzed 90 may be determined.

[0176] Furthermore, in the embodiment, some or all of the processing performed by the program may be replaced with hardware such as circuits.

[0177] While embodiments of the present invention have been described and illustrated in detail, the disclosed embodiments are for illustrative purposes only and are not limiting. The scope of the present invention should be interpreted in accordance with the language of the appended claims. [Explanation of symbols]

[0178] 1. Analysis system, 10 Analyzer, 20 Imaging device, 25 surveillance areas, 70 Analytes 80 cleaning rollers, 81. Bounding box surrounding the cleaning component, 90 people analyzed, 100 Control unit, 110 Image acquisition unit, 120 Skeletal information detection unit, 121 joint points, 121a Ankle joint point, 121S Reference joint point, 130 Presence determination unit, 140 Object detection unit, 150 Analysis target area determination unit, 160 Analysis Department, 161 roller data frame, 170 Display control unit, 180 Hochi Department, 190 areas to be analyzed, 190S reference rectangle, 191 head area, 191S Reference rectangle for the head area, 192 Upper body area, 192S Reference rectangle for the upper body area, 193 Lower body area, 193S Reference rectangle for the lower body area, 200 storage section, 300 display section, 400 Input section, 500 Communications Department, 600 buses.

Claims

1. An image acquisition unit that acquires images of a designated monitoring area where the person being analyzed is located, A body-related information acquisition unit acquires body-related information related to the physique of the person being analyzed, An analysis target area determination unit determines an analysis target area in the image acquired by the image acquisition unit, according to the body size-related information acquired by the body size-related information acquisition unit, An analysis unit that analyzes the subjects of analysis in the area determined by the analysis area determination unit, An analytical system having the following features.

2. The analysis system according to claim 1, wherein the body size-related information acquisition unit acquires skeletal information as body size-related information based on the image acquired by the image acquisition unit.

3. The system further includes an existence determination unit that determines whether the person to be analyzed is present in the monitoring area based on the image acquired by the image acquisition unit, The analysis system according to claim 1, wherein the analysis unit performs an analysis on the person to be analyzed when the existence determination unit determines that the person to be analyzed exists.

4. The analysis system according to claim 1, wherein the analysis target area determination unit determines the analysis target area for each body part of the person to be analyzed based on the body size-related information.

5. The analysis system according to claim 1, wherein the analysis target area determination unit determines the analysis target area obtained by adjusting the size and position of the analysis target area with respect to reference values ​​for size and position of the analysis target area based on the body size-related information.

6. The analysis system according to claim 2, wherein the analysis target area determination unit determines the analysis target area based on the average or median value of multiple frames of the skeletal information of the person to be analyzed, which is obtained as body size-related information from the image acquired by the image acquisition unit.

7. The analysis system according to claim 5, wherein the analysis target area determination unit adjusts the reference value based on the user's instructions.

8. The system further includes an object detection unit that detects an object to be analyzed in the image acquired by the image acquisition unit, The analysis system according to claim 1, wherein, when the analysis unit detects the object to be analyzed within the analysis target area, it stores the information of the analysis target area, the information of the object to be analyzed, and a timestamp in a storage unit in association with each other.

9. The analysis system according to claim 8, wherein the analysis unit stores information of the object to be analyzed in the storage unit as relative coordinates with respect to the area to be analyzed.

10. The analysis system according to claim 8, wherein the object detection unit detects a plurality of objects to be analyzed within the analysis target area.

11. The analysis unit determines whether the object to be analyzed was detected within the analysis area, The analysis system according to claim 8, further comprising a notification unit that issues a warning when the analysis unit determines that the object to be analyzed has been detected within the area to be analyzed.

12. The analysis system according to claim 8, wherein the analysis unit uses the data stored in the memory unit to calculate feature quantities for analyzing the actions of the person being analyzed for each area to be analyzed, and analyzes the actions of the person being analyzed based on the calculated feature quantities.

13. The information of the object to be analyzed is the coordinates of the object to be analyzed. The analysis system according to claim 12, wherein the feature quantity is the area of ​​the convex hull of the coordinates of the object to be analyzed within the analysis area, the trajectory length of the coordinates of the object to be analyzed within the analysis area, or the time during which the coordinates of the object to be analyzed are contained within the analysis area, and is calculated for each analysis area.

14. The analysis system according to claim 13, wherein the analysis unit calculates the feature quantity by dividing the area of ​​the convex hull of the coordinates of the object to be analyzed within the analysis area, the trajectory length of the coordinates of the object to be analyzed within the analysis area, or the time during which the coordinates of the object to be analyzed are contained within the analysis area by the area of ​​the analysis area.

15. The analysis system according to claim 12, further comprising a display unit that displays the aforementioned feature quantities in a visible manner for each of the areas to be analyzed.

16. The analysis system according to claim 8, wherein the object to be analyzed is a cleaning roller.

17. Step (a) acquires an image of a designated monitoring area in which the person to be analyzed is located, (b) A step of obtaining body-related information related to the body size of the person being analyzed, Step (c) is to determine the area to be analyzed in the image obtained in step (a) according to the body size-related information obtained in step (b), An analysis program for causing a computer to perform a process comprising: step (d) analyzing the person to be analyzed in the area to be analyzed determined in step (c);

18. A method performed by an analysis system, Step (a) acquires an image of a designated monitoring area in which the person to be analyzed is located, (b) A step of obtaining body-related information related to the body size of the person being analyzed, Step (c) is to determine the area to be analyzed in the image obtained in step (a) according to the body size-related information obtained in step (b), An analysis method comprising step (d) of analyzing the subjects of analysis in the area of ​​analysis determined in step (c).

Images