Busyness evaluation program, busyness evaluation device, and control system

Abstract

To accurately evaluate the busyness of a worker.SOLUTION: A busyness evaluation device includes an acquisition unit configured to acquire information on a change in a position of a hand of a worker, a specification unit configured to specify a ratio of a movement with a shoulder as a fulcrum to a movement of the hand of the worker based on the information acquired by the acquisition unit, and an evaluation unit configured to evaluate busyness of the worker based on the ratio specified by the specification unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a busyness assessment program, a busyness assessment device, and a control system. [Background technology]

[0002] Manual sorting of objects flowing on conveyors is an essential task in many industries, including agriculture, manufacturing, and waste disposal. Designing and operating a manual sorting line requires assigning an appropriate number of workers to the flow rate of objects, and appropriately controlling the flow rate of objects according to the number of workers.

[0003] However, unlike machine sorting, manual sorting involves workers whose busyness varies depending on their level of experience and the number of workers involved. Therefore, unless workers' busyness levels are understood, it is not possible to design an appropriate line or operate efficiently.

[0004] Conventionally, there is known a technology for remotely grasping the delays in the work of each worker by detecting a series of periodic work actions of workers who repeatedly perform a certain task on a production line (see, for example, Patent Document 1). Also known is a technology for evaluating the busyness of a worker based on the amount of input operations from multiple input devices, and a technology for grasping the busyness of a worker based on the worker's pulse rate (see, for example, Patent Documents 2 and 3). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-67419 [Patent Document 2] Japanese Patent Publication No. 2022-175584 [Patent Document 3] Japanese Patent Application Laid-Open No. 2017-163208 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology in Patent Document 1, mentioned above, identifies work delays relative to the registered work cycles or intervals of workers performing repetitive work by comparing arm acceleration with registered work movements, but is not intended to identify how busy a worker is.

[0007] Furthermore, the technology of Patent Document 2 is based on the premise that a worker operates multiple input devices, and is unable to evaluate how busy a worker is when performing manual sorting, etc. Furthermore, the technology of Patent Document 3 determines how busy a worker is from their pulse rate, which may result in low accuracy.

[0008] The present invention aims to provide a busyness assessment program and busyness assessment device that can accurately assess how busy a worker is, and a control system that can appropriately control equipment according to how busy a worker is. [Means for solving the problem]

[0009] The busyness evaluation program of the present invention is a program that causes a computer to execute the following process: acquire information regarding changes in the position of a worker's hands; based on the acquired information, identify the proportion of the worker's hand movements that are fulcrums at the shoulders; and evaluate the busyness of the worker based on the identified proportion.

[0010] The busyness assessment device of the present invention comprises an acquisition unit that acquires information regarding changes in the position of a worker's hands, an identification unit that identifies the proportion of the worker's hand movements that are centered around the shoulders based on the information acquired by the acquisition unit, and an evaluation unit that evaluates the busyness of the worker based on the proportion identified by the identification unit.

[0011] The control system of the present invention comprises the busyness assessment device described above, and a control device that controls equipment used in the work of the worker based on the assessment result by the assessment section of the busyness assessment device. [Effects of the Invention]

[0012] The busyness assessment program and busyness assessment device of the present invention have the effect of being able to accurately assess the busyness of a worker.

[0013] Furthermore, the control system of the present invention has the effect of being able to appropriately control equipment depending on how busy the worker is. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing the configuration of a manual sorting system according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing the hardware configuration of the busyness evaluation device. [Figure 3] FIG. 3 is a functional block diagram of the busyness assessment device according to the first embodiment. [Figure 4] FIG. 4 is a diagram for explaining the normal working range, the optimum working range, and the maximum working range. [Figure 5] FIG. 5 is a flowchart showing the process of the busyness assessment device according to the first embodiment. [Figure 6] FIG. 6(a) is a diagram showing an example of an image acquired by the image acquisition unit, and FIG. 6(b) is a diagram showing the state in which the feature points of the worker (positions of both shoulders and both hands) have been extracted from the image. [Figure 7] FIG. 7(a) is a diagram showing a case where both hands are within the normal working area, and FIG. 7(b) is a diagram showing a case where the right hand is outside the normal working area. [Figure 8] FIG. 8 is a flowchart (part 1) showing the first modification of the first embodiment. [Figure 9] FIG. 9 is a flowchart (part 2) showing the first modification of the first embodiment. [Figure 10] FIG. 10 is a functional block diagram of a busyness assessment device according to the second embodiment. [Figure 11] 11 is a diagram for explaining the shoulder non-mobilization range, and is a flowchart showing an example of a top dressing amount output process. [Figure 12]12(a) and 12(b) are diagrams showing examples of time-series data of the amount of movement. [Figure 13] FIG. 13 is a flowchart (part 1) showing the process of the busyness assessment device according to the second embodiment. [Figure 14] FIG. 14 is a flowchart (part 2) showing the process of the busyness assessment device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] First Embodiment The first embodiment will be described below with reference to FIGS. 1 to 7(b).

[0016] 1 is a diagram schematically illustrating the configuration of a manual sorting system 200 according to the first embodiment. The manual sorting system 200 according to the first embodiment is a system used for manually sorting items (such as potatoes) in a workplace such as a fruit sorting facility. However, the use of the manual sorting system 200 is not limited to this, and it may also be used for manual sorting carried out in the manufacturing industry or waste disposal industry.

[0017] As shown in FIG. 1, the manual sorting system 200 includes a conveying device 50 as equipment used by an operator, a camera 60 as an imaging device, and a control system 100.

[0018] The conveying device 50 is a device for conveying items (potatoes) and includes a conveyor 30 and a drive device 40. The conveyor 30 is a belt conveyor, a roller conveyor, or the like. The drive device 40 has a motor or the like and drives the conveyor 30 so that the potatoes are conveyed on the conveyor 30 in a predetermined direction (from left to right on the page in FIG. 1). A worker is positioned in front of the conveyor 30 and performs manual sorting work, such as sorting the potatoes conveyed on the conveyor 30 by size and separating out potatoes that are misshapen or rotten.

[0019] The camera 60 is installed above the conveyor 30 (on the ceiling, etc.) and captures from above the workers performing manual sorting work in front of the conveyor 30 and the surrounding area. In this embodiment, a camera 60 is provided at each location where a worker is positioned, and one camera 60 captures (photographs) one worker. The camera 60 transmits the captured image (still image or video) to the busyness evaluation device 10 of the control system 100.

[0020] The control system 100 includes a busyness evaluation device 10 and a conveyor control device 20 as a control device.

[0021] The busyness assessment device 10 is a device that assesses the busyness of a worker performing manual sorting work in front of the conveyor 30 based on images received from the camera 60. The busyness assessment device 10 displays the busyness assessment results and transmits (outputs) them to the conveyor control device 20. Details of the busyness assessment device 10 will be described later.

[0022] The conveyor control device 20 determines the transport speed of the conveyor 30 based on the busyness evaluation result by the busyness evaluation device 10, and controls the drive device 40 to achieve the determined transport speed. For example, if the conveyor control device 20 determines that the worker is busy, it slows down the speed of the conveyor 30 by a predetermined speed, and if it determines that the worker is not busy, it speeds up the speed of the conveyor 30 by a predetermined speed.

[0023] (Regarding the busyness assessment device 10) FIG. 2 shows a schematic diagram of the hardware configuration of the busyness assessment device 10. As shown in FIG. 2, the busyness assessment device 10 includes a central processing unit (CPU) 90, a read-only memory (ROM) 92, a random access memory (RAM) 94, storage (such as a hard disk drive (HDD) or a solid state drive (SSD)) 96, a network interface 97, a display unit 93, an input unit 95, and a portable storage medium drive 99. These components of the busyness assessment device 10 are connected to a bus 98. In the busyness assessment device 10, the CPU 90 executes a program (including a busyness assessment program) stored in the ROM 92 or the storage 96, or a program read by the portable storage medium drive 99 from the portable storage medium 91, thereby realizing the functions of the components shown in FIG. 3. Note that the functions of the components shown in FIG. 3 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0024] Fig. 3 is a functional block diagram of the busyness assessment device 10. A CPU 90 of the busyness assessment device 10 executes a program to function as each unit shown in Fig. 3. That is, the busyness assessment device 10 has the functions of a physique information acquisition unit 11, a normal work area determination unit 12, an image acquisition unit 13, a feature point extraction unit 14, a judgment unit 15, an outside-normal work area ratio calculation unit 16, a busyness assessment unit 17 as an evaluation unit, a display control unit 18 as a notification unit, and a transmission unit 19.

[0025] The physique information acquisition unit 11 receives input of information on the physique of the worker (height in the first embodiment). A manager or the like inputs the information on the physique of the worker via the input unit 95. The information on the physique of the worker may be information on shoulder width or arm length.

[0026] The normal working area determination unit 12 determines a radius R that defines the normal working area as a first range based on the worker's physique information (height). In the first embodiment, referring to industrial engineering, the range in which work can be done with the elbow at the center (fulcrum) is defined as the "normal working area," and the range in front of the body where the normal working areas of both hands overlap (the range in which work can be done with both hands at the elbow at the center (fulcrum)) is defined as the "optimum working area." The normal working area is a range of radius R centered on the shoulder. Here, radius R is correlated with the worker's arm length. It is also generally known that there is a correlation between arm length and height. Therefore, by determining the relationship between radius R and height in advance, radius R can be determined for each worker based on the worker's height. The area outside the normal working area (second range) is called the "maximum working area," and is the area in which work can be done by extending the arms with the shoulder as the fulcrum.

[0027] The image acquisition unit 13 acquires an image captured by the camera 60. The feature point extraction unit 14 extracts the positions of both shoulders and both hands as feature points from the image acquired by the image acquisition unit 13. As a method for extracting feature points from an image, existing methods such as machine learning and deep learning can be used.

[0028] The determination unit 15 determines whether or not either of the worker's hands is outside the normal work area in each image acquired by the image acquisition unit 13. If the determination unit 15 determines that either of the worker's hands is outside the normal work area, it adds an "outside normal work area flag" to that image. On the other hand, if the determination unit 15 determines that neither of the worker's hands is outside the normal work area, it does not add an "outside normal work area flag" to that image.

[0029] The outside-of-normal-work-area ratio calculation unit 16 calculates the ratio of images to which an "outside-of-normal-work-area flag" is added among images captured within a predetermined time period.

[0030] The busyness evaluation unit 17 determines whether the worker is busy or not based on the ratio calculated by the outside-of-normal-work-area ratio calculation unit 16. For example, if the ratio exceeds a predetermined threshold (e.g., 50%), the busyness evaluation unit 17 determines the worker to be "busy," and if the ratio is equal to or less than the threshold, the busyness evaluation unit 17 determines the worker to be "not busy." The busyness evaluation unit 17 may evaluate the degree of busyness in three or more stages, rather than in two stages, "busy" and "not busy." In this embodiment, the busyness evaluation is performed under the assumption that, when performing manual sorting work, a worker who is not busy often manually sorts potatoes within the optimal work area, while a busy worker often cannot keep up with the conveying speed of the conveyor 30 and stretches out his or her hand to pick potatoes that are far away (that have passed in front of the worker).

[0031] The display control unit 18 displays the evaluation result of the busyness evaluation unit 17 on the display unit 93 to notify a manager, etc. The transmission unit 19 transmits the evaluation result of the busyness evaluation unit 17 to the conveyor control device 20.

[0032] (Regarding the processing of the busyness assessment device 10) The processing of the busyness assessment device 10 will be described in detail below. Fig. 5 is a flowchart showing the processing content of the busyness assessment device 10. Note that the flowchart in Fig. 5 is performed for each worker (i.e., for each camera 60). Therefore, when multiple workers work at the same time, processing corresponding to each of the multiple workers is executed simultaneously in parallel.

[0033] 5 starts, first, in step S10, the physique information acquisition unit 11 acquires the physique information of the worker. The physique information acquisition unit 11 displays, for example, "Please input the worker's height" on the display unit 93, and when a worker, manager, or the like inputs height information via the input unit 95, acquires the input height information.

[0034] Next, in step S12, the normal working area determiner 12 determines the radius R of the normal working area from the physique information (height). If the normal working area determiner 12 has an equation showing the relationship between height and radius R, it calculates the radius R by substituting the value of height into the equation. Furthermore, if a machine learning model that uses height as an explanatory variable and radius R as a target variable is available, the normal working area determiner 12 obtains the value of radius R by inputting the value of height into the machine learning model.

[0035] Next, in step S14, the image acquisition unit 13 waits until the worker starts the manual sorting work. Whether the worker has started the manual sorting work may be determined based on the worker's input to the input unit 95, or may be determined from the image captured by the camera 60.

[0036] When the worker starts the manual sorting work, the process proceeds to step S16, and the image acquisition unit 13 acquires an image (one still image or one frame of a video) captured by the camera 60. For example, the image acquisition unit 13 acquires an image of the worker as viewed from above, as shown in FIG. 6(a).

[0037] Next, in step S18, the feature point extraction unit 14 extracts feature points (positions of both shoulders and both hands) of the worker from the image acquired by the image acquisition unit 13. For example, the feature point extraction unit 14 extracts the positions of both shoulders indicated by "●" and "○" in Fig. 6(b) and the positions of both hands indicated by "■" and "□" in Fig. 6(b) from the image of Fig. 6(a).

[0038] Next, in step S20, the determination unit 15 determines whether at least one of the hands is outside the normal working area. Specifically, the determination unit 15 first sets a "normal working area (right)" indicated by a solid line based on the position of the right shoulder (●) and the radius R, and sets a "normal working area (left)" indicated by a dashed line based on the position of the left shoulder (○) and the radius R. The determination unit 15 then determines whether the position of the right hand (■) is outside the normal working area (right) and whether the position of the left hand (□) is outside the normal working area (left). In step S20, if both the right hand and the left hand are inside the normal working area as shown in FIG. 7(a), the determination is "no," and the process proceeds to step S24. On the other hand, if at least one of the right hand and the left hand is outside the normal working area as shown in FIG. 7(b), the determination is "yes," and the process proceeds to step S22.

[0039] When the process proceeds to step S22, the determination unit 15 assigns an outside normal working area flag to the image, and then the process proceeds to step S24.

[0040] In step S24, the outside-of-normal-work-area proportion calculation unit 16 determines whether a predetermined number of images or more have been acquired. Here, the "predetermined number" is the number of images that the camera 60 can capture in the unit time used to determine the worker's busyness. If the determination in step S24 is negative, the process returns to step S16. The processing and determination in steps S16 to S24 are then repeated until the determination in step S24 is positive. Note that if the positions of both shoulders hardly move, the feature point extraction unit 14 may not extract the positions of both shoulders in the second and subsequent iterations of step S18, and may instead use the positions of both shoulders extracted the first time. If the determination in step S24 is positive, the process proceeds to step S26.

[0041] In step S26, the outside-of-normal-work-area calculation unit 16 calculates the proportion of images to which an outside-of-normal-work-area flag has been assigned among the acquired images. That is, the outside-of-normal-work-area calculation unit 16 calculates the proportion of images outside the normal work area based on the following equation (1). Percentage of images outside the normal working area = (number of images flagged as outside the normal working area / number of images acquired) × 100 (%) ... (1)

[0042] It can be said that the above formula (1) essentially determines the proportion outside the normal operating range shown in the following formula (2). Percentage of work outside the normal working area = (Work time outside the normal working area / Total work time) x 100 (%) …(2)

[0043] The percentage outside the normal working range can also be said to be the percentage of the worker's hand movements that are centered around the shoulder.

[0044] Next, in step S28, the busyness evaluation unit 17 determines whether the ratio calculated in step S26 exceeds a predetermined threshold. If the threshold is, for example, 50(%), the determination in step S28 is positive if the calculated ratio exceeds 50%, and negative if the calculated ratio is 50% or less.

[0045] If the determination in step S28 is positive, the process proceeds to step S30, where the busyness assessment unit 17 determines that the worker is "busy." On the other hand, if the determination in step S28 is negative, the process proceeds to step S32, where the busyness assessment unit 17 determines that the worker is "not busy."

[0046] After step S30 or S32, the process proceeds to step S34. In step S34, the display control unit 18 displays the evaluation result (the determination result of step S30 or step S32) on the display unit 93. A manager or the like can confirm whether a worker is busy by referring to the evaluation result displayed on the display unit 93. If it is determined that a worker is busy, the manager or the like can manually adjust the conveying speed of the conveyor 30 to slow it down. Displaying the evaluation result in chronological order allows the manager or the like to know how long the busy period has continued, which can then be used to replace or reassign workers, encourage workers to take breaks, or increase the number of workers. When displaying the evaluation result, the display control unit 18 may generate and display information based on the evaluation result (e.g., information indicating that more workers should be hired or that a break should be taken) together with the evaluation result. In step S34, the transmission unit 19 transmits the evaluation result to the conveyor control device 20. In this case, the conveyor control device 20 adjusts the transport speed of the conveyor 30 based on the busyness evaluation result so that the busyness of the worker is appropriate. If a supplying device that supplies potatoes is located upstream of the conveyor 30, the transmitting unit 19 may transmit the busyness evaluation result to the supplying device. In this case, the supplying device may adjust the amount of potatoes supplied in accordance with the busyness evaluation result.

[0047] Next, in step S36, the image acquisition unit 13 determines whether the worker has finished the manual sorting work. Whether the worker has finished the manual sorting work may be determined based on the worker's input to the input unit 95, or may be determined from the image captured by the camera 60. If the determination in step S36 is negative, the process returns to step S16, but if the determination in step S36 is positive, all of the processing in FIG. 5 is terminated.

[0048] As can be seen from the above description, in the first embodiment, the image acquisition unit 13 and the feature point extraction unit 14 function as an acquisition unit that acquires information regarding changes in the position of the worker's hands. The determination unit 15 and the out-of-normal working area ratio calculation unit 16 function as an identification unit that identifies the ratio of hand movements of the worker that are centered around the shoulders, based on the information acquired by the acquisition unit.

[0049] As described above in detail, in the busyness assessment device 10 of the first embodiment, the feature point extraction unit 14 extracts information about the position of the worker's hands from the image captured by the camera 60. Furthermore, the determination unit 15 and the out-of-normal work area proportion calculation unit 16 calculate the proportion of shoulder-based movements among the worker's hand movements based on the acquired information. The busyness assessment unit 17 then evaluates the worker's busyness based on the calculated proportion. As a result, in the first embodiment, the higher the proportion of shoulder-based movements, the busier the worker is determined to be, and therefore, it is possible to appropriately evaluate the worker's busyness in work that requires manual movement, such as manual sorting.

[0050] Furthermore, in the first embodiment, the out-of-normal working area ratio calculation unit 16 calculates, based on the image, the ratio of hand positions within a second range (maximum working area) outside a first range (normal working area) based on the shoulder position, as the ratio of movements of the worker's hands that are fulcrums around the shoulders. This makes it possible to accurately calculate the ratio of movements of the worker's hands that are fulcrums around the shoulders, based on the results of extracting the shoulder positions and hand positions from the image.

[0051] In the first embodiment, the normal working area is set based on the position of the worker's shoulders and a radius R determined from preset physical information (for example, height) of the worker. This makes it possible to set an appropriate normal working area using a simple method.

[0052] In the first embodiment, the camera 60 is installed above the worker, but instead of or in addition to this, the camera 60 may be installed beside the worker. By using the camera 60 installed beside the worker, it is possible to capture hand movements within a plane including the vertical direction and the front-to-back directions, and by setting a first range (normal working area) within this plane, it is possible to calculate the proportion of hand movements of the worker that are fulcrums at the shoulder, as in the first embodiment.

[0053] In the first embodiment described above, in step S34, the evaluation result of the busyness evaluation unit 17 is displayed on the display unit 93 and transmitted to the conveyor control device 20, but this is not limited to this, and the evaluation result may be only displayed or only transmitted to the conveyor control device 20.

[0054] In the first embodiment, when the determination unit 15 determines that either of the worker's hands is outside the normal work area in each image, the determination unit 15 adds an "outside normal work area flag" to the image, and the out-of-normal work area ratio calculation unit 16 calculates the ratio of images captured within a predetermined time period to which the "out-of-normal work area flag" is added. However, the present invention is not limited to this. For example, when the determination unit 15 determines that either of the worker's hands is outside the normal work area in each image, the determination unit 15 may associate and manage the identification information and time information of the image with the "out-of-normal work area flag," and the out-of-normal work area ratio calculation unit 16 may calculate the ratio of the identification information and time information of the images captured within a predetermined time period that are associated with the "out-of-normal work area flag."

[0055] (Variation 1) In the first embodiment, the positions of both shoulders and both hands are extracted as feature points in step S18 of Fig. 5, and it is determined in step S20 whether at least one of the hands is outside the normal working area. However, the present invention is not limited to this. For example, instead of Fig. 5, the processes of Fig. 8 and Fig. 9 may be executed.

[0056] In the process of FIG. 8, after the image acquisition unit 13 acquires an image in step S16, the processes of steps S18A, S20A, and S22A and the processes of steps S18B, S20B, and S22B are executed simultaneously in parallel. That is, in step S18A, the feature point extraction unit 14 extracts the feature points (positions of the left shoulder and left hand) of the worker from the image acquired by the image acquisition unit 13. In addition, in step S20A, the determination unit 15 determines whether or not the left hand is outside the normal work area (left). If the determination result of step S20A is "yes," the process proceeds to step S22A, where the determination unit 15 assigns an outside-normal work area flag (left) to the image. Meanwhile, in step S18B, the feature point extraction unit 14 extracts the feature points (positions of the right shoulder and right hand) of the worker from the image acquired by the image acquisition unit 13. Furthermore, in step S20B, the determination unit 15 determines whether or not the right hand is outside the normal working area (right). If the determination result in step S20B is "yes," the process proceeds to step S22B, where the determination unit 15 assigns an outside normal working area flag (right) to the image. Note that both the outside normal working area flag (left) and the outside normal working area flag (right) may be assigned to the image. Note that if the shoulder position hardly moves, the feature point extraction unit 14 may not extract the positions of both shoulders in steps S18A and S18B from the second time onwards, and may instead reuse the positions of both shoulders extracted in the first time.

[0057] After steps S18A, S20A, and S22A and steps S18B, S20B, and S22B are simultaneously processed in parallel, the process proceeds to step S24, where the outside-of-normal-work-area calculation unit 16 determines whether or not a predetermined number of images have been acquired. If the determination in step S24 is negative, the process returns to step S16. However, if the determination in step S24 is positive, the process proceeds to step S26A in FIG. 9. In step S26A, the outside-of-normal-work-area calculation unit 16 calculates the proportion of images to which the outside-of-normal-work-area flag (left) has been assigned among the acquired images. That is, the outside-of-normal-work-area calculation unit 16 calculates the proportion of images outside the normal work area (left hand) based on the following equation (3): Percentage of images outside the normal working area (left hand) = (number of images with the out-of-normal working area flag (left) / number of images acquired) × 100 (%) ... (3)

[0058] In the next step S26B, the outside-of-normal-work-area proportion calculation unit 16 calculates the proportion of images to which an outside-of-normal-work-area flag (right) has been assigned among the acquired images. That is, the outside-of-normal-work-area proportion calculation unit 16 calculates the proportion outside of the normal work area (right hand) based on the following equation (4). Percentage of images outside the normal working area (right hand) = (number of images with the out-of-normal working area flag (right) / number of acquired images) × 100 (%) ... (4)

[0059] Next, in step S28', it is determined whether the proportion outside the normal working area (left hand) and the proportion outside the normal working area (right hand) satisfy a predetermined criterion for "busy."

[0060] Various criteria can be set for determining whether a person is "busy." For example, the following criteria can be set:

[0061] (a) At least one of the percentage outside the normal working area (left hand) and the percentage outside the normal working area (right hand) exceeds the threshold. (b) Both the percentage outside the normal working range (left hand) and the percentage outside the normal working range (right hand) exceed the threshold. (c) The average of the percentage outside the normal working range (left hand) and the percentage outside the normal working range (right hand) exceeds the threshold. (d) The sum of the percentage outside the normal working range (left hand) and the percentage outside the normal working range (right hand) exceeds the threshold. (e) The value obtained by substituting the percentage outside the normal working range (left hand) and the percentage outside the normal working range (right hand) into a predetermined formula exceeds a predetermined threshold.

[0062] In this modification, the busyness assessment unit 17 may also assess the degree of busyness in three or more stages, instead of assessing the busyness in two stages, busy / not busy.

[0063] Other processes are the same as those in the first embodiment (FIG. 5). Even when Modification 1 is adopted, the same effects as those in the first embodiment can be obtained.

[0064] (Variation 2) In the first embodiment, the positions of the worker's shoulders and hands are extracted using the camera 60. However, this is not limiting. For example, a sensor device such as an IMU (Inertial Measurement Unit) may be attached to the worker's body, and the positions of the worker's shoulders and hands may be extracted based on the detection results of the sensor device. When an IMU is used, for example, an IMU is attached to each of the worker's shoulders, upper arms, forearms, and hands. Then, the dimensions of the upper arms, forearms, and hands are calculated based on the worker's physique information (e.g., height), and the distance between the shoulders and hands is calculated based on these dimensions and angle information detected by the IMU. In this case, if the calculated distance between the shoulders and hands is greater than the radius R, it is determined that the hands are outside the normal working area. In this way, even when this second modification is adopted, the same effects as those of the first embodiment can be obtained.

[0065] Second Embodiment Next, the second embodiment will be described in detail with reference to Fig. 10 to Fig. 14. The manual sorting system of the second embodiment has the same configuration as that of Fig. 1, but the functions and processing contents of the busyness evaluation device 10 are different.

[0066] Fig. 10 shows a functional block diagram of a busyness assessment device 10 according to the second embodiment. As can be seen by comparing Fig. 10 with Fig. 3, the busyness assessment device 10 of the second embodiment has a shoulder non-mobilization range determination unit 112, a feature point movement amount extraction unit 114, a flag assignment unit 115, and a shoulder mobilization proportion calculation unit 116 instead of the normal work area determination unit 12, the feature point extraction unit 14, the judgment unit 15, and the outside-normal work area proportion calculation unit 16 of the first embodiment (Fig. 3) (see the bold frame in Fig. 10).

[0067] The shoulder non-mobilization range determination unit 112 determines the shoulder non-mobilization range based on the worker's physique information (e.g., height) acquired by the physique information acquisition unit 11. Here, the "shoulder non-mobilization range" refers to the range within which the worker's hand can move when moving the hand with the elbow as the center (fulcrum). In manual sorting work, when the hand is moved with the elbow as the center, the shoulder non-mobilization range is a fan-shaped range (the range surrounded by a thick solid line) as shown in FIG. 11. In FIG. 11, "○" indicates the position of the elbow, and "●" indicates the position of the hand. Here, because the range of shoulder rotation for Japanese people is approximately 140°, the central angle of the shoulder non-mobilization range in FIG. 11 is set to 140°.

[0068] The range Ly of the shoulder non-mobilization range in the front-to-back direction (Y direction) in FIG. 11 is Ly = L(1 - sin(10°)). Therefore, the shoulder non-mobilization range determination unit 112 determines the range Ly by calculating the length L from the elbow to the hand from the height. Furthermore, the range Lx of the shoulder non-mobilization range in the left-to-right direction (X direction) is Lx = L(cos(10°) + sin(60°)). Therefore, the shoulder non-mobilization range determination unit 112 determines the range Lx from the length L.

[0069] In the second embodiment, if the amount of forward and backward movement of the hand exceeds Ly (= L(1-sin(10°)) or if the amount of left and right movement of the hand exceeds Lx (= L(cos(10°) + sin(60°))), it is considered that the hand has moved with the shoulder as the center (fulcrum).

[0070] The feature point movement amount extraction unit 114 generates time series data of the forward / backward movement amount of the left hand, time series data of the left lateral movement amount of the left hand, time series data of the forward / backward movement amount of the right hand, and time series data of the right lateral movement amount of the right hand from images acquired over a predetermined time period. For example, data such as that shown in FIG. 12(a) is generated as time series data of the forward / backward movement amount of the left or right hand, and data such as that shown in FIG. 12(b) is generated as time series data of the forward / backward movement amount of the left or right hand. Note that the vertical axis of FIG. 12(a) represents the forward / backward position relative to the initial position (the position of the hand obtained from the first image captured within the predetermined time period). In other words, the graph of FIG. 12(a) can be said to show how the hand moves forward / backward and the change in the forward / backward movement amount of the hand over time. Furthermore, the vertical axis of FIG. 12(b) represents the left / right position relative to the initial position. In other words, the graph of FIG. 12(b) can be said to show how the hand moves forward / backward and the change in the left / right movement amount of the hand over time.

[0071] The flag assigning unit 115 identifies extreme values ​​from each piece of time-series data and identifies the length between adjacent extreme values ​​(called amplitude) (see FIG. 12(a)). Then, for each identified amplitude, the flag assigning unit 115 determines whether it exceeds the shoulder non-mobilization range (Lx or Ly), and assigns a "shoulder non-mobilization flag" to amplitudes that do not exceed the shoulder non-mobilization range (Lx or Ly). In addition, the flag assigning unit 115 assigns a "shoulder mobilization flag" to amplitudes that exceed the shoulder non-mobilization range (Lx or Ly).

[0072] The shoulder mobilization ratio calculation unit 116 calculates the ratio of amplitudes to which a "shoulder mobilization flag" is assigned (referred to as a "shoulder mobilization ratio") among all amplitudes included in each time series data (amplitudes to which a "shoulder mobilization flag" or a "shoulder non-mobilization flag" is assigned). The shoulder mobilization ratio can be said to be the ratio of movements of the worker's hands that use the shoulder as a fulcrum. The busyness evaluation unit 17 evaluates the busyness of the worker based on the ratio calculated by the shoulder mobilization ratio calculation unit 116.

[0073] (Regarding the processing of the busyness assessment device 10) The processing of the busyness assessment device 10 according to the second embodiment will be described in detail below. Figures 13 and 14 are flowcharts showing the processing content of the busyness assessment device 10 according to the second embodiment. Note that the flowcharts in Figures 13 and 14 are performed for each worker (i.e., for each camera 60). Therefore, when multiple workers work at the same time, processing corresponding to each of the multiple workers is executed simultaneously in parallel.

[0074] When the process starts, first, in step S50 of FIG. 13, the physique information acquiring unit 11 acquires physique information (for example, height) of the worker.

[0075] Next, in step S52, the shoulder non-mobilization range determination unit 112 determines the shoulder non-mobilization range (Lx, Ly) from the physique information (height). Note that if the physique information acquisition unit 11 has acquired the length (L) from the worker's elbow to the hand as physique information, the shoulder non-mobilization range (Lx, Ly) can be determined using the length L.

[0076] Next, in step S54, the image acquisition unit 13 waits until the worker starts the manual sorting work, and when the manual sorting work starts, in step S56, the image acquisition unit 13 acquires an image (one still image or one frame of a video) taken by the camera 60. This image acquisition is repeatedly executed until a predetermined time (a time corresponding to the time series data in Figures 12(a) and 12(b)) has elapsed (step S58). Then, when the predetermined time has elapsed since the start of image acquisition, the process proceeds to step S60.

[0077] In step S60, the feature point movement amount extraction unit 114 extracts the worker's feature points (positions of both hands) from each image acquired by the image acquisition unit 13, and generates time-series data on the amount of movement of both hands in the forward / backward direction and time-series data on the amount of movement in the left / right direction. Thereafter, the processes of steps S62A to S68A (referred to as forward / backward direction flag assignment process) and steps S62B to S68B (referred to as left / right direction flag assignment process) are repeatedly executed simultaneously in parallel until the determination in step S70 is affirmative.

[0078] (Front and rear direction flag assignment process (S62A to S68A)) In the front-rear direction flagging process, first, in step S62A, the flagging unit 115 identifies one amplitude of the front-rear direction time-series data. Next, in step S64A, the flagging unit 115 determines whether the identified amplitude exceeds the range Ly. If the determination in step S64A is positive, the process proceeds to step S66A, where the flagging unit 115 assigns a shoulder mobilization flag to the identified amplitude. On the other hand, if the determination in step S64A is negative, that is, if the identified amplitude is equal to or less than the range Ly, the process proceeds to step S68A, where the flagging unit 115 assigns a shoulder non-mobilization flag to the identified amplitude.

[0079] (Left / right direction flag assignment process (S62B to S68B)) In the left-right direction flag assignment process, first, in step S62B, the flag assignment unit 115 identifies one amplitude of the left-right direction time-series data. Next, in step S64B, the flag assignment unit 115 determines whether the identified amplitude exceeds the range Lx. If the determination in step S64B is positive, the process proceeds to step S66B, where the flag assignment unit 115 assigns a shoulder mobilization flag to the identified amplitude. On the other hand, if the determination in step S64B is negative, that is, if the identified amplitude is equal to or less than the range Lx, the process proceeds to step S68B, where the flag assignment unit 115 assigns a shoulder non-mobilization flag to the identified amplitude.

[0080] The flag assigning unit 115 repeatedly performs the forward / backward direction flag assigning process (S62A to S68A) and the left / right direction flag assigning process (S62B to S68B) and when all amplitudes have been identified (when flags have been assigned to all amplitudes), the determination in step S70 becomes positive and the process proceeds to step S72 in Fig. 14. When the process proceeds to step S72, flags have been assigned to each amplitude of the time series data of the forward / backward direction of the left hand, each amplitude of the time series data of the left left direction, each amplitude of the time series data of the forward / backward direction of the right hand, and each amplitude of the time series data of the left left direction.

[0081] In step S72, the shoulder recruitment ratio calculation unit 116 calculates the shoulder recruitment ratio based on the flag assigned to the amplitude. For example, the shoulder recruitment ratio calculation unit 116 counts up the flags assigned to all the time-series data, and calculates the shoulder recruitment ratio based on the following equation (5). Shoulder mobilization ratio = (number of shoulder mobilization flags / total number of flags) × 100 (%) ... (5)

[0082] Next, in step S74, the busyness evaluation unit 17 determines whether the shoulder mobilization ratio calculated in step S72 exceeds a predetermined threshold. If the threshold is, for example, 50(%), the determination in step S74 is positive if the calculated shoulder mobilization ratio exceeds 50%, and negative if it is 50% or less.

[0083] If the determination in step S74 is affirmative, the process proceeds to step S76, where the busyness assessment unit 17 determines that the worker is "busy." On the other hand, if the determination in step S74 is negative, the process proceeds to step S78, where the busyness assessment unit 17 determines that the worker is "not busy."

[0084] After step S76 or S78, the process proceeds to step S80, where the display control unit 18 displays the evaluation result (the determination result of step S76 or S78) on the display unit 93, and the transmission unit 19 transmits the evaluation result to the conveyor control device 20. Next, in step S82, similar to step S36 in Fig. 5, the image acquisition unit 13 determines whether the worker has finished the manual sorting work. If the determination in step S82 is negative, the process returns to step S56 in Fig. 13, but if the determination in step S82 is positive, all of the processes in Figs. 13 and 14 are terminated.

[0085] As can be seen from the above description, in the second embodiment, the image acquisition unit 13 and the feature point movement amount extraction unit 114 function as an acquisition unit that acquires information related to changes in the position of the worker's hand. Furthermore, the flag assignment unit 115 and the shoulder recruitment ratio calculation unit 116 function as an identification unit that identifies the ratio of movement of the worker's hand that is centered around the shoulder, based on the information acquired by the acquisition unit.

[0086] As described above in detail, in the busyness assessment device 10 of the second embodiment, the feature point movement amount extraction unit 114 generates time-series data of the movement amount of the worker's hand from images captured by the camera 60. Furthermore, the flag assignment unit 115 and the shoulder mobilization proportion calculation unit 116 calculate the proportion of the worker's hand movements that involve shoulder-based fulcrums (shoulder mobilization proportion) based on the generated time-series data. The busyness assessment unit 17 then evaluates the busyness of the worker based on the calculated proportion. As a result, in the second embodiment, the worker is determined to be busier the higher the proportion of shoulder-based fulcrum movements, and therefore it is possible to appropriately evaluate the busyness of a worker in work that requires manual movement, such as manual sorting.

[0087] In the second embodiment, the shoulder recruitment ratio calculation unit 116 calculates the ratio of movements of the worker's hands in a predetermined direction whose movement amount exceeds a threshold as the shoulder recruitment ratio. This makes it possible to accurately calculate the ratio of movements of the worker's hands that use the shoulder as a fulcrum.

[0088] In step S72, the shoulder recruitment ratio calculation unit 116 aggregates flags assigned to all time-series data and calculates the shoulder recruitment ratio from equation (5) above. However, the present invention is not limited to this. For example, the shoulder recruitment ratio calculation unit 116 may aggregate flags assigned to time-series data of the left hand to calculate the shoulder recruitment ratio of the left hand, and may aggregate flags assigned to time-series data of the right hand to calculate the shoulder recruitment ratio of the right hand. In this case, the busyness evaluation unit 17 may determine that the user is "busy" when both the left-hand shoulder recruitment ratio and the right-hand shoulder recruitment ratio exceed a threshold, or may determine that the user is "busy" when at least one of the left-hand shoulder recruitment ratio and the right-hand shoulder recruitment ratio exceeds a threshold. The busyness evaluation unit 17 may also determine that the user is "busy" when a value (such as a total or average value) calculated from the left-hand shoulder recruitment ratio and the right-hand shoulder recruitment ratio exceeds a threshold.

[0089] Furthermore, for example, the shoulder mobilization ratio calculation unit 116 may calculate the shoulder mobilization ratio in the up-down direction by aggregating flags assigned to the time-series data of both hands in the up-down direction, and may calculate the shoulder mobilization ratio in the left-right direction by aggregating flags assigned to the time-series data of both hands in the left-right direction. In this case, the busyness evaluation unit 17 may determine that the user is "busy" when both the shoulder mobilization ratio in the up-down direction and the shoulder mobilization ratio in the left-right direction exceed a threshold, or may determine that the user is "busy" when at least one of the shoulder mobilization ratios exceeds a threshold. Furthermore, the busyness evaluation unit 17 may determine that the user is "busy" when a value (such as a total value or an average value) calculated from the shoulder mobilization ratio in the up-down direction and the shoulder mobilization ratio in the left-right direction exceeds a threshold.

[0090] Also, for example, the shoulder mobilization ratio calculation unit 116 may calculate the shoulder mobilization ratio from each of the four time-series data. In this case, the busyness evaluation unit 17 may determine that the person is "busy" when all of the shoulder mobilization ratios exceed a threshold, or may determine that the person is "busy" when at least one of the shoulder mobilization ratios exceeds a threshold. Furthermore, the busyness evaluation unit 17 may determine that the person is "busy" when a value (such as a total value or an average value) obtained from the calculated shoulder mobilization ratios exceeds a threshold.

[0091] The busyness assessment unit 17 may assess the degree of busyness in three or more stages, instead of assessing the busyness in two stages, busy / not busy.

[0092] In the second embodiment, the shoulder mobilization rate is calculated as the rate at which the amount of hand movement in the forward / backward and left / right directions exceeds a threshold value. However, the present invention is not limited to this, and the shoulder mobilization rate may be calculated as the rate at which the amount of hand movement in the up / down direction exceeds a threshold value. In this case, the camera 60 may be installed next to the worker.

[0093] In the second embodiment, the camera 60 is used to extract the movement amount of the worker's hand. However, the present invention is not limited to this. For example, a sensor device such as an IMU may be attached to the worker's body, and the movement amount of the worker's hand may be extracted based on the detection results of the sensor device. The sensor device may be capable of detecting the acceleration and velocity of the hand, and time-series data on the movement amount of the hand may be generated by integrating the acceleration and velocity of the hand. Alternatively, a vector (movement amount + direction) indicating the movement of the hand may be acquired from the camera 60 or the sensor device, and the shoulder recruitment rate may be calculated based on the time change in the movement amount of the hand obtained from the vector. Alternatively, time-series data on the movement with the shoulder as the fulcrum may be extracted from the position, velocity, and acceleration amplitude of the hand, and the shoulder recruitment rate may be calculated from the extracted time-series data.

[0094] In the above embodiments and modifications, the busyness assessment device 10 has been described as assessing the busyness of workers who manually sort agricultural produce, but the present invention is not limited to this. The busyness assessment device 10 may also assess the busyness of workers working on other lines, such as manufacturing lines or waste disposal lines.

[0095] The above processing functions can be realized by a computer. In this case, a program is provided that describes the processing contents of the functions that the processing device should have. By executing the program on a computer, the above processing functions are realized on the computer. The program that describes the processing contents can be recorded on a computer-readable storage medium (excluding carrier waves).

[0096] When distributing a program, it is sold in the form of a portable storage medium on which the program is recorded, such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory).The program can also be stored in the storage device of a server computer and transferred from the server computer to other computers via a network.

[0097] A computer that executes a program stores, for example, a program recorded on a portable storage medium or a program transferred from a server computer in its own storage device. The computer then reads the program from its own storage device and executes processing in accordance with the program. Note that the computer can also read the program directly from a portable storage medium and execute processing in accordance with that program. The computer can also execute processing in accordance with the program received each time a program is transferred from the server computer.

[0098] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this and can be modified in various ways without departing from the spirit of the present invention. [Explanation of symbols]

[0099] 10 Busyness assessment device 13 Image acquisition unit (part of the acquisition unit) 14 Feature point extraction unit (part of the acquisition unit) 15 Judgment section (part of the identification section) 16 Calculation of percentage outside normal working area (part of specific section) 17 Busyness Evaluation Department (Evaluation Department) 18 Display control unit (notification unit) 20 Conveyor control device (control device) 50 Transport equipment (equipment) 114 Feature point movement amount extraction unit (part of acquisition unit) 115 Flagging section (part of specific section) 116 Shoulder mobilization rate calculation section (part of specific section)

Claims

1. Obtaining information about changes in the worker's hand position; Based on the acquired information, a proportion of the hand movements of the worker that are centered around the shoulder is identified; evaluating the busyness of the worker based on the identified ratio; A busyness evaluation program that causes a computer to execute a process.

2. In the acquiring process, information regarding a change in the position of the worker's hand is acquired based on an image capturing result of an image capturing device that captures an image of the worker. In the process of specifying, a proportion of the hand positions that are within a second range that is outside a first range based on the shoulder position is specified as a proportion of the hand movements of the worker that are centered around the shoulder.

2. The busyness evaluation program according to claim 1.

3. 3. The busyness evaluation program according to claim 2, wherein the first range is set based on the position of the worker's shoulders identified from the imaging results and predetermined information regarding the worker's physique.

4. In the acquiring process, information on a change over time in the amount of movement of the worker's hand in a predetermined direction is acquired; In the identifying process, a proportion of movements of the worker's hand in the predetermined direction in which the movement amount exceeds a predetermined amount is identified as a proportion of movements of the worker's hand with the shoulder as a fulcrum, based on information on changes over time in the movement amount.

2. The busyness evaluation program according to claim 1.

5. 5. The busyness evaluation program according to claim 4, wherein the predetermined amount is set based on information about the physical build of the worker that is set in advance.

6. 5. The busyness evaluation program according to claim 4, wherein the acquiring step acquires information on the change in the amount of movement over time from an imaging device that captures an image of the worker.

7. 2. The busyness evaluation program according to claim 1, wherein in the evaluation process, the worker is determined to be busy when the ratio exceeds a predetermined threshold value.

8. an acquisition unit that acquires information regarding changes in the position of the worker's hands; an identification unit that identifies a proportion of hand movements of the worker that are centered around a shoulder, based on the information acquired by the acquisition unit; an evaluation unit that evaluates the busyness of the worker based on the ratio identified by the identification unit; A busyness assessment device comprising:

9. The busyness evaluation device according to claim 8 , further comprising a notification unit that notifies information based on the evaluation result of the evaluation unit.

10. The busyness assessment device according to claim 8 ; a control device that controls equipment used in the work of the worker based on the evaluation result by the evaluation unit of the busyness evaluation device; A control system comprising:

Images