Sensor arrangement presentation device and sensor arrangement presentation method

Abstract

The present invention provides technology for presenting an optimal sensor arrangement according to changes in to-be-measured work details, the work environment, workers, and the like. The present invention comprises: a storage unit in which sensor information and measurement necessity scores are stored; a sensor arrangement candidate generation unit that generates sensor arrangement candidates in accordance with the sensor information; a visibility score calculation unit that calculates a visibility score for each sensor arrangement candidate; a measurement score calculation unit that calculates measurement scores from the measurement necessity scores and the visibility scores; an arrangement determination unit that, on the basis of the measurement scores, determines a recommended sensor arrangement from among the sensor arrangement candidates; and an arrangement presentation unit that presents, to a user, the recommended sensor arrangement determined by the arrangement determination unit.

Description

Sensor Arrangement Suggestion Device and Sensor Arrangement Suggestion Method

[0001] The present invention relates to a sensor arrangement suggestion device and a sensor arrangement suggestion method.

[0002] Conventionally, the operation of an operator has been analyzed by utilizing sensors, and equipment has been controlled based on the analysis results. At that time, it takes a great deal of effort to determine the arrangement of the sensors. As a technology for assisting in determining the arrangement of sensors, there is one that assists in the design of the ceiling layout of a building (see Patent Document 1). Patent Document 1 describes creating layout data indicating the layout of sensors based on sensor information, building information, equipment information, and indoor layout information. Further, Patent Document 1 describes calculating an evaluation score for predetermined evaluation indicators such as comfort, energy saving, and cost, and creating layout data by utilizing the calculated evaluation score.

[0003] Japanese Patent Application Laid-Open No. 2014-74948

[0004] By the way, the analysis results of the operation of an operator utilizing sensors can also be used, for example, to reduce work errors and transfer the skills of skilled workers. For this purpose, it is necessary to acquire the operation of the operator as completely as possible to improve the accuracy of the analysis results. Therefore, it is conceivable to determine the arrangement of sensors using the technology described in Patent Document 1.

[0005] However, for example, when arranging sensors in a narrow space such as a maintenance work space of a device, the probability that the operator himself / herself hides the operation of the operator or the operator deviates from the viewing angle of the sensor increases. Also, the optimal sensor arrangement changes depending on the work content to be measured and the body part of the operator. Furthermore, the optimal sensor arrangement may change depending on differences in the physique, dominant hand, and work environment of the operator. Therefore, with the technology described in Patent Document 1 that utilizes an evaluation score for predetermined evaluation indicators, there is a possibility that the sensors cannot be appropriately arranged so as to sufficiently acquire the operation of the operator.

[0006] This invention was made to solve the above problems. One of the objectives of this invention is to provide a technology that suggests the optimal sensor placement in response to changes in the work content to be measured, the work environment, the worker, etc.

[0007] One sensor placement presentation device for solving the above problems is a sensor placement presentation device that presents the placement of sensors for measuring the movements of an operator to a user, and comprises: a storage unit that stores sensor information relating to the sensors and a measurement necessity score indicating how much importance should be placed on measuring the part of the operator to be measured; a sensor placement candidate generation unit that generates sensor placement candidates according to the sensor information; a visibility score calculation unit that calculates a visibility score for each sensor placement candidate indicating whether or not the part to be measured can be measured by the sensor; a measurement score calculation unit that calculates a measurement score from the measurement necessity score and the visibility score; a placement determination unit that determines a recommended sensor placement from among the sensor placement candidates based on the measurement score; and a placement presentation unit that presents the recommended sensor placement determined by the placement determination unit to the user.

[0008] According to the present invention, in worker measurement, the optimal arrangement can be presented even if the task to be measured, the environment, or the worker changes. The effects described herein are not necessarily limited, and any of the effects described in this disclosure may be present.

[0009] This is a diagram illustrating the usage of the sensor placement display device according to the embodiment. This is a diagram showing the functional blocks of the sensor placement display device according to the embodiment. This is a diagram showing an example of the measurement necessity score to be set for each task according to the embodiment. This is a diagram showing an example of the measurement necessity score to be set for each work action according to the embodiment. This is a diagram showing an example of the measurement necessity score to be set for each time period of work action according to the embodiment. This is a diagram showing an example of the measurement necessity score to be set for each body part of the worker according to the embodiment. This is a diagram showing an example of the visibility score for each task according to the embodiment. This is a diagram showing an example of the visibility score for each work action according to the embodiment. This is a diagram showing an example of the visibility score for each time period of work action according to the embodiment. This is a diagram showing an example of the visibility score for each body part of the worker according to the embodiment. This is a flowchart showing an example of the processing of the simulation unit according to the embodiment. This is a diagram showing an example of a method for calculating the visibility score using light rays according to the embodiment. This is a diagram showing an example of a method for calculating the visibility score using a cone-shaped object according to the embodiment. This is a diagram showing an example of a method for calculating the visibility score using a cone-shaped object and a movement trajectory object according to the embodiment. This is a diagram showing an example of the placement content presented to the placement display unit according to the embodiment.

[0010] Embodiments of the present invention will be described in detail below with reference to the drawings. Figure 1 is a diagram illustrating the usage of a sensor placement display device according to an embodiment. In other words, Figure 1 is a diagram showing an example of the positional relationship between a sensor, an operator, a target device, and an analysis device at a location where a sensor is placed using the sensor placement display device according to this embodiment.

[0011] The sensor placement display device (which may also be called a sensor placement display system) 1 shown in Figure 1 is intended to be connected to an analysis device (which may also be called an analysis system) 3 for analyzing data obtained from the placed sensors 2. However, the sensor placement display device 1 can be connected to any device or system, in particular to a system that requires sensor data (for example, a data storage system). The sensor placement display device (sensor placement display system) 1 may also include the analysis device 3.

[0012] As shown in Figure 1, multiple sensors 2, in this example sensors 2A and 2B, are placed in the location where the target device 5, which is the object of work performed by worker 4, is installed. Sensors 2A and 2B are installed, for example, on the ceiling or walls of the space where the target device 5 is installed. The number of sensors 2 is not particularly limited and may be three or more, or it may be just one. In this example, worker 4 is a single worker, and worker 4 performs various tasks, such as maintenance on the target device 5, which is a semiconductor manufacturing device. Sensors 2A and 2B sense the movements of worker 4 during work and output the sensing data to the analysis device 3.

[0013] The analysis device 3 analyzes the movements of worker 4 based on sensing data input from sensor 2. By analyzing the movements of worker 4, it is possible to extract movement characteristics from worker 4, for example, or to issue warnings when incorrect movements are made.

[0014] The present invention aims to provide an optimized arrangement of sensors 2 so as to improve the accuracy of the analysis performed by the analysis device 3. For example, if the arrangement of sensors 2A and 2B is inappropriate, the movements of the worker 4 may be obscured by the worker 4 themselves. More specifically, if sensors 2A and 2B are positioned behind the worker 4, and the worker 4's hands are obscured by the worker 4 themselves, sensors 2A and 2B may not be able to properly sense the movements of the worker 4. As a result, the analysis device 3 may not be able to properly analyze the movements of the worker 4. Furthermore, the appropriate arrangement of sensors 2 may change depending on the worker 4's physique and dominant hand. In addition, the appropriate arrangement of sensors 2 may change depending on the working environment around the target device 5 (for example, the lighting environment, the presence and position of furniture installed around the target device 5, etc.).

[0015] Furthermore, the sensing data emphasized in the analysis performed by the analysis device 3 changes depending on the purpose of the analysis. For example, the sensing data emphasized changes if the work performed by the analysis device 3 changes, or if the body part (also called the measurement target part) of the worker 4 performing the analysis changes. Thus, the optimal position of the sensor 2 may also change due to changes in the sensing data emphasized.

[0016] The objective of the present invention is to provide the user with information for appropriately positioning the sensor 2 even when the worker 4, the work environment, and the work being analyzed change as described above. This will enable the sensor 2 to appropriately sense the movements of the worker 4. As a result, the accuracy of the analysis results obtained by the analysis device 3 can be improved.

[0017] The placement of sensor 2 in the appropriate location based on the provided information may be done by worker 4 or a third party. Alternatively, sensor 2 may be automatically placed by a robot or the like. Furthermore, the above information is provided, for example, when worker 4, the work environment, or the work content changes as described above.

[0018] Figure 2 is a functional block diagram of a sensor placement display device according to an embodiment of the present invention. The sensor placement display device 1 comprises a storage unit 10, a simulation unit 11, a placement content generation unit 12, a placement display unit 13, and a placement output unit 14.

[0019] The memory unit 10 stores work information 101, sensor information 102, measurement necessity score 103, worker information 104, work environment information 105, past worker action information 106, etc. The memory unit (storage device) 10 is configured to include, for example, flash memory or a hard disk. The information stored in the memory unit 10 must be saved before the start of the simulation performed by the simulation unit 11. This information is input by, for example, an external device (input device) not shown and stored in the memory unit 10.

[0020] Work information 101 is information about each task performed by the worker 4 when carrying out various processes on the target device 5. Work information 101 includes, for example, the expected sequence of tasks and the expected time required for each task. In addition, work information 101 may include information about the target device 5. Information about the target device 5 includes, for example, a 3D model of the target device 5 and information about the movable parts of the target device 5. Furthermore, if various sensors are attached to the target device 5, information about the areas where these sensors can be attached is also included in the information about the target device 5.

[0021] Sensor information 102 is information about each sensor 2 to be optimized. The information about sensor 2 includes, for example, the type of sensor 2, the sensing range of sensor 2, the size of sensor 2, the weight of sensor 2, and the 3D model of sensor 2. In the example in Figure 1, sensor information 102 is information about sensors 2A and 2B. Here, the information about each sensor 2A and 2B included in sensor information 102 may be information about multiple types of sensors, or information about sensors of the same type. Also, if sensors 2A and 2B are of the same type, sensor information 102 may include information about individual differences between each sensor 2A and sensor 2B.

[0022] The measurement necessity score 103 is a score that indicates how much importance the analysis device 3 places on each task performed by the worker 4 on the target device 5, and can be set arbitrarily by the user. The measurement necessity score 103 may be set independently for each task, or it may be set for each of the multiple actions performed in each task, or it may be set for each time period of the multiple actions performed in each task.

[0023] Figures 3 to 6 show examples of measurement necessity scores. As shown in Figure 3 as an example, the processing performed on the target device 5 can be broken down into several tasks, and a measurement necessity score can be set independently for each task. This example shows the maintenance process of a semiconductor control device broken down into several tasks, and examples of tasks include "detaching and attaching part X," "cleaning the target device," and "installing part X." The measurement necessity score is set to a higher value the higher the importance placed by the analysis device 3.

[0024] Furthermore, as shown in Figure 4, each task can be further broken down into multiple actions by worker 4, and a measurement necessity score can be set for each action. In the example in Figure 4, the actions for the task of "cleaning the target device" are set as "wiping part Y" and "wiping part Z," and the measurement necessity score is set higher for the action of "wiping part Z."

[0025] Furthermore, as shown in Figure 5, a measurement necessity score can be set for each time period of the multiple operations shown in Figure 4. In the example in Figure 5, each operation of the cleaning work (wiping work) is broken down into multiple time periods: "start of wiping," "during wiping," and "end of wiping," and the measurement necessity score is changed for each of these time periods.

[0026] For example, in the wiping process for semiconductor manufacturing equipment, the effectiveness of the wiping process can vary greatly depending on how the wiping begins and ends. Therefore, it is preferable to accurately analyze the effects of the "beginning of wiping" and "end of wiping" using the analysis device 3. As shown in the example in Figure 5, by setting the measurement necessity score for "beginning of wiping" and "end of wiping" higher than that for "during wiping," the actions of the worker 4 during the "beginning of wiping" and "end of wiping" periods can be sufficiently sensed. As a result, the effectiveness of the wiping process can be analyzed with high accuracy using the analysis device 3. In particular, by sufficiently sensing the actions of the worker 4 during the "end of wiping" period, the effectiveness of the wiping process can be analyzed with even higher accuracy using the analysis device 3.

[0027] Furthermore, as shown in Figure 6, the measurement necessity score 103 may be set for each body part (measurement target part) of the worker 4 that is subject to measurement. More specifically, for each measurement target part of the worker 4, a measurement necessity score may be set for each time period of each movement (work movement) performed in each task. In the example shown in Figure 6, a measurement necessity score 103 is set for the worker 4's "dominant hand," "non-dominant hand," and "head." The measurement necessity score 103 may also be set in more detail. For example, a measurement necessity score 103 may be set for each finger of the worker 4's hand.

[0028] By setting a measurement necessity score 103 for each part of the worker 4 that is to be measured, the sensor placement can be optimized so that the sensor 2 can sense the parts of the measurement target that are important for the analysis, even if the parts of the measurement target differ depending on the analysis performed by the analyzer 3.

[0029] Returning to Figure 2, the worker information 104 stored in the memory unit 10 is information about worker 4. The worker information 104 includes numerical data such as worker 4's height and weight. The worker information 104 may also include a three-dimensional model that reproduces worker 4's body. In addition, the worker information 104 may include information about worker 4's dominant hand and foot, gait, and characteristics of their movements (such as habits).

[0030] The work environment information 105 is environmental information for the work performed by worker 4. The work environment information 105 includes, for example, information on the location and brightness of lighting installed around the target device 5, and information on furniture (such as steps) installed around the target device 5. The work environment information 105 may also include information on the location and orientation of the target device 5. Furthermore, the work environment information 105 may include information on areas around the target device 5 that people are not allowed to enter.

[0031] Past worker motion information 106 is information about the worker 4's movements during past work. Preferably, the information about worker 4's movements during past work is that of the worker 4 themselves, but it may also be that of another worker. Past worker motion information 106 may include information that records the positional relationship of each joint at each time point during the work. Furthermore, past worker motion information 106 may also be animation information of a 3D model that reproduces the body of worker 4 included in worker information 104.

[0032] Furthermore, the past worker motion information 106 may be information on multiple different actions, or it may be information on an average of multiple actions. If the past worker motion information 106 is information on an average of multiple actions, statistically determined probability information from the multiple actions may be added. In addition, the past worker motion information 106 may be created by motion capturing the worker 4 that will perform the work in advance. Alternatively, the past worker motion information 106 may be manually input via an input device (not shown) and stored in the storage unit 10.

[0033] Next, the simulation unit 11 includes a worker motion reproduction unit 111, a device state reproduction unit 112, a sensor placement candidate generation unit 113, a visibility score calculation unit 114, a measurement score calculation unit 115, and a placement determination unit 116.

[0034] The simulation unit 11 generally performs the following processes. Based on the information stored in the memory unit 10, the simulation unit 11 reproduces the workspace, the movements of the worker 4, and the arrangement of multiple sensors 2 in three-dimensional space to generate candidate sensor placements. For the generated candidate sensor placements, the simulation unit 11 calculates a visibility score to determine how well each sensor 2 can sense the target area of ​​the worker 4. Furthermore, the simulation unit 11 independently calculates a measurement score for each candidate sensor placement, indicating whether the desired target area of ​​the worker 4 can be measured, based on the calculated visibility score and the measurement necessity score 103 stored in the memory unit 10. The sensor placement with the highest measurement score is then determined as the recommended sensor placement.

[0035] Here, Figures 7 to 10 show an example of the visibility score. The visibility score is calculated in correspondence with each of the measurement necessity scores described above. For example, if the measurement necessity score is set independently for each of the tasks performed on the target device 5, by breaking down the process into several tasks (see Figure 3), then the visibility score is also calculated independently for each task, as shown in Figure 7.

[0036] Furthermore, if each task is further broken down into multiple actions of worker 4, and a measurement necessity score is set for each action (see Figure 4), the visibility score is also calculated for each action, as shown in Figure 8. In addition, if a measurement necessity score is set for each time period of multiple actions (see Figure 5), the visibility score is calculated for each time period of multiple actions, as shown in Figure 9. Also, if the measurement necessity score 103 is set for each body part of worker 4 that is subject to measurement (measurement target body part) (see Figure 6), the visibility score is also calculated for each body part of worker 4, as shown in Figure 10.

[0037] Figure 11 is a flowchart showing the processing procedure performed by the simulation unit 11. The functions of each part of the simulation unit 11 will be explained in accordance with the flowchart shown in Figure 11.

[0038] Step S401: The sensor placement candidate generation unit 113 executes the sensor placement candidate generation process. More specifically, the sensor placement candidate generation unit 113 generates one or more sensor placement candidates. The generated sensor placement candidates are stored as a sensor placement candidate list, for example, within the sensor placement candidate generation unit 113. Each sensor placement candidate generated by the sensor placement candidate generation unit 113 has a different position and installation angle of the sensor 2 in three-dimensional space. In this example, the sensor placement candidate generation unit 113 generates multiple sensor placement candidates based on, for example, work information 101, sensor information 102, work environment information 105, etc., stored in the storage unit 10. For example, locations where the sensor 2 can be installed are stored as, for example, work environment information 105.

[0039] The sensor placement candidate generation unit 113 generates sensor placement candidates by reconstructing the sensors 2 in three-dimensional space from the information of each sensor 2 included in the sensor information 102, but in doing so, it ensures that the sensors 2 are not placed in the sensor-mountable area of ​​the target device 5 included in the work information 101.

[0040] The sensor placement candidate generation unit 113 can combine a plurality of sensors 2 to form one sensor placement candidate. When information on a plurality of sensors of the same type is stored as the sensor information 102, the sensor placement candidate generation unit 113 can install a plurality of the same sensors 2 in the three-dimensional space. When information on a plurality of types of sensors is stored as the sensor information 102, the sensor placement candidate generation unit 113 can install a plurality of one or more types of sensors in the three-dimensional space.

[0041] Note that the number of sensor placement candidates generated by the sensor placement candidate generation unit 113, that is, the number of sensor placement candidates to be included in the sensor placement candidate list, is preferably input to the sensor placement candidate generation unit 113 in advance. Information on the number of sensor placement candidates generated by the sensor placement candidate generation unit 113 may be stored in the storage unit 10 as the sensor information 102.

[0042] The sensor placement candidate generation unit 113 can also refer to the past operator motion information 106 when generating a sensor placement candidate. For example, by referring to the motions of the past operator 4 included in the past operator motion information 106, when generating each sensor placement candidate, the direction of the sensor 2 can be determined in the direction where the probability of the presence of the operator 4 is high.

[0043] When the sensor placement candidate generation unit 113 arranges a plurality of sensors 2, the sensor placement candidate generation unit 113 may generate a sensor placement candidate so that the distance between the sensors 2 does not become less than a preset threshold value so that the positions of the sensors 2 do not get too close. Further, when the sensor 2 is composed of a camera or the like and there is a measurable range (measurable area) that can be measured by the sensor 2, it is preferable to determine the positions and directions of the respective sensors 2 so that the measurable areas of the respective sensors 2 do not overlap more than a preset set range (which can also be said to be a set ratio).

[0044] When the sensor placement candidate generation process by such a sensor placement candidate generation unit 113 is completed, the process by the simulation unit 11 transitions to step S402.

[0045] Step S402: The simulation unit 11 determines whether or not a sensor placement candidate exists in the sensor placement candidate list. If a sensor placement candidate exists in the sensor placement candidate list (Step S402: Yes), the process proceeds to step S403. For example, when the processing by the simulation unit 11 progresses from step S401 to step S402, a sensor placement candidate exists in the sensor placement candidate list generated by the sensor placement candidate generation unit 113, so the processing by the simulation unit 11 proceeds to step S403. On the other hand, if a sensor placement candidate is deleted from the sensor placement candidate list in a step described later, and there are no longer any sensor placement candidates in the sensor placement candidate list (Step S402: No), the processing by the simulation unit 11 proceeds to step S410.

[0046] Step S403: The simulation unit 11 retrieves one sensor placement candidate from the list of sensor placement candidates stored in the sensor placement candidate generation unit 113 and saves it in the simulation unit 11 as the current sensor placement candidate. After that, the processing by the simulation unit 11 proceeds to step S404.

[0047] Step S404: The worker motion reproduction unit 111 reproduces the worker 4's movements for each task in the same three-dimensional space as the sensor placement candidates, based on the work information 101, worker information 104, and past worker motion information 106, etc. At this time, the worker 4's movements may be reproduced for each time period, the movement trajectory of the worker 4's measurement target part may be reproduced as a three-dimensional object, or both may be reproduced. By reproducing the movement trajectory of the worker 4's measurement target part as a three-dimensional object, the processing time required to calculate the visibility score can be shortened.

[0048] The operator motion reproduction unit 111 searches for the past operator motions of the same operator and the same operation from the past operator motion information 106, and reproduces those motions as the motions of operator 4. When the past operator motions of the same operator do not exist in the past operator motion information 106, the past operator motions of an operator with a similar physique and dominant hand are searched from the past operator motion information 106, and those motions are reproduced as the motions of operator 4. When information on operators with similar physiques and dominant hands does not exist in the past operator motion information 106, or when the past operator motions of the same operation do not exist in the past operator motion information 106, the operator motion reproduction unit 111 reproduces the motions of operator 4 so that the motions are ergonomically reasonable. At this time, the operator information 104 is referred to, and the motions of operator 4 are reproduced so as to reflect the characteristics of the motions of operator 4.

[0049] Also, the operator motion reproduction unit 111 may reproduce the motions of operator 4 by referring to the work environment information 105. For example, when the work environment information 105 includes information on areas where people cannot enter, the operator motion reproduction unit 111 reproduces the motions of operator 4 so that operator 4 does not enter those areas.

[0050] Also, the operator motion reproduction unit 111 may reproduce the motions of a plurality of operator 4s for the same operation. For example, when there are a plurality of types of past operator motions for the same operation by the same operator in the past operator motion information 106, the operator motion reproduction unit 111 may reproduce a plurality of types of motions of operator 4 based on the past operator motion information 106.

[0051] Step S404: The device state reproduction unit 112 reproduces the state of the target device 5 for each operation on the same three-dimensional space as the operator motion reproduction space of the motions of operator 4 reproduced by the operator motion reproduction unit 111. The device state reproduction unit 112 can reproduce the state of the target device 5 by referring to the work information 101. The state of the target device 5 is, for example, the moving state or the detaching / attaching state of the movable parts or detachable parts of the target device when they exist.

[0052] The processing by the simulation unit 11 proceeds to step S405 once the worker motion reproduction processing by the worker motion reproduction unit 111 and the device state reproduction processing by the device state reproduction unit 112 are completed.

[0053] Step S405: The visibility score calculation unit 114 calculates the visibility score. The visibility score is a score that indicates how well the worker 4, which has been simulated and reproduced by the worker motion reproduction unit 111, can be sensed by the sensor 2 in the current candidate sensor placement. This visibility score can be calculated for each task, for each work action, or for each time period within a work action. The visibility score calculation unit 114 may also calculate the visibility score for each measurement target part of the worker 4, or it may calculate a single visibility score for the worker 4.

[0054] Here, we will explain a specific example of how the visibility score calculation unit 114 calculates the visibility score. Figure 12 is a diagram illustrating an example of calculating the visibility score for the right hand, which is the measurement target area of ​​the worker 4, and shows an example of the positional relationship between the worker 4's right hand 4a and the sensor 2 reproduced by the simulation unit 11.

[0055] The visibility score calculation unit 114, in order to calculate the visibility score, for example, as shown in Figure 12, emits a virtual ray L1 from the sensor 2 toward the measurement point Po on the worker's right hand 4a. The position of the measurement point Po on the right hand 4a is predetermined and stored in the storage unit 10 as, for example, work information 101.

[0056] The visibility score calculation unit 114 calculates the visibility score using a point-addition method, as an example. Specifically, as shown in the example in Figure 12, if there are no other objects on the light beam L1 other than the measurement point Po, the visibility score calculation unit 114 determines that the measurement point Po is visible to the sensor 2 and adds points to the visibility score of the right hand. The points to be added are predetermined for each part of the worker 4 that is measured, or for each measurement point Po, and are stored in advance in the memory unit 10, etc.

[0057] On the other hand, if there is another object other than the measurement point Po on the light ray L1, the visibility score calculation unit 114 determines that the measurement point Po is not visible to the sensor 2 and does not add any points to the visibility score. If the measurement point Po is set at multiple locations on the right hand 4a, the visibility score calculation unit 114 calculates a visibility score for each measurement point Po using an additive scoring method and calculates the visibility score of the right hand 4a based on the visibility scores at each measurement point Po. As an example, the sum of the visibility scores at each measurement point Po is calculated as the visibility score of the right hand 4a.

[0058] The method for calculating the visibility score by the visibility score calculation unit 114 does not necessarily have to be an additive method; for example, it may be a subtractive method. In the subtractive method, if there is another object other than the measurement point Po on the light beam L1, the visibility score calculation unit 114 determines that the measurement point Po is not visible to the sensor 2 and subtracts points from the visibility score of the right hand 4a. On the other hand, if there is no other object other than the measurement point Po on the light beam L1, the visibility score calculation unit 114 determines that the measurement point Po is visible to the sensor 2 and does not subtract points from the visibility score. The number of points to be subtracted is set in advance for each measurement target part of the worker 4 or for each measurement point Po, and is stored in advance in the memory unit 10, etc.

[0059] Furthermore, the visibility score calculation unit 114 may calculate the visibility score by combining an addition method and a subtraction method. The visibility score calculation unit 114 may determine that if there are no other objects on the light beam L1 other than the measurement point Po, the measurement point Po is visible to the sensor and add points to the visibility score of the right hand, and if there are other objects on the light beam L1 other than the measurement point Po, the measurement point Po is not visible to the sensor 2 and subtract points from the visibility score of the right hand 4a.

[0060] As another example, the visibility score calculation unit 114 can also calculate the visibility score using the field of view of the sensor 2 instead of light rays. Figure 13, similar to Figure 12, illustrates an example of calculating the visibility score for the right hand 4a, which is the measurement target area of ​​the worker 4, and shows an example of the positional relationship between the worker's right hand 4a and the sensor 2 reproduced by the simulation unit 11.

[0061] In the example shown in Figure 13, unlike the example shown in Figure 12, the visibility score calculation unit 114 places a virtual conical object Ob1 representing the field of view of the sensor 2, and calculates the visibility score based on the surface area or volume of the region A1 where the conical object Ob1 and the right hand 4a overlap.

[0062] The cone-shaped object Ob1 may also be a square pyramid object, a cylindrical object, or a rectangular prism object. Furthermore, this cone-shaped object Ob1 does not necessarily have to match the actual field of view of sensor 2; for example, only the field of view region where sensor 2 has high accuracy may be set as the cone-shaped object Ob1.

[0063] The visibility score calculation unit 114, as an example, calculates the surface area or volume of region A1 where the conical object Ob1 and the right hand 4a overlap, and multiplies this surface area or volume by a certain coefficient to calculate the visibility score. In this case, the larger the surface area or volume of region A1, the larger the absolute value of the visibility score. However, the visibility score calculation unit 114 may also use the surface area or volume of region A1 where the conical object Ob1 and the right hand 4a overlap directly as the visibility score without applying a coefficient.

[0064] Furthermore, the visibility score calculation unit 114 may deduct points from the visibility score if another object other than the right hand overlaps with the conical object Ob1, and that other object is in front of the right hand 4a as viewed from the sensor 2. In this case, the surface area or volume of the overlapping region between the conical object Ob1 and the other object other than the right hand 4a can be calculated, and the calculated surface area or volume multiplied by a certain coefficient can be subtracted from the visibility score. Alternatively, the visibility score calculation unit 114 may subtract the calculated surface area or volume directly from the visibility score without applying a certain coefficient.

[0065] Furthermore, if the worker motion reproduction unit 111 reproduces the trajectory (body part trajectory) of the worker's 4 body part (measurement target part) as a 3D object, the visibility score calculation unit 114 may calculate the visibility score using the reproduced 3D object. Figure 14 is a diagram illustrating an example of calculating the visibility score using a 3D object of a body part trajectory, where the trajectory (body part trajectory) of the worker's 4 right hand 4a is represented as a 3D object. As shown in Figure 14, the visibility score calculation unit 114 finds the surface area or volume of the region A2 where the right hand trajectory object Ob2, which is a movement trajectory object, and the cone-shaped object Ob1 overlap. The visibility score calculation unit 114 may add a value obtained by multiplying the found surface area or volume by a predetermined coefficient to the visibility score. Alternatively, the visibility score calculation unit 114 may add the found surface area or volume directly to the visibility score without multiplying by a predetermined coefficient.

[0066] Furthermore, the visibility score calculation unit 114 may deduct points from the visibility score if the three-dimensional object, which includes all of the worker's movements in each task, cannot be measured by the sensor 2 by a certain value (a certain percentage). For example, if the percentage of the entire right-hand trajectory object Ob2 that is visible from the sensor 2 is smaller than a threshold, the visibility score calculation unit 114 may deduct points from the visibility score.

[0067] Furthermore, the visibility score calculation unit 114 may calculate the visibility score by combining the methods shown in Figures 12 to 14. In this case, the visibility score calculation unit 114 may interrupt the visibility score calculation process midway if it determines that the visibility score will not ultimately exceed a predetermined value. For example, the visibility score calculation unit 114 may first calculate the visibility score using the right-hand trajectory object Ob2 (see Figure 14), interrupt the calculation process if it determines that the visibility score will not exceed a predetermined value, and only calculate the visibility score using the ray L1 if it determines that the visibility score will exceed a predetermined value (see Figure 12).

[0068] Calculating the visibility score using the light ray L1 takes more computation time than calculating the visibility score using the right-hand trajectory object Ob2. Therefore, if the visibility score does not exceed a specified value, that is, if it is determined that the current sensor placement candidate is not a recommended sensor placement, the overall processing time can be shortened by interrupting the visibility score calculation process. In addition, if the calculation process is interrupted, the visibility score calculation unit 114 can also set the visibility score of the current sensor placement candidate to 0. This prevents the placement determination unit 116, which will be described later, from mistakenly selecting the sensor placement candidate for which the visibility score calculation was interrupted as a recommended sensor placement.

[0069] Furthermore, if sensor information 102 contains information about sensors corresponding to specific body parts of the worker 4, the visibility score calculation unit 114 may use that information to calculate the visibility score. For example, if the sensor information 102 contains information about a hand motion sensor capable of measuring hand movements, the unit may determine that the shape and movement of the fingers can always be measured and add a certain additional score to the visibility score of the fingers, which are the measurement target area.

[0070] Furthermore, the visibility score calculation unit 114 may calculate a single visibility score by combining multiple sensors 2. For example, if information from the hand motion sensor is included in the sensor information 102, and the measurement point of the worker's 4 wrist is visible from the sensor 2, which is a camera, it can be determined that the entire hand is visible. This example can be used when the relative position of the fingers to the worker's 4 wrist can be obtained by the hand motion sensor, but the absolute position cannot be obtained. If the absolute position of the worker's 4 wrist can be measured by the sensor 2, the absolute position of the fingers can be determined from the absolute position of the wrist measured by the sensor 2 and the relative position of the fingers to the wrist measured by the hand motion sensor, even if the fingers are not visible from the sensor 2.

[0071] Furthermore, if the worker motion reproduction unit 111 reproduces multiple worker motions for the same task, the visibility score calculation unit 114 may calculate a visibility score for each worker motion.

[0072] Once the visibility score calculation unit 114 has completed the calculation of the visibility score, the simulation unit 11 proceeds to step S406.

[0073] Step S406: The measurement score calculation unit 115 calculates the measurement score. The measurement score is a score that indicates how well the parts that need to be measured (target parts to be measured) can be measured in the current candidate sensor placement. Specifically, the measurement score is calculated from the visibility score calculated by the visibility score calculation unit 114 and the measurement necessity score 103 stored in the storage unit 10.

[0074] For example, if the visibility score calculation unit 114 calculates a visibility score for each task, the measurement score calculation unit 115 can multiply the measurement necessity score for each task, as shown in Figure 3, by the visibility score for each task, as shown in Figure 7, and use the sum of the multiplied scores as the measurement score.

[0075] Furthermore, for example, if the visibility score calculation unit 114 calculates a visibility score for each operation, the measurement score calculation unit 115 can multiply the measurement necessity score for each operation within each task, as shown in Figure 4, by the visibility score for each operation within each task, as shown in Figure 8, and use the sum of the multiplied scores as the measurement score.

[0076] Furthermore, for example, if the visibility score calculation unit 114 calculates a visibility score for each time period within the operation, the measurement score calculation unit 115 can multiply the measurement necessity score for each time period within the operation, as shown in Figure 6, by the visibility score for each time period within the operation, as shown in Figure 9, and use the sum of the multiplied scores as the measurement score.

[0077] Furthermore, the measurement score calculation unit 115 may calculate the measurement score using a different calculation method based on the visibility score and the measurability score. For example, the measurement score calculation unit 115 may calculate the measurement score using a trained model generated using a machine learning algorithm including deep learning. Alternatively, the measurement score calculation unit 115 may calculate the measurement score using some kind of two-variable function.

[0078] Furthermore, if the worker motion reproduction unit 111 reproduces multiple worker motions for the same task, and the visibility score calculation unit 114 calculates a visibility score for each worker motion, the measurement score calculation unit 115 calculates a measurement score for each worker motion. For example, if an additive scoring method is used to calculate the visibility score, the measurement score calculation unit 115 can use the highest score among the measurement scores for each worker motion as the measurement score for the current sensor placement candidate. Also, for example, if a subtractive scoring method is used to calculate the visibility score, the measurement score calculation unit 115 can use the lowest score among the measurement scores for each worker motion as the measurement score for the current sensor placement candidate.

[0079] In this case, the information on the current candidate sensor placement may include, for example, not only information on the placement of sensor 2, but also information on worker movements and the position of worker 4 (hereinafter also referred to as worker position). The visibility score calculation unit 114 may then calculate a visibility score for each position of worker 4 in the simulation.

[0080] Once the measurement score calculation unit 115 has completed calculating the measurement score, the simulation unit 11 proceeds to step S407.

[0081] Step S407: The placement determination unit 116 determines a recommended sensor placement from the list of sensor placement candidates based on the measurement score of each sensor placement candidate. More specifically, the placement determination unit 116 determines a recommended sensor placement based on the difference between the measurement score of each sensor placement candidate and a preset threshold. In this example, in step S407, the placement determination unit 116 compares the measurement score of the current recommended score placement with the measurement score of the current sensor placement candidate and selects either the current recommended sensor placement or the current sensor placement candidate. In other words, the placement determination unit 116 determines whether the current sensor placement candidate is better than the current recommended sensor placement. In this example, the measurement score of the current recommended score placement can be said to correspond to the threshold.

[0082] For example, if an additive scoring method is used to calculate the visibility score, the placement determination unit 116 will select the one with the higher measurement score from the current recommended score placement and the current sensor placement candidates. Also, for example, if a subtractive scoring method is used to calculate the visibility score, the placement determination unit 116 will select the one with the lower measurement score from the current recommended score placement and the current sensor placement candidates. As a result, the placement determination unit 116 will select the sensor placement candidate with the highest measurement score among the sensor placement candidates as the recommended sensor placement.

[0083] Furthermore, if no recommended sensor placement is saved, the placement determination unit 116 determines that the current sensor placement candidate is good. However, even if no recommended sensor placement is saved, the placement determination unit 116 does not determine that the current sensor placement candidate is good if the visibility score calculation unit 114 has interrupted the calculation of the visibility score as described above.

[0084] In this example, one of the sensor placement candidates in the sensor placement candidate list is designated as the recommended sensor placement, but multiple sensor placement candidates may be designated as the recommended sensor placement. For example, if a point-based scoring method is used to calculate the visibility score, the placement determination unit 116 may designate all of the multiple sensor placement candidates included in the sensor placement candidate list whose measured scores are equal to or greater than a threshold as recommended score placements. In this case, in step S407, the placement determination unit 116 compares the measured score of the current sensor placement candidate with the threshold, and if the measured score of the current recommended score placement is equal to or greater than the threshold, it saves the current sensor placement candidate as one of the recommended sensor placements.

[0085] Once the arrangement determination unit 116 has completed its determination of whether the current sensor arrangement candidate is better than the current recommended sensor arrangement, the simulation unit 11 proceeds to step S408 or step S409.

[0086] Step S408: If the placement determination unit 116 determines in step S407 that the current candidate sensor placement is better than the current recommended sensor placement (step S407: Yes), the simulation unit 11 proceeds to step S408, and the placement determination unit 116 saves the current candidate sensor placement as the current recommended sensor placement.

[0087] If the placement determination unit 116 completes the process of saving the current recommended sensor placement in step S408, the simulation unit 11 proceeds to step S409. Also, if the placement determination unit 116 does not determine in step S407 that the current sensor placement candidate is better than the current recommended sensor placement (step S407: No), the simulation unit 11 proceeds to step S409.

[0088] Step S409: The simulation unit 11 deletes the current sensor placement candidate, that is, the sensor placement candidate extracted in step S403, from the list of sensor placement candidates stored in the sensor placement candidate generation unit 113. After that, the processing by the simulation unit 11 proceeds to step S402. In step S402, as described above, it is determined whether or not a sensor placement candidate exists in the list of sensor placement candidates. If the current sensor placement candidate is deleted from the list of sensor placement candidates in step S409 and there are no longer any sensor placement candidates in the list of sensor placement candidates (step S402: No), the processing by the simulation unit 11 proceeds to step S410.

[0089] Step S410: The placement determination unit 116 outputs the current recommended sensor placement to the placement content generation unit 12 and the placement output unit 14. This completes the series of processes performed by the simulation unit 11. If no current recommended sensor placement exists in step S410, the unit outputs to the placement content generation unit 12 and the placement output unit 14 that no current optimal placement exists.

[0090] When the placement content generation unit 12 receives information on recommended sensor placement from the placement determination unit 116, it generates placement content to present recommended sensor placement to the user of the sensor placement presentation device 1, for example, a person who will install the sensor 2 in the real space, based on the input information, and outputs it to the placement presentation unit 13. The placement content consists of content such as text, still images, moving images, and audio. The placement content may consist of a single piece of content or a combination of multiple pieces of content.

[0091] The placement presentation unit 13 presents the placement content generated by the placement content generation unit 12 to the user. The placement presentation unit 13 is composed of a display 130 and a speaker (not shown), for example, as shown in Figure 14. In the example shown in Figure 14, the placement content 120, along with the sensor placement diagram, is displayed on the display 130, along with the images from each of the sensors 2A and 2B, which are cameras.

[0092] Furthermore, if the placement content generation unit 12 receives input from the placement determination unit 116 indicating that no recommended sensor placement exists, it generates placement content indicating that no optimal sensor placement exists. For example, it can generate a message as placement content in text format, such as "No optimal sensor placement exists. Please add sensors."

[0093] The placement display unit 13 may be, for example, a mobile device such as a smartphone or tablet, or a personal computer. Furthermore, the content of the placement content 120 is not particularly limited. For example, the sensor placement diagram may only show the recommended sensor placement, or if information on the current sensor placement is available, it may show both the recommended and current sensor placements. Additionally, audio explanations may be output from a speaker.

[0094] Furthermore, the placement content 120 may be presented interactively to the user of the sensor placement display device 1, for example, the person who places the sensor 2 in the real world. In this case, the sensor placement display device 1 may be equipped with an input function to receive input from the user and may be able to change the placement content 120 based on the user's input. For example, the person who places the sensor 2 in the real world may be able to change the placement of the sensor 2 on the placement content 120, and a warning message may be displayed if the sensor 2 is placed in an inappropriate location.

[0095] The input function may include, for example, input functions using a keyboard or mouse, as well as input functions using voice input and voice recognition, eye-tracking input, or hand gesture recognition. For example, the placement content generation unit 12 may place input interfaces such as buttons on the placement content 120. When a user makes an input to an input interface on the placement content 120, the placement presentation unit 13 outputs that information to the placement content generation unit 12.

[0096] Furthermore, the placement display unit 13 may be composed of a wearable device such as a head-mounted display or smart glasses. In addition, the placement display unit 13 may be composed of an AR (Augmented Reality) device. If the placement display unit 13 is composed of an AR device, the placement content generation unit 12 can also generate placement content that places the sensor 2 in a virtual space that reproduces the real space and highlights the sensor 2 in that virtual space.

[0097] Incidentally, the sensor placement display device 1 is a device that primarily presents information regarding the placement of sensors 2 to the user, but it may also present additional information to the user along with the information regarding the placement of sensors 2.

[0098] For example, along with information regarding the placement of sensor 2, information such as worker movements and worker position can also be presented to the user, worker 4. For instance, if information such as worker movements and worker position is added to the recommended sensor placement information input from placement determination unit 116, the placement content generation unit 12 can also generate content to present the worker movements and worker position information to worker 4.

[0099] For example, if the placement presentation unit 13 is composed of an AR device (augmented reality device) such as smart glasses, the placement content generation unit 12 reproduces worker movements in a virtual space that replicates the real space and generates content that plays the worker movements in accordance with the worker movements in the real space. By presenting such content to the worker 4 using the placement presentation unit 13, the optimal actions for the worker 4 can be communicated. Furthermore, if information such as the worker's position is added to the recommended sensor placement information, the worker 4's standing position can be highlighted in the virtual space that replicates the real space, thereby suggesting a more appropriate standing position to the worker 4.

[0100] The placement output unit 14 outputs information such as recommended sensor placement input from the placement determination unit 116, or information such as worker position, to an external device such as the analysis device 3. The placement output unit 14 is equipped with wired communication means such as Ethernet (trademark registered) and USB (trademark registered), and wireless communication means such as Wi-Fi (trademark registered) and Bluetooth (trademark registered), and can output information such as recommended sensor placement or worker position to an external device such as the analysis device 3 or an external system using these means.

[0101] Furthermore, the functions of the sensor placement suggestion device 1 and the analysis device 3 described above can also be implemented as software. If an external device such as the analysis device 3 is operating on the same hardware as the sensor placement suggestion device 1, the placement output unit 14 can output recommended sensor placement information to the external device such as the analysis device 3 via a storage device provided on that hardware.

[0102] As described above, the sensor placement display device and sensor placement display method according to the present invention can provide the user with information for installing sensors in an appropriate configuration, even when the worker, work environment, or the work being analyzed changes. By optimally arranging the sensors based on the presented information, the user can improve the accuracy of the analysis performed by the analysis device.

[0103] The present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are described in detail for the purpose of clearly illustrating the present invention, and are not necessarily limited to those having all the configurations described.

[0104] In the above-described embodiment, an example was given in which the visibility score is calculated based on the results of a simulation performed by the simulation unit 11. However, the visibility score does not necessarily have to be calculated based on the results of a simulation. The visibility score can also be calculated using, for example, actual work video footage by the worker 4. In this case, the visibility score calculation unit 114, the measurement score calculation unit 115, and the placement determination unit 116 do not need to be provided by the simulation unit 11.

[0105] Furthermore, although the above-described embodiment explains an example in which the placement determination unit 116 determines the recommended sensor placement based on a measurement score calculated from the measurement necessity score and the visibility score, the method for determining the recommended sensor placement is not limited to this. The placement determination unit 116 can also determine the recommended sensor placement from the visibility score without using the measurement necessity score.

[0106] 1...Sensor placement display device (sensor placement display system), 2...Sensor, 3...Analysis device (analysis system), 10...Storage unit, 101...Work information, 102...Sensor information, 103...Measurement necessity score, 104...Worker information, 105...Work environment information, 106...Past worker action information, 11...Simulation unit, 111...Worker action reproduction unit, 112...Device state reproduction unit, 113...Sensor placement candidate generation unit, 114...Visibility score calculation unit, 115...Measurement score calculation unit, 116...Placement determination unit, 12...Placement content generation unit, 13...Placement display unit, 14...Placement output unit

Claims

1. A sensor placement presentation device for presenting to a user the arrangement of sensors for measuring the movements of an operator, comprising: a storage unit that stores sensor information relating to the sensors and a measurement necessity score indicating how important it is to measure the part of the operator to be measured; a sensor placement candidate generation unit that generates sensor placement candidates according to the sensor information; a visibility score calculation unit that calculates a visibility score for each of the sensor placement candidates indicating whether or not the part to be measured can be measured by the sensors; a measurement score calculation unit that calculates a measurement score from the measurement necessity score and the visibility score; a placement determination unit that determines a recommended sensor placement from among the sensor placement candidates based on the measurement score; and a placement presentation unit that presents the recommended sensor placement determined by the placement determination unit to the user.

2. A sensor placement suggestion device according to claim 1, wherein the placement determination unit determines the recommended sensor placement based on the difference between the measurement score and a preset threshold.

3. A sensor placement display device according to claim 1, wherein the measurement necessity score is set according to the time period of work performed by the worker.

4. A sensor placement display device according to claim 1, wherein the storage unit stores work information relating to work performed by the worker, the device includes a worker motion reproduction unit that reproduces the worker's movements by simulation based on the work information, and the visibility score calculation unit calculates the visibility score based on the simulation performed by the worker motion reproduction unit.

5. A sensor placement display device according to claim 4, wherein the storage unit stores worker information, and the worker motion reproduction unit simulates the worker's motion based on the work information and the worker information.

6. A sensor placement display device according to claim 5, wherein the storage unit stores work environment information, and the worker motion reproduction unit simulates the worker's motion based on the work information and the work environment information.

7. A sensor placement display device according to claim 4, wherein the worker motion reproduction unit performs a plurality of simulations corresponding to the worker's position, and the visibility score calculation unit calculates the visibility score at each position of the worker in the simulation.

8. A sensor placement display device according to claim 4, wherein the storage unit stores past worker action information, and the worker action reproduction unit reproduces the worker's actions from the worker's past action data included in the past worker action information.

9. A sensor placement display device according to claim 1, wherein the storage unit stores work information relating to work performed by the worker, and the device state reproduction unit reproduces the state of the device that is the target of work performed by the worker by simulation based on the work information, and the visibility score calculation unit calculates the visibility score based on the simulation by the device state reproduction unit.

10. A sensor placement display device according to claim 1, wherein the visibility score calculation unit determines that the shape of the hand can always be measured by the hand motion sensor when the sensor information includes information from a hand motion sensor, and adds a predetermined additional score to the visibility score when the part of the worker to be measured is the hand.

11. A sensor placement presentation device according to claim 1, comprising: a placement content generation unit that generates content for presenting the recommended sensor placement determined by the placement determination unit to the user, wherein the placement presentation unit presents the content generated by the placement content generation unit to the user.

12. Sensor placement presentation device according to claim 11, wherein the placement content generation unit generates content that places and highlights the sensors on a virtual space that reproduces a real space, and the placement presentation unit presents the content to the user using an augmented reality device.

13. A sensor placement display device according to claim 1, wherein the visibility score calculation unit deducts points from the visibility score if a three-dimensional object including all of the worker's movements in the work performed by the worker cannot be measured by the sensor at a certain value or higher.

14. A sensor placement presentation device according to claim 1, further comprising a placement output unit that outputs at least one piece of information, namely the recommended sensor placement determined by the placement determination unit and the position of the worker, to an analysis device that analyzes the worker's movements.

15. A sensor placement presentation method for presenting to a user the arrangement of sensors for measuring the movements of an operator, comprising: a storage process for storing sensor information relating to the sensors and a measurement necessity score indicating how important it is to measure the part of the operator to be measured in a storage unit; a sensor placement candidate generation process for generating candidate sensor placements according to the sensor information; a visibility score calculation process for calculating a visibility score for each candidate sensor placement to indicate whether the sensor can measure the part of the operator to be measured; a measurement score calculation process for calculating a measurement score from the measurement necessity score and the visibility score; a placement determination process for determining a recommended sensor placement from among the candidate sensor placements based on the measurement score; and a placement presentation process for presenting the recommended sensor placement determined by the placement determination process to the user.

Images