Information processing method

Abstract

To reduce the cost in a system for evaluating movements of a worker.SOLUTION: An information processing method includes acquiring high-dimensional sensor data sensed in a first dimension and using first high-dimensional feature data obtained based on the high-dimensional sensor data to generate a first movement map. The method further includes generating low-dimensional sensor data obtained when sensing the position of a body part of a first worker in a second dimension and generating dimension conversion information that represents a correspondence between the first high-dimensional feature data and first low-dimensional feature data obtained based on the low-dimensional sensor data. The method further includes sensing a predetermined movement performed by a second worker in a second dimension to acquire low-dimensional sensor data and extracting corresponding high-dimensional feature data from the dimension conversion information. The method further includes evaluating a degree of agreement between the predetermined movement and a reference movement based on a result of comparing the first movement map with a second movement map generated using the extracted high-dimensional feature data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a technique for evaluating a worker's motion. [Background technology]

[0002] Many techniques for evaluating the movements of workers are known. In this regard, Patent Document 1 discloses a motion analysis device that acquires motion information including skeletal information indicating the positions of multiple body parts associated with the skeletons of multiple subjects, classifies the motion information based on the skeletal information, and analyzes the motions of the subjects. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-200241 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to reduce costs in a system for evaluating the actions of a worker. [Means for solving the problem]

[0005] One aspect of the present disclosure is to A method for generating a first motion map, which is a map showing a time transition of a plurality of postures, by obtaining high-dimensional sensor data that is data obtained by sensing positions of body parts of a first worker performing a reference motion in a first dimension, using first high-dimensional feature data that is a feature obtained based on the high-dimensional sensor data, and generating low-dimensional sensor data obtained by converting the high-dimensional sensor data when the positions of the body parts of the first worker are sensed in a second dimension lower than the first dimension, and acquiring second low-dimensional sensor data which is data obtained by sensing a predetermined action performed by a second worker in the second dimension, and extracting second high-dimensional feature data which is high-dimensional feature data corresponding to the second low-dimensional feature data obtained based on the second low-dimensional sensor data from the dimension conversion information; and evaluating a degree of correspondence between the predetermined action and the reference action based on a result of comparing the first action map with a second action map generated using the extracted second high-dimensional feature data. It is an information processing method.

[0006] Other aspects include an apparatus for executing the above-mentioned method, a program for executing the above-mentioned method, or a computer-readable storage medium non-transitoryly storing the program. Effect of the Invention

[0007] According to the present disclosure, it is possible to reduce costs in a system for evaluating the actions of a worker. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram of a process realized by an information processing device that executes an information processing method according to an embodiment. [Diagram 2]3 is a diagram showing a data flow used in the information processing method according to the embodiment; [Diagram 3] 1 is a diagram illustrating components of an information processing apparatus that executes an information processing method according to an embodiment. [Figure 4] 4 is a flowchart of a process executed by a control unit of an information processing device that executes an information processing method according to an embodiment. [Diagram 5] FIG. 11 is a diagram showing an example of dimension conversion information. [Figure 6] FIG. 11 is a diagram illustrating an example of a result of comparing a first operation map with a second operation map. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In a factory or the like, a series of actions of a worker performing work may be defined as a reference action. In such an environment, a system is known that analyzes the actions performed by a worker and evaluates the degree to which the actions match the reference action.

[0010] For example, the system acquires the three-dimensional coordinates of the worker's body parts using a sensor, generates motion information including skeletal information from the acquired sensor information, and classifies the motion information to analyze the worker's motion. Based on the three-dimensional sensor information, the system can detect the presence or absence of abnormalities in the worker's motion (for example, whether the worker is performing an unconventional motion).

[0011] However, this system requires the worker to always wear a 3D sensor when capturing the worker's movements. For example, the worker needs to wear multiple sensors of the motion capture device on each part of the body when performing the movement to be analyzed. In this case, it is troublesome for the worker to work while always wearing multiple sensors. It is also possible to optically detect the three-dimensional coordinates of a worker's body parts, but such systems are expensive and difficult to apply to environments where there are many workers to be evaluated. Therefore, it is desirable for the system to be able to capture and analyze the worker's movements without using a three-dimensional sensor. The information processing method according to the present embodiment solves this problem.

[0012] An information processing method according to one aspect of the present disclosure includes acquiring high-dimensional sensor data that is data obtained by sensing positions of body parts of a first worker performing a reference motion in a first dimension, generating a first motion map that is a map representing temporal progression of a plurality of postures using first high-dimensional feature data that is a feature obtained based on the high-dimensional sensor data, generating low-dimensional sensor data obtained when positions of body parts of the first worker are sensed in a second dimension lower than the first dimension by converting the high-dimensional sensor data, and converting the first high-dimensional feature data obtained based on the high-dimensional sensor data and the high-dimensional sensor data. generating dimension conversion information representing a correspondence relationship with first low-dimensional feature data obtained based on the low-dimensional sensor data converted from the data; acquiring second low-dimensional sensor data which is data obtained by sensing in the second dimension a predetermined action performed by a second worker, and extracting from the dimension conversion information second high-dimensional feature data which is high-dimensional feature data corresponding to the second low-dimensional feature data obtained based on the second low-dimensional sensor data; and evaluating a degree of correspondence between the predetermined action and the reference action based on a result of comparing the first action map with a second action map generated using the extracted second high-dimensional feature data.

[0013] The high-dimensional sensor data is sensor data that indicates a position of a body part of a first worker sensed in a first dimension. For example, the high-dimensional sensor data may be data that represents the position of a body part of a first worker in three-dimensional coordinates.

[0014] The first motion map represents a transition over time of a plurality of postures included in the motion of the first worker. The first motion map may be, for example, a map in which the horizontal axis represents time steps and the vertical axis represents a plurality of postures. The first operation map can be generated based on first high-dimensional feature amount data calculated based on the high-dimensional sensor data.

[0015] The low-dimensional sensor data is sensor data that is assumed to be obtained when the first worker is sensed in a second dimension (lower dimension). For example, if the first dimension is three dimensions, the low-dimensional sensor data is the sensor data that would be expected to be obtained by sensing the same worker in two dimensions (e.g., using a camera that has no depth information).

[0016] The dimension conversion information is a one-to-one correspondence between first high-dimensional feature data and first low-dimensional feature data calculated from low-dimensional sensor data obtained by converting the high-dimensional sensor data that is the basis of the first high-dimensional feature data.

[0017] The second low dimensional sensor data is data obtained by sensing the position of a body part of a second worker in a second dimension (lower dimension) lower than the first dimension. For example, the second low dimensional sensor data may be data obtained by sensing the position of a body part of a worker on a two-dimensional image.

[0018] The method according to the present disclosure extracts, from the dimension transformation information, high-dimensional feature data corresponding to second low-dimensional feature data that is a feature calculated from the second low-dimensional sensor data, and uses this to generate a second operation map.

[0019] The second movement map is a diagram showing the transition over time of multiple postures included in the movement of the second worker. The second movement map may be, for example, a map in which the horizontal axis represents time steps and the vertical axis represents multiple postures.

[0020] The information processing method disclosed herein calculates high-dimensional features from data obtained by sensing the positions of body parts of a first worker in a first dimension, and generates a first motion map. Also, calculates features from data obtained by sensing the positions of body parts of a second worker in a second dimension, converts the features into high-dimensional features, and generates a second motion map. By comparing these, it is possible to evaluate the degree of agreement between a reference motion performed by the first worker and a motion performed by the second worker.

[0021] When the dimensions of sensing are different during learning and evaluation, it is not possible to compare the two based on the features obtained at each time. However, the information processing method according to the present disclosure uses dimension conversion information to obtain high-dimensional features that cannot be obtained in a low-dimensional sensing environment.

[0022] As a result, the information processing method according to the embodiment of the present disclosure makes it possible to evaluate a change in the posture of the second worker using only the sensor data acquired in the second dimension (low dimension) without using the sensor data acquired in the first dimension (high dimension). Therefore, the information processing method according to the embodiment of the present disclosure makes it possible to evaluate the movement of the second worker at a lower cost.

[0023] Each of the first high-dimensional feature data, the first low-dimensional feature data, the second high-dimensional feature data, and the second low-dimensional feature data may be data representing a ranking of the distance between combined body parts for all combinations of two of the plurality of body parts of the worker.

[0024] Moreover, the information processing method according to the present disclosure may extract, from the dimension transformation information, the second high-dimensional feature data corresponding to the first low-dimensional feature data that is most similar to the second low-dimensional feature data.

[0025] As a result, the information processing method according to the embodiment of the present disclosure can efficiently extract the second high-dimensional feature data corresponding to the second low-dimensional feature data.

[0026] Moreover, the information processing method disclosed herein may generate the second action map for each of a plurality of the second workers.

[0027] In this way, the information processing method according to the embodiment of the present disclosure can evaluate the actions of a plurality of second workers.

[0028] In addition, in the information processing method disclosed herein, generating the first movement map may include classifying a plurality of postures of the first worker by clustering the first high-dimensional feature data, and generating the first movement map representing the temporal progression of the plurality of postures based on a result of the classification.

[0029] As a result, the information processing method disclosed herein can appropriately detect the posture of the first worker from the feature amount data.

[0030] Specific embodiments of the present disclosure will be described below with reference to the drawings. Unless otherwise specified, the hardware configuration, module configuration, functional configuration, and the like described in each embodiment are not intended to limit the technical scope of the disclosure to only those.

[0031] (Embodiment) An overview of processing performed by an information processing device that executes an information processing method according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a conceptual diagram of processing realized by an information processing device 100 that executes an information processing method according to an embodiment.

[0032] In the following description, "sensing in three dimensions" means acquiring sensor data having three-dimensional information (X, Y, Z coordinates) using a sensor capable of acquiring three-dimensional coordinates. In addition, "two-dimensional sensing" refers to acquiring sensor data with two-dimensional information (X, Y coordinates) using a two-dimensional sensor (for example, a camera that does not have depth information). It refers to doing something.

[0033] In this embodiment, in a learning phase, the information processing device 100 senses the movements of the first worker 10 in three dimensions. Then, the information processing device 100 determines the posture of the first worker 10 from features calculated from the three-dimensional sensed data, and generates a first movement map indicating the transition of the posture of the first worker 10. After that, the information processing device 100 obtains association between the feature amount calculated based on the sensor data obtained in three dimensions and the feature amount calculated based on the sensor data obtained in two dimensions.

[0034] On the other hand, in the evaluation phase, the information processing device 100 senses the movement of the second worker 20 in two dimensions. Based on the above-mentioned association, the information processing device 100 acquires feature amounts that are expected to be obtained when the same movement is sensed in three dimensions from feature amounts calculated from data obtained by sensing the movement of the second worker 20 in two dimensions. Then, the information processing device 100 determines the posture of the second worker 20 from the acquired feature amounts, and calculates the posture of the second worker 20. A second movement map showing the transition of the posture of the worker 20 is generated. Then, the information processing device 100 compares the first action map with the second action map to evaluate the degree of agreement between the action learned in three dimensions and the action sensed in two dimensions.

[0035] A more detailed explanation will be given of the processing performed by the information processing device 100. Fig. 2 is a diagram showing the flow of data used in the information processing method according to the embodiment. First, in a learning phase, the information processing device 100 uses multiple cameras 200 to three-dimensionally sense the movements of a first worker 10 wearing a marker 210. The first worker 10 wears a marker 210 on each part of the body (e.g., shoulder, elbow, knee, toe, back of the hand, top of the head, etc.). Then, multiple dedicated cameras 200 sense the positions of the markers 210 in accordance with the movements of the first worker 10. The results of sensing by the cameras 200 may be data that represents the position of each part of the worker's body by three-dimensional coordinates. This is called high-dimensional sensor data.

[0036] The information processing device 100 calculates a feature amount from the coordinates of each part of the body of the first worker 10 sensed in three dimensions. The feature amount may be, for example, data (hereinafter referred to as a rank feature amount) that indicates the rank order of the distance between the combined parts for all combinations of two parts among the multiple body parts of the first worker 10. For example, the rank feature amount may be a vector of values ​​corresponding to all combinations of two of the shoulder, elbow, and back of the hand. Specifically, for example, the rank feature amount may be a vector in which an element corresponding to the combination of the shoulder, elbow, and back of the hand with the longest distance between the combined parts is 1, and an element corresponding to the combination of the shoulder, elbow, and back of the hand with the second longest distance between the combined parts is 2. The feature amount extracted here is called first high-dimensional feature amount data. The feature amount becomes data representing the posture of the first worker 10 at a certain point in time.

[0037] Next, the information processing device 100 generates a first action map from the first high-dimensional feature data calculated for each time step. The information processing device 100 determines the posture of the first worker 10 from the first high-dimensional feature data. For example, the information processing device 100 may cluster the multiple first high-dimensional feature data obtained for each time step, and associate the multiple clusters obtained as a result with each of the multiple postures taken by the first worker 10. The first action map may be a plot of the transition of the posture of the first worker 10 for each time step, with the time step on the horizontal axis and the multiple postures on the vertical axis.

[0038] Next, the information processing device 100 calculates first low-dimensional feature data corresponding to the first high-dimensional feature data. First, the information processing device 100 converts the sensing data acquired in three dimensions into low-dimensional sensor data, which is sensing data that would be obtained when the same object is sensed in two dimensions. Then, the information processing device 100 calculates features from the converted low-dimensional sensor data. The features calculated here are defined as first low-dimensional feature data.

[0039] Then, the information processing device 100 stores the first high-dimensional feature data and the first low-dimensional feature data in association with each other. That is, the information processing device 100 stores a certain first high-dimensional feature data and a first low-dimensional feature data corresponding to the first high-dimensional feature data in association with each other.

[0040] Next, in the evaluation phase, the information processing device 100 senses the movement of the first worker 10 in two dimensions. In this embodiment, the information processing device 100 senses the position of each part of the body of the first worker 10 over time from a two-dimensional image of the first worker 10 captured by a camera shooting from one fixed viewpoint. The sensor data thus obtained is used as the second This is called low-dimensional sensor data.

[0041] Next, the information processing device 100 calculates second low-dimensional feature amount data from the second low-dimensional sensor data.

[0042] Next, the information processing device 100 acquires second high-dimensional feature data corresponding to the second low-dimensional feature data, based on the correspondence relationship between the first low-dimensional feature data and the first high-dimensional feature data stored in the learning phase.

[0043] Then, the information processing device 100 generates a second action map from the second high-dimensional feature amount data. The second action map is a map that represents a change in the posture of the worker over time, similar to the first action map.

[0044] Finally, the information processing device 100 compares the first action map with the second action map. Then, the information processing device 100 evaluates the degree of agreement between the first action map and the second action map. For example, the solid line 410 represents the posture of the first worker 10 represented in the first action map, and the dashed line 420 represents the posture of the first worker 10 represented in the second action map. The horizontal axis of the graph 400 represents the time step t. For example, the information processing device 100 may evaluate the degree of agreement between the solid line 410 and the dashed line 420 in the time step direction.

[0045] Next, a detailed description will be given of each element constituting a system including the information processing device 100. Fig. 3 is a diagram illustrating components of the information processing device 100 that executes the information processing method according to the embodiment.

[0046] The information processing device 100 according to this embodiment includes a control unit 110, a storage unit 120, and a communication unit 130. The information processing device 100 communicates via the communication unit 130 with a plurality of cameras 200 that sense markers 210, or with a camera that captures an image for sensing the movement of the first worker 10 in two dimensions.

[0047] The control unit 110 is realized by a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) and a memory. The control unit 110 includes, as functional modules, an acquisition unit 111, a learning unit 112, and an evaluation unit 113. These functional modules may be realized by the control unit 110 executing a program.

[0048] The acquisition unit 111 acquires high-dimensional sensor data. Here, the high-dimensional sensor data is a result of sensing one or more markers 210 by multiple cameras 200, and is data expressing the positions of body parts of the first worker 10 performing a reference motion in three-dimensional coordinates. The camera 200 is an example of a sensor for sensing in the first dimension. Furthermore, three dimensions are a specific example of the first dimension. The reference motion is a predetermined motion performed by the first worker 10, and may include multiple postures of the first worker 10. The reference motion can be, for example, a motion defined for a worker who assembles a predetermined product.

[0049] The acquisition unit 111 also acquires second low-dimensional sensor data. Here, the second low-dimensional sensor data is a result of sensing in two dimensions by a sensor other than the camera 200, which is a dedicated camera for motion capture, and is data expressing the positions of body parts of the second worker 20 performing a movement following the reference movement in two-dimensional coordinates. The sensor may be, for example, a camera sensor that acquires a two-dimensional image. Here, the two dimensions are a specific example of the second dimension.

[0050] The learning unit 112 generates a motion map representing changes in posture of the first worker 10 who has performed a reference motion, based on the results of sensing the first worker 10. First, the learning unit 112 generates first high-dimensional feature data based on the high-dimensional sensor data. The first high-dimensional feature data is a feature obtained from the high-dimensional sensor data. Then, the learning unit 112 determines a transition of the posture of the first worker 10 from the first high-dimensional feature data. The learning unit 112 may determine a plurality of postures of the first worker 10 included in the reference motion by clustering the first high-dimensional feature data. Then, the learning unit 112 generates a first motion map which is a map representing a transition over time of the determined plurality of postures.

[0051] Furthermore, for each piece of high-dimensional feature data, the learning unit 112 calculates first low-dimensional feature data, which is a feature obtained when sensing in two dimensions the same movement of the first worker 10 as the movement when the high-dimensional feature data was obtained. Specifically, the learning unit 112 converts the high-dimensional sensor data in three dimensions into low-dimensional sensor data, which is sensing data that would be obtained when sensing in two dimensions. Then, the learning unit 112 calculates a feature based on the low-dimensional sensor data, which is the first low-dimensional feature data.

[0052] Then, the learning unit 112 generates dimension conversion information in which the first low-dimensional feature amount data and the first high-dimensional feature amount data corresponding to the same high-dimensional sensor data are associated in a one-to-one manner, and stores the dimension conversion information in the storage unit 120.

[0053] The evaluation unit 113 generates a movement map representing the transition of the posture of the second worker 20 based on the result of sensing the second worker 20, and evaluates the movement of the second worker 20 based on this. The evaluation unit 113 calculates features from the second low-dimensional sensor data acquired by the acquisition unit 111, and sets the features as second low-dimensional feature data. Then, the evaluation unit 113 extracts second high-dimensional feature data corresponding to the second low-dimensional feature data based on the dimension conversion information. That is, the evaluation unit 113 converts the positions of each part of the body of the first worker 10 acquired in two dimensions into data when the positions are sensed in three dimensions, and extracts features of the data.

[0054] Furthermore, the evaluation unit 113 determines, from the second high-dimensional feature data, a plurality of postures of the second worker 20 who performed an action following the reference action. The evaluation unit 113 may determine the plurality of postures by clustering the second high-dimensional feature data. Then, the evaluation unit 113 generates a second motion map which is a map showing the time progression of the determined plurality of postures.

[0055] Then, the evaluation unit 113 compares the first action map with the second action map to evaluate the degree of agreement between the reference action of the first worker 10 and the action of the second worker 20. The evaluation unit 113 may give a higher evaluation to the action of the second worker 20 as the difference between the first action map and the second action map for each time step is smaller.

[0056] The storage unit 120 is a main storage device such as a RAM or a ROM, an EPROM, a hard disk drive, and an auxiliary storage device such as a removable medium. An operating system (OS), various programs, various tables, and the like are stored in the auxiliary storage device, and by executing the programs stored therein, each function that matches the predetermined purpose of each part of the control unit 110 can be realized. However, some or all of the functions may be realized by a hardware circuit such as an ASIC or an FPGA.

[0057] The storage unit 120 stores data and the like used or generated in the processing performed by the control unit 110.

[0058] The communication unit 130 is configured with a communication circuit for performing wireless communication. The communication unit 130 may be a communication circuit that performs wireless communication using 4th Generation (G) or 5th Generation (5G). The communication unit 130 may be a communication circuit that performs wireless communication using LTE (Long Term Evolution) or LPWA (Low Power Wide Area). The communication unit 130 may be a communication circuit that performs communication using a standard wireless communication area (Wi-Fi).

[0059] Next, a specific content of the process performed by the information processing device 100 that executes the information processing method according to the embodiment will be described. Fig. 4 is a flowchart of the process performed by the control unit 110 of the information processing device 100 that executes the information processing method according to the embodiment.

[0060] FIG. 4(A) is a flowchart of the processing executed by information processing device 100 in the learning phase, and FIG. 4(B) is a flowchart of the processing executed by information processing device 100 in the evaluation phase. In the example shown in Fig. 4, the process in Fig. 4(A) is performed, followed by the process in Fig. 4(B). The process in Fig. 4(B) does not have to be performed consecutively to the process in Fig. 4(A) as long as it is performed after the process in Fig. 4(A). First, the process in FIG.

[0061] First, in step S10, the acquisition unit 111 acquires high-dimensional sensor data by one or more first sensors. For example, the acquisition unit 111 acquires the results of sensing the markers 210 attached to each part of the body of the first worker 10 by the camera 200, which is a dedicated camera for motion capture.

[0062] Next, in step S11, the learning unit 112 generates first high-dimensional feature data based on the high-dimensional sensor data, and generates a first action map representing a time transition of multiple postures of the first worker 10 from the first high-dimensional feature data. The learning unit 112 generates the first high-dimensional feature data by extracting features from the high-dimensional sensor data. Then, the learning unit 112 determines multiple postures of the first worker 10 included in the generated first high-dimensional feature data. The posture determination may be performed by clustering the multiple first high-dimensional feature data. Next, the learning unit 112 calculates, from the high-dimensional feature data, a time-series transition of a plurality of postures determined to be included in the movement of the first worker 10, and represents the transition in a first movement map. Note that the first high-dimensional feature data calculated by the learning unit 112 may be a rank feature.

[0063] Here, after extracting the first high dimensional feature amount data, learning unit 112 may perform a dimensionality reduction process on the first high dimensional feature amount data before determining a plurality of postures from the first high dimensional feature amount data. For example, when the learning unit 112 collects position data for 15 body parts of the first worker 10, the rank feature for the 15 body parts is a combination of 15 elements, and therefore becomes a 105-dimensional vector. When N features, which are 105-dimensional vectors, are collected, the order of the data is 105×N. Therefore, PCA (Principal Inverse Analysis) is used. The features may be compressed into low-dimensional (e.g., two-dimensional) data using any of the following techniques: Multi-dimensional Component Analysis (MDS), Multi-dimensional scaling (MDS), t-distributed Stochastic Neighbor Embedding (t-SNE), Uniform Manifold Approximation and Projection (Umap), etc.

[0064] In addition, when the learning unit 112 is learning data about a plurality of first workers 10, the learning unit 112 performs a dimensionality reduction process on the first high-dimensional feature amount data, and then performs a first high-dimensional feature amount reduction process on the first high-dimensional feature amount data for each of the plurality of first workers 10. A shift in the time direction of the first high-dimensional feature data may be corrected. A method called DTW (Dynamic Time Warping) may be used to correct the shift in the time direction of the first high-dimensional feature data.

[0065] Next, in step S12, the learning unit 112 calculates first low-dimensional feature data corresponding to the first high-dimensional feature data. First, the learning unit 112 converts three-dimensional sensing data into low-dimensional sensor data, which is sensing data obtained when sensing is performed in two dimensions. Then, the learning unit 112 calculates low-dimensional features, which are features, from the converted low-dimensional sensor data as first low-dimensional feature data. Note that the first low-dimensional feature data calculated by the learning unit 112 may be rank features.

[0066] Next, in step S13, the learning unit 112 generates dimension conversion information in which the first low-dimensional feature amount data and the first high-dimensional feature amount data are associated with each other on a one-to-one basis.

[0067] Here, the information processing device 100 temporarily ends the process.

[0068] Next, the process of FIG. 4(B) will be described.

[0069] The process of step S14 in FIG. 4B is started at the timing when second worker 20 starts to move after the process of FIG. 4A is completed.

[0070] First, in step S14, the evaluation unit 113 obtains second high-dimensional feature data from the second low-dimensional feature data in the second dimension. Specifically, the acquisition unit 111 first acquires second low-dimensional sensor data that is a result of sensing the motion of the second worker 20 in the second dimension. Then, the evaluation unit 113 generates second low-dimensional feature data by calculating features from the second low-dimensional sensor data.

[0071] Then, the evaluation unit 113 extracts second high-dimensional feature data corresponding to the second low-dimensional feature data based on the dimension conversion information.

[0072] FIG. 5 is a diagram illustrating an example of the dimension conversion information. For example, as shown in Fig. 5, in the dimension conversion information in this embodiment, the feature of the sensing data of the action of the second worker 20 sensed in two dimensions is paired with the feature of the sensing data when the same action is sensed in three dimensions. For example, when the evaluation unit 113 obtains a feature F_2d(t) as low-dimensional feature data, it refers to the dimension conversion information to obtain F_3D(t), which is a feature corresponding to F_2d(t). The feature is a feature obtained when the same action as the action sensed in two dimensions is sensed in three dimensions.

[0073] If the dimension conversion information does not store features that completely match the second low-dimensional feature data, the evaluation unit 113 may extract first high-dimensional feature data corresponding to the first low-dimensional feature data that is most similar to the second low-dimensional feature data. The evaluation unit 113 may perform the above process using a nearest neighbor search method including LSH (Locality Sensitive Hashing).

[0074] After extracting the second high-dimensional feature amount data, evaluation unit 113 may perform a dimensionality reduction process on the second high-dimensional feature amount data before determining a plurality of postures from the second high-dimensional feature amount data.

[0075] Next, in step S16, the evaluation unit 113 compares the first operation map with the second operation map. For example, as shown in FIG. 6, the evaluation unit 113 generates a map by superimposing the first and second motion maps, and detects the difference between the first and second motion maps. FIG. 6 is a diagram showing an example of the result of comparing the first and second motion maps. A solid line 430 indicates a portion where the postures appearing in the first and second motion maps match in the corresponding time step. A dashed line 440 indicates a portion where the postures appearing in the first and second motion maps do not match in the corresponding time step.

[0076] Next, in step S17, the evaluation unit 113 determines whether the difference between the first operation map and the second operation map is less than a predetermined value. For example, the evaluation unit 113 may determine whether the proportion of the portion represented by the dashed line in FIG. 6 is less than a predetermined amount with respect to the proportion of the portion represented by the solid line.

[0077] When the evaluation unit 113 determines that the difference between the first operation map and the second operation map is less than the predetermined value, this step is determined as positive.

[0078] If the determination in this step is positive, the process proceeds to step S18.

[0079] If the determination in this step is negative, the process proceeds to step S19.

[0080] When the process transitions to step S18, the evaluation unit 113 gives an evaluation higher than a predetermined value to the target motion of the second worker 20 that has been the subject of the evaluation. The evaluation unit 113 may, for example, determine the target motion as pass out of the pass / fail determinations.

[0081] When the process transitions to step S19, the evaluation unit 113 assigns an evaluation lower than a predetermined value to the target motion of the second worker 20 that has been the subject of the evaluation. For example, the evaluation unit 113 may determine the target motion as failing in the pass / fail determination process.

[0082] In this example, a pass / fail judgment result is generated as the evaluation result, but an evaluation value for the action performed by second worker 20 may be generated based on the degree of agreement between the first action map and the second action map. The evaluation result may be output via a predetermined interface.

[0083] As described above, in the learning phase, the information processing device 100 according to the embodiment generates a first action map based on data obtained by sensing the first worker 10 in the first dimension. At this time, the information processing device 100 generates dimension conversion information that associates the first low-dimensional feature data with the first high-dimensional feature data.

[0084] Then, in the evaluation phase, the information processing device 100 uses the dimension conversion information to generate a second action map from data obtained by sensing the second worker 20 in a dimension lower than the first dimension. Then, the information processing device 100 compares the first action map with the second action map to evaluate the degree of agreement between the reference action performed by the first worker 10 and the action performed by the second worker 20.

[0085] According to this configuration, it is possible to convert the feature amount calculated from the low-dimensional sensor data into the feature amount expected to be obtained when sensing is performed in a high dimension. That is, it is possible to obtain the same results as when sensing is performed in a high dimension without using a device for sensing in a high dimension (for example, a motion capture device). Therefore, it is possible to reduce the cost for evaluating the motion of the worker.

[0086] (Modification) The above embodiment is merely an example, and the present disclosure may be modified without departing from the spirit and scope of the present disclosure. This may be modified as necessary.

[0087] For example, the processes and means described in this disclosure can be freely combined and implemented as long as no technical contradiction occurs.

[0088] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be executed by one device. In a computer system, the hardware configuration (server configuration) by which each function is realized can be flexibly changed.

[0089] The present disclosure can also be realized by supplying a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors of the computer read and execute the program. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (floppy disk, hard disk drive (HDD), etc.), an optical disk (CD-ROM, DVD disk, Blu-ray disk, etc.), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0090] 10. First worker 20. Second worker 100.... Information processing device 110... Control unit 111...Acquisition part 112...Learning Department 113...Evaluation section 120...Storage section 130 Communications Department 200···Camera 210 Marker

Claims

1. acquiring high-dimensional sensor data that is data obtained by sensing a position of a body part of a first worker performing a reference motion in a first dimension; generating a first motion map, the first motion map being a map representing a transition over time of a plurality of postures, using first high-dimensional feature data, the first high-dimensional feature data being a feature obtained based on the high-dimensional sensor data; converting the high-dimensional sensor data to generate low-dimensional sensor data obtained when sensing a position of a body part of the first worker in a second dimension lower than the first dimension; generating dimension conversion information representing a correspondence relationship between the first high-dimensional feature amount data obtained based on the high-dimensional sensor data and the first low-dimensional feature amount data obtained based on the low-dimensional sensor data converted from the high-dimensional sensor data; acquiring second low-dimensional sensor data which is data obtained by sensing a predetermined action performed by a second worker in the second dimension, and extracting, from the dimension conversion information, second high-dimensional feature data which is high-dimensional feature data corresponding to the second low-dimensional feature data obtained based on the second low-dimensional sensor data; evaluating a degree of agreement between the predetermined motion and the reference motion based on a result of comparing the first motion map with a second motion map generated using the extracted second high-dimensional feature data; Including, Information processing methods.

2. Each of the first high-dimensional feature data, the first low-dimensional feature data, the second high-dimensional feature data, and the second low-dimensional feature data is data representing a ranking of the distance between all combinations of two of the plurality of body parts of the first worker. The information processing method according to claim 1 .

3. Extracting the second high-dimensional feature data from the dimension transformation information includes: extracting the second high-dimensional feature data corresponding to the first low-dimensional feature data that is most similar to the second low-dimensional feature data; 3. The information processing method according to claim 1.

4. generating the second movement map for each of the second workers; 3. The information processing method according to claim 1.

5. Generating the first motion map includes: classifying a plurality of postures of the first worker by clustering a plurality of the first high-dimensional feature data, and generating the first movement map representing a time transition of the plurality of postures based on a result of the classification.

3. The information processing method according to claim 1.

Images