Information processing device

Abstract

To provide an information processing device capable of identifying a worker with a simple configuration.SOLUTION: In an information processing device 10, an image acquisition unit 11 acquires time-series images including a person and an object, a person position recognition unit 12 recognizes skeletal positions of the person at predetermined time points from the time-series images, an object position recognition unit 13 recognizes positions of the object at the predetermined time points from the time-series images, a calculation unit 14 calculates velocities and accelerations of the person at the predetermined time points, moving distances of the person between the predetermined time points, and positional relations between the person and the object at the predetermined time points, based on the recognized information, and a learning unit 15 generates a model for determining whether or not the person is a worker by machine learning based on the calculated information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device. [Background technology]

[0002] In recent years, the demand for small-lot, multi-item production has been increasing due to diversifying consumer needs, but small-lot, multi-item production leads to poorer workability compared to mass production. In order to reduce the cost of products, it is necessary to improve productivity by improving work processes. To achieve this, it is essential to introduce a system that identifies workers. However, in the case of small-lot, multi-item production, the workplace has a high degree of freedom, and people other than workers, such as developers and designers, are often present in the workplace. For this reason, a system that can easily identify workers is desired.

[0003] Patent Document 1 discloses a technique that can acquire skeleton information of a single worker even when there are multiple workers. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-162889 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, before acquiring skeletal information of a worker, it is necessary to identify the worker in advance using a face recognition system, a barcode, a QP code, etc. Therefore, Patent Document 1 has a problem that the worker cannot be identified with a simple configuration.

[0006] In view of such problems, the present disclosure has an object to provide an information processing device capable of identifying a worker with a simple configuration. [Means for solving the problem]

[0007] The information processing device disclosed herein includes an image acquisition unit that acquires time-series images including a person and an object, a person position recognition unit that recognizes the skeletal position of the person at a predetermined time from the time-series images, an object position recognition unit that recognizes the position of the object at a predetermined time from the time-series images, a calculation unit that calculates the speed and acceleration of the person at a predetermined time, the distance traveled by the person between predetermined times, and the positional relationship between the person and the object at the predetermined time based on information recognized by the person position recognition unit and the object position recognition unit, and a learning unit that generates a model for determining whether the person is a worker by machine learning based on the information calculated by the calculation unit. Effect of the Invention

[0008] The present disclosure makes it possible to provide an information processing device capable of identifying a worker with a simple configuration. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of a configuration of an information processing device according to a first embodiment. [Diagram 2] 5 is a flowchart showing an example of a learning operation of the information processing device according to the first embodiment. [Diagram 3] 3 is a diagram showing an example of the coordinate positions of each joint of a person recognized by the information processing device according to the first embodiment; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (First embodiment) First, the configuration of an information processing device 10 according to the first embodiment will be described with reference to FIG. 1 is a block diagram showing an example of a configuration of an information processing device 10 according to the first embodiment. As shown in FIG. The imaging device 1 is a device, such as a camera, that captures time-series images. The imaging device 1 is installed, for example, in a workshop. The workshop refers to a place where workers and other people such as developers and designers work.

[0011] The information processing device 10 is a device for identifying a worker in a workplace, such as a PC (Personal Computer). The information processing device 10 includes an image acquisition unit 11, a person position recognition unit 12, an object position recognition unit 13, a calculation unit 14, a learning unit 15, and a discrimination unit 16. The image acquisition unit 11 acquires from the imaging device 1 one or more time-series images including a person. The person position recognition unit 12 recognizes the skeletal position of a person in the time-series images at a given time by using a skeleton recognition technique such as PoseNet. The skeletal position of a person in the time-series images is, for example, the coordinate position of each joint of the person in the time-series images.

[0012] The object position recognition unit 13 recognizes the position of an object in the time-series images at a predetermined time by using an object recognition technique such as YOLO or SSD. The position of an object in the time-series images is, for example, the coordinate position of the object in the time-series images. The calculation unit 14 calculates the speed and acceleration of the person at a specified time, the distance traveled by the person between specified times, and the positional relationship between the person and the object at a specified time, based on the skeletal position of the person and the position of the object in the time-series images at a specified time. The learning unit 15 generates a model for determining whether or not a person is a worker by machine learning, based on the information calculated by the calculation unit 14. Hereinafter, the generated model is referred to as a trained model.

[0013] Furthermore, the calculation unit 14 calculates the positional relationship between each of the person's joints at a given time (that is, the person's posture) based on the skeleton positions of the person and the positions of objects in the time-series images at the given time. The learning unit 15 generates a model by machine learning for determining whether a person is a worker or not and for determining the work content of the worker, based on the information calculated by the calculation unit 14.

[0014] The discrimination unit 16 determines whether a person in the time-series images is a worker by inputting information on the speed and acceleration of the person at a predetermined time, the moving distance of the person during a predetermined time, the positional relationship between the person and an object at a predetermined time, and the positional relationship between each joint of the person at a predetermined time into the trained model. The discrimination unit 16 also determines the work content of the worker in the time-series images.

[0015] Next, the operation of the information processing device 10 according to the first embodiment will be described. First, a learning operation of the information processing device 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the learning operation of the information processing device 10 according to the first embodiment. In the example of Fig. 2, the imaging device 1 is placed in a workshop. The imaging device 1 is also placed so that a person is included in the angle of view. Then, the imaging device 1 captures one or more time-series images or videos including the person.

[0016] First, in step S101, the image acquisition unit 11 of the information processing device 10 acquires one or more time-series images from the imaging device 1. Note that the image acquisition unit 11 may acquire a moving image from the imaging device 1. In this case, the image acquisition unit 11 converts the acquired moving image into one or more time-series images and acquires the converted one or more time-series images.

[0017] Next, in step S102, the person position recognition unit 12 recognizes the coordinate positions of each joint of the person in the time-series images at a predetermined time (t1, t2, t3, t4, ...) using a skeleton recognition technology such as PoseNet, as shown in Table 1. The coordinate positions indicate the x and y coordinates of a two-dimensional coordinate system. Here, the coordinate positions of each joint of the person are shown in FIG. 3. FIG. 3 is a diagram showing an example of the coordinate positions of each joint of the person recognized by the information processing device 10 according to the first embodiment.

[0018] [Table 1]

[0019] Here, "null" in Table 1 indicates a state where recognition is impossible due to a part of a joint being hidden, etc. In this embodiment, the person position recognition unit 12 recognizes the x and y coordinates of each joint of the person in a two-dimensional coordinate system. However, the person position recognition unit 12 is not limited to this, and may recognize the z coordinate in addition to the x and y coordinates of each joint of the person in a three-dimensional coordinate system.

[0020] Next, in step S103, the object position recognition unit 13 recognizes the coordinate position of an object in the time-series images at a predetermined time (t1, t2, t3, t4, ...) using an object recognition technique such as YOLO or SSD. The coordinate position indicates the x-coordinate and the y-coordinate in a two-dimensional coordinate system.

[0021] Next, in step S104, the calculation unit 14 calculates the coordinate positions of each joint of the person at the predetermined time (t 0 , t final ) at each time t n The velocity v and acceleration a of the person at the time, and 0 ~t final ) to calculate the moving distance L of the person. 1 is set to the time when the person starts moving, and final is set to the time when the person stops moving. 1 and time t final is not limited to these, and any time may be set. Specifically, the calculation unit 14 calculates the time (t 0 , t 1 , t 2 , t final ), select the coordinate position of the joint closest to the waist from the coordinate positions of each joint of the person, and calculate the coordinate position of the person ((X0,Y0), (X1,Y1), (X2,Y2), ..., (X final ,Y final )).

[0022] [Table 2]

[0023] The calculation unit 14 calculates the velocity v and acceleration a of the person at a given time by numerically differentiating the coordinate position of the person at the given time with respect to the elapsed time as shown in the following formulas (1) and (2). 1 The formula for calculating the person's velocity v and acceleration a is shown below.

[0024]

number

[0025]

number

[0026] In addition, the calculation unit 14 calculates the moving distance L of the person by performing numerical integration with respect to the elapsed time from the coordinate position of the person at the predetermined time, as shown in the following formula (3). 0 ~t final ) represents the calculation formula for the moving distance L of a person.

[0027]

number

[0028] Furthermore, in step S105, the calculation unit 14 calculates the relative positions between the joints of the person (that is, the posture of the person) from the coordinate positions of the joints of the person at a predetermined time. In step S106, the calculation unit 14 calculates the coordinate positions of each joint of the person and the coordinate positions of the object at the predetermined time (t 0 , t final The relative position of the person to the object in ((X0-x0, Y0-y0), . . ., (X final -x final ,Y final -y final In Table 3, the time (t 0 , t finalThe coordinate position of the person in final ,Y final ) and at a given time (t 0 , t final The coordinate position of the object in is (x0, y0),...,(x final ,y final The calculation unit 14 may calculate the relative position of the person with respect to each of a plurality of objects, not limited to a single object.

[0029] [Table 3]

[0030] The calculation unit 14 also calculates the relative positions between the joints of the same person shown in step S105 in the same manner as in Table 3.

[0031] Here, the speed, acceleration, and moving distance of the person calculated by the calculation unit 14 can also be feature quantities for determining whether or not the person is a worker. For example, workers often move around. On the other hand, people other than workers often sit on a workbench and work, and are stationary. In addition, the relative position of the person with respect to an object calculated by the calculation unit 14 can be a feature quantity for determining whether or not the person is a worker. For example, workers often stay near a vehicle. On the other hand, people other than workers, such as developers, often stay near a workbench. In addition, the relative positions between the joints of the person calculated by the calculation unit 14 (the person's posture) can be a feature quantity for determining the work content of the person. For example, in assembly work, workers often go under the vehicle and take a posture with their hands raised above their shoulders.

[0032] In step S107, the learning unit 15 generates a model for determining whether a person is a worker and the work content of the worker by machine learning based on the information calculated by the calculation unit 14 in steps S104 to S106. Specifically, the learning unit 15 creates teacher data by annotating the information calculated by the calculation unit 14 in steps S104 to S106. The annotated information is information on whether a person is a worker or another person such as a developer or designer. In addition, the annotated information is information on the work content of the person, such as assembly work. Then, the learning unit 15 learns the learning model by machine learning using the teacher data.

[0033] Next, the determination operation of the information processing device 10 according to the first embodiment will be described. The imaging device 1 is placed in a workplace where workers and other people actually work. The imaging device 1 captures one or more time-series images including a person. First, the information processing device 10 executes the above-mentioned processes of steps S101 to S106 from one or more time-series images, etc. By doing so, the information processing device 10 calculates the speed and acceleration of a person in the time-series images at a given time, and the moving distance of the person between the given times. In addition, the information processing device 10 calculates the relative positions between each joint of the person and the relative position of the person with respect to an object at the given time.

[0034] The discrimination unit 16 of the information processing device 10 inputs the calculated information into the trained model generated in the above-mentioned step S107, thereby discriminating whether or not the person in the time-series images is a worker. Furthermore, if the person is a worker, the discrimination unit 16 discriminates the work content.

[0035] As described above, the information processing device 10 according to the first embodiment generates a model for determining whether or not a person is a worker by machine learning based on the speed, acceleration, and moving distance of a person in a time-series image, the positional relationship between the person and an object, and the positional relationship between each joint of the person. Then, the information processing device 10 determines whether or not a person in a time-series image is a worker by using the generated trained model from time-series images actually captured in a workplace. Therefore, the information processing device 10 can determine whether or not a person is a worker with a simple configuration without the need to identify the worker in advance using a face authentication system, a barcode, a QP code, or the like before determining whether or not the person is a worker.

[0036] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention.

[0037] Each component of the information processing device 10 in the above-mentioned embodiment is configured by hardware or software, or both, and may be configured by one piece of hardware or software, or may be configured by multiple pieces of hardware or software. Specifically, each component of the information processing device 10 in the above-mentioned embodiment is configured by a computer including a processor and a memory. The processor may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor may include multiple processors. The memory is configured by a combination of a volatile memory and a non-volatile memory. The memory may include a storage located away from the processor. In this case, the processor may access the memory via an I / O interface not shown. The processor executes one or more programs including a group of instructions for causing the computer to execute the algorithm described using the drawings.

[0038] The program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals. [Explanation of symbols]

[0039] REFERENCE SIGNS LIST 1 imaging device, 10 information processing device, 11 image acquisition unit, 12 person position recognition unit, 13 object position recognition unit, 14 calculation unit, 15 learning unit, 16 discrimination unit

Claims

[Claim 1] an image acquisition unit for acquiring time-series images including people and objects; a person position recognition unit that recognizes a skeletal position of the person at a predetermined time from the time-series images; an object position recognition unit that recognizes a position of the object at a predetermined time from the time-series images; a calculation unit that calculates a speed and acceleration of the person at a predetermined time, a moving distance of the person between predetermined times, and a positional relationship between the person and the object at the predetermined time based on information recognized by the person position recognition unit and the object position recognition unit; A learning unit that generates a model for determining whether the person is a worker or not by machine learning based on the information calculated by the calculation unit. Information processing device.

Images