Component management device
The parts management device addresses high operating costs by using a machine-learned discrimination model to activate sensors only during specific operations, reducing power consumption and aging, thus optimizing sensor usage.
Patent Information
- Application Number
- JP2023078704
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2043-05-11
AI Technical Summary
Existing parts management systems face increased operating costs due to continuous sensor operation, which leads to high power consumption and aging deterioration.
A parts management device that includes a management unit, determination unit, and control unit to detect worker motions using a machine-learned discrimination model, activating sensors only during specific operations like take-out or replenishment, and stopping them during work or no operation.
Reduces operating costs by minimizing sensor power consumption and aging deterioration through intelligent activation based on worker actions, thereby optimizing sensor usage.
Smart Images

Figure 2024162798000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a parts management device. [Background technology]
[0002] For example, it is known that a parts counting device detects and manages parts using a sensor (for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-96329 Summary of the Invention [Problem to be solved by the invention]
[0004] When a worker works with parts, the worker takes out the parts from, for example, a parts shelf installed in the work area. In order to manage the number of parts, for example, a sensor that detects the parts is provided near the parts shelf. In parts management, the sensor is constantly performing detection operations, which can cause problems such as increased operating costs due to power consumption and deterioration over time.
[0005] SUMMARY OF THE PRESENT DISCLOSURE The present invention has been made in consideration of the above problems, and has an object to provide a parts management device that can reduce operation costs. [Means for solving the problem]
[0006] The parts management device of the present invention has a management unit that manages the number of parts stored in a storage location by detecting the parts with a sensor, a discrimination unit that discriminates a removal action of a worker removing a part from the storage location by detecting the skeletal structure of the worker from an image captured of the worker using the parts to perform work, and a control unit that activates the sensor when the discrimination unit discriminates the removal action.
[0007] In the above-mentioned parts management device, the discrimination unit may detect the skeletal structure of the worker from the image to discriminate the work movement performed by the worker using the parts, and the control unit may stop the sensor when the discrimination unit discriminates the work movement.
[0008] In the above-mentioned parts management device, the discrimination unit may detect the skeleton of one of the worker and the other worker from the image, and thereby discriminate a replenishment action of one of the worker and the other worker replenishing the parts to the storage location, and the control unit may activate the sensor when the discrimination unit discriminates the replenishment action.
[0009] In the above-mentioned parts management device, the discrimination unit may discriminate the removal action from the image using a machine-learned discrimination model through supervised learning, which receives as input the coordinates of the skeleton in the image and outputs a type of the worker's action including the removal action.
[0010] In the above-mentioned parts management device, when the storage number is less than a predetermined number, the management section may request the worker or another worker to replenish the parts in the storage location. Effect of the Invention
[0011] According to the present invention, the operating costs of the parts management device can be reduced. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a configuration diagram showing an example of a parts management system. [Diagram 2] FIG. 2 is a diagram showing an example of an image captured by a camera device while a worker performs an acquiring operation. [Diagram 3] FIG. 3 is a diagram showing an example of an image captured by a camera device of a working motion of a worker. [Figure 4] FIG. 4 is a diagram showing an example of an image captured by a camera device while a worker is performing a replenishment operation. [Diagram 5]FIG. 5 is a flowchart illustrating an example of a control process of the sensor device. [Figure 6] FIG. 6 is a flowchart showing an example of a process for managing the number of stored parts. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] (Example of a parts management system configuration) 1 is a configuration diagram showing an example of a parts management system 9. As an example, the parts management system 9 is installed in an assembly plant for automobiles or the like.
[0014] The parts management system 9 includes a parts management server 1, a camera device 2, a sensor device 3, and a terminal 4 that can communicate with each other via a LAN (Local Area Network) 90. The parts management server 1 is an example of a parts management device. The parts management server 1 manages the number of various parts used in assembly. The camera device 2 captures an image of a worker who works with parts. The sensor device 3 is an example of a sensor, and detects parts stored on the parts shelf. The terminal 4 is carried by a worker who replenishes parts on the parts shelf.
[0015] The parts management server 1 is an example of a computer, and includes a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, a HDD (Hard Disk Drive) 13, and a communication port 14. The CPU 10 is electrically connected to the ROM 11, the RAM 12, the HDD 13, and the communication port 14 via a bus 19 so as to be able to input and output signals to and from each of them.
[0016] The ROM 11 stores a program that drives the CPU 10. The RAM 12 functions as a working memory for the CPU 10. The communication port 14 is, for example, a wireless LAN (Local Area Network) card, and processes communication of the CPU 10 via a LAN 90.
[0017] When the CPU 10 reads the program from the ROM 11, it forms a device control unit 100, a parts management unit 101, an operation discrimination unit 102, and a sensor control unit 103 as software functions. In addition, the HDD 13 stores skeleton data 130 and parts number data 131. The device control unit 100 instructs the parts management unit 101, the operation discrimination unit 102, and the sensor control unit 103 to operate in accordance with a sequence defined in the program. Note that the device control unit 100, the parts management unit 101, the operation discrimination unit 102, and the sensor control unit 103 are not limited to software, and may be realized by hardware such as an ASIC (Application Specified Integrated Circuit).
[0018] The parts management unit 101 is an example of a management unit. The parts management unit 101 manages the number of parts stored in a storage location by detecting the parts stored in the storage location with the sensor device 3. The parts management unit 101 updates the parts number data 131 based on the detection result of the sensor device 3.
[0019] The action discrimination unit 102 is an example of a discrimination unit. The action discrimination unit 102 discriminates various actions of the worker by detecting the skeleton of the worker from the image of the camera device 2. The actions include a removal action in which the worker removes a part from a storage location, a work action in which the worker works with a part, and a replenishing action in which the worker replenishes parts in a storage location.
[0020] For example, the motion discrimination unit 102 stores the coordinates of multiple points of the worker's skeleton as time-series skeleton data 130 in the HDD 13. The skeleton data 130 indicates, for example, the coordinates of the worker's nose, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, and right hip for each frame number of the image. The coordinates are indicated by the X coordinate and the Y coordinate (x, y) when the horizontal and vertical directions of the rectangular frame of the image are defined as the X axis and the Y axis, respectively. There is no limitation on the position of the origin.
[0021] The sensor control unit 103 is an example of a control unit. The sensor control unit 103 activates the sensor device 3 when the operation determination unit 102 determines a removal operation or a refill operation. The sensor control unit 103 also stops the sensor device 3 when the operation determination unit 102 determines a work operation. Therefore, the sensor device 3 is activated when the worker changes the number of stored parts, and is stopped when the worker does not change the number of stored parts. This reduces the operating time of the sensor device 3. Each operation of the worker is described below.
[0022] (Acquisition operation) 2 is a diagram showing an example of images G1a and G2a captured by the camera device 2 while the operator H is performing an action to obtain an image. The X-axis and Y-axis in the images G1a and G2a correspond to the horizontal and vertical directions of the paper, respectively. This also applies to the subsequent drawings.
[0023] Images G1a, G2a of camera device 2 capture worker H in the workplace. In the workplace, as an example, a workbench 80 on which work is performed, parts shelves 81, 82 storing multiple parts boxes 810, 820, a transport conveyor 83, and part sensors 31, 32 constituting sensor device 3 are installed. Multiple parts C1, C2 of different types are stored in the parts boxes 810, 820, respectively. The parts shelves 81, 82 are an example of storage locations for parts C1, C2.
[0024] As shown in images G1a and G2a, worker H takes out parts boxes 810 and 820 from parts shelves 81 and 82. As will be described later, worker H performs an operation of assembling parts C1 and C2 to a workpiece (not shown). The workpiece is transported by a transport conveyor 83.
[0025] The component sensors 31, 32 detect the component boxes 810, 820, respectively. The component sensors 31, 32 are, for example, optical sensor arrays, and output detection light L toward the component shelves 81, 82 (see dotted lines), and detect each of the component boxes 810, 820 by the reflected light. The component sensors 31, 32 transmit detection signals indicative of the detection results to the component management unit 101 via the LAN 90. The component management unit 101 increases or decreases the storage quantity of each of the components C1, C2 based on the detection signals.
[0026] Note that the part sensors 31, 32 are not limited to this and may be other sensors such as weight sensors. The part sensors 31, 32 are normally in a low power consumption sleep state in which the detection operation is stopped, and when they receive a start-up signal from the part management server 1 and are started up, the sleep state is released and the detection operation is started.
[0027] The movement discrimination unit 102 detects the coordinates of the skeleton of the worker H from the images G1a and G2a. Examples of the coordinates of the skeleton include the position P0 of the nose of the worker H, the position P23 of the left shoulder, the position P13 of the right shoulder, the position P22 of the left elbow, the position P12 of the right elbow, the position P21 of the left hand, the position P11 of the right hand, the position P32 of the left hip, and the position P31 of the right hip. Note that the positions P0, P11 to P13, P21 to P23, P31, and P32 are shown only in the image G1a, and are omitted from the images G2a and onward.
[0028] The action discrimination unit 102 detects the coordinates of the skeleton of the worker H by image analysis using, for example, a general-purpose AI (Artificial Intelligence). The action discrimination unit 102 identifies the take-out action of the worker H based on the time change of the coordinates of the skeleton. The image G1a shows the worker H stretching both hands toward the parts shelf 81, and the image G2a shows the worker H taking down the parts box 810 from the parts shelf 81. The action discrimination unit 102 discriminates the take-out action by pattern matching of the time change of the coordinates of the skeleton in the images G1a and G2a and the coordinates of the skeleton in the images of other frames not shown. Note that in this example, the take-out action of the worker H taking out the parts box 810 from one parts shelf 81 is exemplified, but the take-out action of taking out the parts box 820 from the other parts shelf 82 can also be discriminated in the same way.
[0029] When the operation determination unit 102 determines that a removal operation has been performed, the sensor control unit 103 uses an activation signal to activate the component sensors 31, 32. When the component sensors 31, 32 are activated, the sleep state is released and the component sensors 31, 32 start detecting the component boxes 810, 820.
[0030] In this way, the part sensors 31, 32 operate in response to a decrease in the number of stored parts C1, C2 resulting from the removal operation of the worker H, and therefore the operating time is reduced compared to when they are constantly operating. Therefore, the power consumption and deterioration over time of the part sensors 31, 32 are suppressed, and the operating cost of the part management server 1 is reduced.
[0031] The operation discrimination unit 102 separately discriminates between the removal operation of one parts box 810 and the removal operation of the other parts box 820. When the sensor control unit 103 discriminates the removal operation of the parts box 810, it activates the parts sensor 31, and when the sensor control unit 103 discriminates the removal operation of the parts box 820, it activates the parts sensor 32.
[0032] (Working motion) Fig. 3 is a diagram showing an example of images G1b, G2b captured by the camera device 2 of the working movement of a worker H. In Fig. 3, the same components as those in Fig. 2 are given the same reference numerals, and the description thereof will be omitted.
[0033] In this example, the work operation using the part C1 includes, but is not limited to, an operation of assembling the part C1 acquired in the acquisition operation in Fig. 2 to the work W. Examples of the work operation include an operation of painting the part C1, an operation of welding the part C1 to the work W, and an operation of inspecting the part C1.
[0034] The motion discrimination unit 102 identifies the work motion of the worker H based on the change over time in the coordinates of each part of the skeleton. Image G1b shows the worker H holding the work W with his left hand and aligning the installation position of the part C1 with his right hand, and image G2b shows the worker H assembling the part C1 to the work W. The motion discrimination unit 102 discriminates the work motion by pattern matching of the change over time in the coordinates of the skeleton in images G1b and G2b and the coordinates of the skeleton in images of other frames not shown. Note that in this example, the work motion of the worker H assembling one part C1 to the work W is given, but the work motion of assembling the other part C2 can also be discriminated in a similar manner.
[0035] When the motion discrimination unit 102 discriminates a work motion, the sensor control unit 103 stops the part sensors 31, 32 by a stop signal. When stopped, the part sensors 31, 32 transition to a sleep state and interrupt the detection operation of the part boxes 810, 820. While the worker H is performing a work motion, the number of stored parts C1, C2 does not change, so by stopping the part sensors 31, 32 in accordance with the work motion, the power consumption and deterioration over time of the part sensors 31, 32 are more efficiently suppressed.
[0036] (Replenishment operation) Fig. 4 is a diagram showing an example of images G1c, G2c captured by the camera device 2 of the replenishment operation of the worker H. In Fig. 4, the same components as those in Fig. 2 are given the same reference numerals, and the description thereof will be omitted.
[0037] The motion discrimination unit 102 identifies the replenishment motion of the worker H based on the change over time in the coordinates of the skeleton. The image G1c shows the worker H lifting a new parts box 810 with both hands next to the parts shelf 81, and the image G2c shows the worker H placing the new parts box 810 on the parts shelf 81. The motion discrimination unit 102 discriminates the replenishment motion by pattern matching of the change over time in the coordinates of the skeleton in the images G1c and G2c and the coordinates of the skeleton in the images of other frames not shown. Note that in this example, the work motion of the worker H replenishing one part C1 is given, but the replenishment motion of replenishing the other part C2 can also be discriminated in the same way.
[0038] When the operation determination unit 102 determines that a replenishing operation is being performed, the sensor control unit 103 uses an activation signal to activate the part sensors 31, 32. When the part sensors 31, 32 are activated, the sleep state is released and they start detecting the part boxes 810, 820.
[0039] In this way, the part sensors 31, 32 operate in response to an increase in the number of stored parts C1, C2 accompanying the removal operation of the worker H, and therefore the operating time is reduced compared to when the part sensors are constantly operating. Therefore, the power consumption and deterioration over time of the part sensors 31, 32 are more effectively suppressed.
[0040] The replenishing action may be performed by another worker (not shown) other than the worker H. In this case, the action determination unit 102 detects the skeleton of the other worker in the same manner as described above to determine the replenishing action of the other worker.
[0041] Furthermore, the operation discrimination unit 102 separately discriminates between a replenishing operation for one parts box 810 and a replenishing operation for the other parts box 820. The sensor control unit 103 activates the parts sensor 31 when a replenishing operation for the parts box 810 is discriminated, and activates the parts sensor 32 when a replenishing operation for the parts box 820 is discriminated.
[0042] In this way, the action discrimination unit 102 discriminates the action of the worker H from the coordinates of the skeleton. For example, AI can be used for this discrimination. For example, the action discrimination unit 102 discriminates the take-out action, the work action, and the replenishing action from the images G1a to G1c, G2a to G2c using a discrimination model that has been machine-learned by supervised learning and that inputs the coordinates of the skeleton in the images G1a to G1c, G2a to G2c and outputs the types of actions including the take-out action, the work action, and the replenishing action.
[0043] Specifically, in the learning process of the discrimination model, annotation is performed on the type of motion of the worker H from the time change of the coordinates of the skeleton of the teacher data. The discrimination model is a neural network that mathematically models the function of the human brain, and determines the weighting coefficient of the activation function of the part corresponding to the neuron based on machine learning. In this way, the motion discrimination unit 102 can discriminate between the take-out motion, the work motion, and the replenishment motion with high accuracy by using the discrimination model that has been machine-learned by supervised learning.
[0044] (Parts management server operation) 5 is a flowchart showing an example of the control process of the sensor device 3. First, the motion discrimination unit 102 receives image data from the camera device 2 via the communication port 14 (step St1). The motion discrimination unit 102 receives the above images G1a to G1c, G2a to G2c as image data in frame units.
[0045] Next, the movement discrimination unit 102 detects the coordinates of the skeleton of the worker H from the image (step St2). Next, the movement discrimination unit 102 stores the detection results for each frame of the image in the HDD 13 as skeleton data 130 (step St3). The movement discrimination unit 102 discriminates the movement of the worker H based on the skeleton data 130 (step St4).
[0046] If a removal operation is determined (Yes in step St5) or a refill operation is determined (No in step St5, Yes in step St6), the sensor control unit 103 starts the sensor device 3 (step St8). If a work operation is determined (No in step St6, Yes in step St7), the sensor control unit 103 stops the sensor device 3 (step St9). If none of the removal operation, refill operation, and work operation is determined (No in step St7), the process of step St10 described below is performed.
[0047] Next, the operation discrimination unit 102 judges whether or not to continue this process (step St10). If the process continues (Yes in step St10), each process from step St1 onward is performed again, and if the process ends (No in step St10), the operation ends.
[0048] 6 is a flowchart showing an example of a management process of the number of stored parts C1 and C2. First, the part management unit 101 determines whether or not a detection signal has been received from the sensor device 3 (step St11). If a detection signal has not been received (No in step St11), the process of step St15 described later is performed.
[0049] If a detection signal is received (Yes in step St11), the parts management unit 101 updates the parts count data 131 based on the detection result of the sensor device 3 (step St12). The parts management unit 101 calculates the storage count for each of the parts C1 and C2, for example, by multiplying the number of parts boxes 810 and 820 detected by the sensor device 3 by the number of parts C1 and C2 contained in one box.
[0050] Next, the parts management unit 101 compares the number of parts C1 and C2 stored with the predetermined number TH (step St13). If the number of parts stored is less than the predetermined number TH (Yes in step St13), the parts management unit 101 transmits a replenishment request for parts C1 and C2 less than the predetermined number TH to the terminal 4 owned by the worker H or another worker (step St14). In this way, if the number of parts stored is less than the predetermined number TH, the parts management unit 101 requests the worker H or another worker to replenish the parts C1 and C2 on the parts shelves 81 and 82. Therefore, the parts C1 and C2 can be replenished appropriately. The predetermined number TH may be determined according to, for example, the reduction speed of the parts C1 and C2 or the progress speed of the work. If the number of parts stored is equal to or greater than the predetermined number TH (No in step St13), the parts management unit 101 determines that a replenishment request for parts C1 and C2 is unnecessary and executes the process of the following step St15.
[0051] Next, the parts management unit 101 compares the number of parts C1, C2 taken out from the parts shelves 81, 82 (taken number) with the number of workpieces W completed by assembling the parts C1, C2 (completed number) (step St15). If the taken number does not match the completed number (No in step St15), the parts management unit 101 outputs an alarm (step St16). The alarm output means is, for example, a speaker or a monitor, but is not limited to such. If the taken number matches the completed number (Yes in step St15), the process of step St17 described below is executed. The completed number is acquired from another server (not shown) via the LAN 90.
[0052] Next, the part management unit 101 determines whether or not to continue this process (step St17). If the process continues (Yes in step St17), each process from step St11 onward is performed again, and if the process ends (No in step St17), the operation ends.
[0053] In this embodiment, the example is given in which the worker H takes out the components C1 and C2 from the component boxes 810 and 820, works on them, and then replenishes them, but these actions may be performed on the component C1 and C2 basis. In this case, the component sensors 31 and 32 detect the components C1 and C2 stored on the component shelves 81 and 82 one by one, respectively.
[0054] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this embodiment and can be modified in various ways without departing from the scope of the present invention. [Explanation of symbols]
[0055] 1 Parts management server (parts management device), 3 Sensor device (sensor), 10 CPU, 101 Parts management unit (management unit), 102 Motion discrimination unit (discrimination unit), 103 Sensor control unit (control unit), 130 Skeleton data, H Worker
Claims
1. a management unit that manages the number of parts stored in a storage location by detecting the parts stored in the storage location with a sensor; a discrimination unit that discriminates a removal action of the worker removing the part from the storage location by detecting a skeleton of the worker from an image captured of the worker using the part in a task; and a control unit that activates the sensor when the determination unit determines that the removal operation has been performed. Parts management device.
2. the discrimination unit discriminates a work motion performed by the worker using the part by detecting a skeleton of the worker from the image; The control unit stops the sensor when the determination unit determines that the work motion has occurred. The parts management device according to claim 1 .
3. the discrimination unit discriminates a replenishment action of one of the worker and the other worker replenishing the part to the storage location by detecting a skeleton of one of the worker and the other worker from the image; The control unit activates the sensor when the determination unit determines that the refill operation is performed. The parts management device according to claim 1 or 2.
4. the discrimination unit discriminates the take-out motion from the image using a discrimination model that has been machine-learned through supervised learning and that outputs a type of the motion of the worker including the take-out motion, using coordinates of the skeleton in the image as an input; The parts management device according to claim 1 or 2.
5. When the storage number is less than a predetermined number, the management unit requests the worker or another worker to replenish the parts in the storage location. The parts management device according to claim 1 or 2.
Citation Information
Patent Citations
Direct-use dispensing system
JP2013546089A
Work analyzer and work analysis program
JP2021163293A
Action recognition program, action recognition method, and information processing apparatus
JP2022189456A
System and method for automatically restocking items on shelves
US20180322448A1
Temporary small article sorting and counting device
JP1999096329A
Cited By
Work support system
JP2025017100A
Work support system
JP7928390B2
