Parts counting system, learning device, inference device, parts counting method, and program

Abstract

This allows for accurate quantity verification of the parts to be dispensed. [Solution] The parts counting system 1 is a parts counting system that counts the quantity of parts to be dispensed in a dispensing process in which parts necessary for product production are taken out of a warehouse, and comprises a storage device 400 and an inference device 300. The storage device 400 stores a trained model that has learned the relationship between captured images and mass measurements and the quantity of parts by machine learning using part-related information including captured images of parts, mass measurements, and count values ​​based on each of these, and example information showing examples related to parts counting. The inference device 300 applies the captured images and mass measurements to be inferred to the trained model to infer the quantity of parts.

Description

Technical Field

[0001] The present disclosure relates to a component counting system, a learning device, an inference device, a component counting method, and a program.

Background Art

[0002] In the dispensing process of preparing and recording components in the warehouse and dispensing them to the manufacturing site, the preparation of components, recording work, etc. required a great deal of manpower, time, etc. This is because it was carried out by a method of visually checking the quantity and quality by workers. Therefore, there has been a need for a technology that can accurately recognize an object by image recognition and efficiently process the obtained data to obtain accurate information on the quantity of components.

[0003] For example, Patent Document 1 discloses an inspection system that acquires three-dimensional image data of materials, performs deep learning based on this three-dimensional image data, and generates material data including at least the type, dimensions, and quantity for each material.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the technology disclosed in Patent Document 1 calculates the quantity based on the three-dimensional image data of large building materials arranged in an aligned manner. For example, it is difficult to calculate the quantity of small components stacked on top of each other. Thus, when performing quantity confirmation only by image recognition, there is a problem that the way the target articles are placed is restricted and it is impossible to accurately confirm the quantity of articles of various sizes and types.

[0006] This disclosure is made in view of the circumstances described above, and aims to enable accurate quantity verification of the parts to be issued. [Means for solving the problem]

[0007] To achieve the above objectives, the parts counting system according to this disclosure is a parts counting system that counts the quantity of parts to be dispensed in a dispensing process in which parts necessary for product production are taken out of a warehouse, and comprises a storage device and an inference device. The storage device stores a trained model that has learned the relationship between captured images and mass measurements and the quantity of parts by machine learning using parts-related information including captured images of parts, mass measurements, and count values ​​based on each of these, and example information showing examples related to parts counting. The inference device applies the captured images and mass measurements to be inferred to the trained model to infer the quantity of parts. [Effects of the Invention]

[0008] According to this disclosure, the trained model is generated by machine learning using part-related information, including captured images of parts, mass measurements, and count values ​​based on each of these, as well as example information illustrating examples of part counting. As a result, the quantity of parts to be dispensed can be verified with high accuracy. [Brief explanation of the drawing]

[0009] [Figure 1] Block diagram showing an example configuration of a parts counting system. [Figure 2A] Block diagram showing an example of the hardware configuration of a parts counting device. [Figure 2B] Block diagram showing an example of the functional configuration of a parts counting device. [Figure 3A] Block diagram showing an example of the hardware configuration of a learning device. [Figure 3B] Block diagram showing an example of the functional configuration of a learning device. [Figure 4A] Block diagram showing an example of the hardware configuration of the inference device. [Figure 4B] Block diagram showing an example of the functional configuration of the inference device. [Figure 5A] A diagram showing an example of parts counting information. [Figure 5B] A diagram showing an example of inference result information. [Figure 6] A diagram to explain the algorithm of a neural network. [Figure 7] Flowchart showing the process of generating a trained model [Figure 8] A flowchart illustrating the process for generating inference result information. [Modes for carrying out the invention]

[0010] The parts counting system, learning device, inference device, parts counting method, and program according to the embodiments of this disclosure will be described in detail below with reference to the drawings. The inspection result information falsification detection system according to the embodiments of this disclosure counts the quantity of parts to be dispensed with high accuracy.

[0011] As shown in Figure 1, the parts counting system 1 according to Embodiment 1 of this disclosure comprises a parts counting device 100 for counting the quantity of parts, a learning device 200 for learning the relationship between captured images, mass measurements, and the quantity of parts, an inference device 300 for inferring the quantity of parts to be inferred, and a storage device 400 for storing information. The parts counting device 100, the learning device 200, the inference device 300, and the storage device 400 are connected to enable the transmission and reception of information via, for example, a wireless LAN (Local Area Network) (not shown).

[0012] The parts counting device 100 is a general-purpose computer device, such as a personal computer or server computer, that counts the parts to be dispensed. The parts counting device 100 is connected to an imaging device IE and a weighing device WE in a manner that enables information transmission and reception. For example, the parts counting device 100 acquires images of parts being continuously transported on a conveyor belt by the imaging device IE installed on a factory production line, and records the parts counting results obtained by image analysis of the acquired images. The acquired images may be either still images or moving images. Furthermore, the acquired images may be limited to images acquired during a specific period. For example, the acquired images may be images acquired between a start instruction and an end instruction, which are set in advance by the user as the shooting period, the training data collection period, etc. The acquired images are not limited to those output from the imaging device IE, but may also be images that have been acquired in advance and stored in a storage device. The imaging device IE is a camera equipped with a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) as a light-receiving element. Furthermore, the imaging device IE shall be a 3D camera capable of capturing images of the shooting range in three dimensions. The type of 3D camera is arbitrary; for example, a stereo system, a ToF (Time of Flight) system, etc., can be used.

[0013] Furthermore, the parts counting device 100 records the counting results obtained by counting parts that are continuously transported on a conveyor belt, in conjunction with image analysis of captured images, using, for example, a weighing device WE installed on the factory's production line. The weighing device WE is, for example, a counting scale that measures the weight of parts and calculates their number.

[0014] Furthermore, the parts counting device 100 records, for example, counting cases in image analysis of captured images and mass measurement in a database. These counting cases include both successful and unsuccessful cases. In this embodiment, the count value of parts based on captured images by the parts counting device 100 is referred to as the "image count value," and the count value of parts based on mass measurement values ​​is referred to as the "mass count value."

[0015] The learning device 200 is a general-purpose computer device such as a personal computer or a server computer that learns the relationship between the captured image, the mass measurement value, and the quantity of parts. The learning device 200 acquires the inspection result information output from the parts counting device 100 and the inspection-related information related to the inspection process, and stores them in the storage device 400. Further, the learning device 200 generates a learned model by performing machine learning using learning data based on parts-related information including the captured image of the parts, the mass measurement value, and the count value based on each of them. The storage device 400 outputs the generated learned model to the storage device 400 and stores it.

[0016] The inference device 300 is a general-purpose computer device such as a personal computer or a server computer that infers the quantity of parts to be inferred. The inference device 300 acquires the learned model from the storage device 400, and acquires the captured image and the mass measurement value of the inference target from the parts counting device 100 as inference data. The storage device 400 inputs the inference data acquired from the parts counting device 100 into the learned model, and generates and outputs inference result information indicating the inference result of the quantity of parts.

[0017] The storage device 400 is, for example, a NAS (Network Attached Storage) device that can be accessed via a network. The storage device 400 stores, for example, the learned model generated by the learning device 200.

[0018] (Hardware configuration example of parts counting device 100) Next, a hardware configuration example of the parts counting device 100 will be described. As shown in FIG. 2A, physically, the parts counting device 100 includes a processor 101, a memory 102, a storage 103, a display device 104, an input / output interface (input / output I / F (InterFace)) 105, and a communication interface (communication I / F) 106. These components are electrically connected to each other via a bus line 107.

[0019] The processor 101 is an arithmetic unit that controls the operation of the entire parts counting device 100. The processor 101 is a general-purpose processor such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit). Furthermore, the processor 101 is not limited to a general-purpose processor, but may also be a dedicated processor composed of an ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc.

[0020] Memory 102 is the main memory and includes ROM (Read Only Memory), RAM (Random Access Memory), etc. The processor 101 reads programs and various data from ROM or storage 103 onto RAM and executes processing to realize the overall control and functions of the parts counting device 100.

[0021] Storage 103 is an auxiliary storage device that stores programs and various data necessary for program execution, and includes non-volatile storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives). Storage 103 stores, for example, the OS (Operating System), which is the basic software that controls the entire parts counting device 100, and applications that run on the OS and provide various functions. Some of the programs and various data necessary for program execution may be stored in ROM.

[0022] The display device 104 is an image display device such as an LCD (Liquid Crystal Display), PDP (Plasma Display Panel), or organic EL (Electro-Luminescence) display, and displays various images according to the control of the processor 101. The display device 104 may also be used in a configuration connected to the input / output interface 105.

[0023] The input / output interface 105 is an interface for connecting to input devices such as keyboards and mice that input operation signals, and output devices such as speakers that output audio data. For example, the input / output interface 105 takes in operation data input by the operator via the input device and outputs data such as video and audio to the output device.

[0024] The communication interface 106 connects to a communication network and is an interface for the parts counting device 100 to communicate data with the learning device 200 and the storage device 400. The communication interface 106 includes, for example, a network board, a wireless LAN module, etc.

[0025] (Example of the functional configuration of the parts counting device 100) Next, an example of the functional configuration of the parts counting device 100 will be described. As shown in Figure 2B, the parts counting device 100 functionally comprises a control unit 110, a storage unit 120, a display unit 130, an input / output unit 140, and a communication unit 150.

[0026] The control unit 110 is implemented, for example, by the process by which the processor 101 shown in Figure 2A executes a program that has been loaded from the storage 103 into the RAM of the memory 102.

[0027] The control unit 110 controls the overall functions of the parts counting device 100. The control unit 110 includes a parts counting unit 111 that counts the parts to be dispensed, a case information management unit 112 that manages case information related to parts counting, a parts-related information management unit 113 that manages parts-related information, and a parts counting information management unit 114.

[0028] The parts counting unit counts the parts to be dispensed. It includes a parts shape counting unit 111a that counts based on the shape of the parts, and a parts mass counting unit 111b that counts based on the mass of the parts.

[0029] The part shape counting unit 111a acquires, for example, images captured by an imaging device IE installed on the production line, and uses known image analysis techniques to calculate the quantity of each part based on the shape of the part shown in the image. The part shape counting unit 111a also reads the label attached to the packaging bag of the part and records the part drawing number linked to the shape. In cases where, for example, it is difficult to recognize the shape of the part, or when the part label is difficult to read, the shape, part drawing number, etc. may be registered by an operator through manual input.

[0030] The parts mass counting unit 111b calculates the quantity of each part based on its mass, for example, using a weighing device WE installed on the production line. The parts mass counting unit 111b works in conjunction with the parts shape counting unit 111a to recognize the parts, and counts each part by comparing the part drawing number, mass, etc., with pre-registered parts mass data.

[0031] The case information management unit 112 stores case information regarding the counting of parts that have actually occurred in the case information database 121 of the storage unit 120. For example, the case information management unit 112 generates case information based on user input information and stores the generated case information in the case information database 121. In the description of this embodiment, "database" may be written as "DB".

[0032] The parts-related information management unit 113 stores parts-related information regarding the parts counting results from the parts counting unit 111 in the parts-related information database 122 of the storage unit 120. The parts-related information management unit 113, for example, refers to case information stored in the case information database 121 and appropriately adjusts the variability of the parts counting results from the parts shape counting unit 111a and the parts mass counting unit 111b of the parts counting unit 111 to generate more reliable parts counting results. The parts-related information management unit 113 stores parts-related information, including parts counting results with improved accuracy based on the case information, in the parts-related information database 122.

[0033] The parts counting information management unit 114 generates parts counting information by adding other information to parts-related information and stores it in the storage unit 120. Specifically, as shown in Figure 5A, the parts counting information is information that can be represented in a table format, with each part having associated information such as "part drawing number," "dispatch quantity," "shape," "image count value," "mass," "mass count value," "dispatch frequency," and "imported image." Here, "shape" is represented by, for example, a polygon mesh, voxel data, point cloud data, etc. Also, "dispatch frequency" is expressed as a numerical value on a 10-point scale, with a higher number indicating a higher frequency of issuance. For example, the parts counting information management unit 114 generates parts counting information by grouping the parts necessary for the manufacture of each product. For example, when the parts counting information management unit 114 receives inference result information from the inference device 300 showing the inference result from a trained model, it generates parts counting information and displays the parts counting information via the display unit 130. This allows users to easily understand the counting status of each component.

[0034] The storage unit 120 stores control programs executed by the control unit 110, various data, etc. The storage unit 120 includes, for example, an incident information database 121 that stores incident information related to the counting of parts that have actually occurred, and a parts-related information database 122 that stores parts-related information including captured images of parts, mass measurement values, and count values ​​based on each of these.

[0035] The display unit 130 displays various information according to the control unit 110. For example, the display unit 130 displays images related to the generation process of parts-related information on the display screen. The input / output unit 140 receives various data from the input device and outputs various data to the output device according to the control unit 110. The communication unit 150 transmits and receives various data to and from the parts counting device 100, the inference device 300, and the storage device 400 via wireless LAN according to the control unit 110. For example, the communication unit 150 transmits parts-related information to the learning device 200 and receives inference information from the inference device 300.

[0036] (Example of hardware configuration for learning device 200) Next, an example of the hardware configuration of the learning device 200 will be described. As shown in Figure 3A, the learning device 200 physically comprises a processor 201, memory 202, storage 203, display device 204, input / output interface (input / output I / F) 205, and communication interface (communication I / F) 206. Each of these components is electrically connected to the others via a bus line 207. Note that the example of the hardware configuration of the learning device 200 is the same as the example of the hardware configuration of the parts counting device 100, so a detailed explanation will be omitted.

[0037] The processor 201 is an arithmetic unit that controls the operation of the entire learning device 200. The processor 201 is a general-purpose processor such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit). Furthermore, the processor 201 is not limited to a general-purpose processor, but may also be a dedicated processor composed of an ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc.

[0038] Memory 202 is the main memory and includes ROM (Read Only Memory), RAM (Random Access Memory), etc. The processor 201 controls and functions the entire learning device 200 by reading programs and various data from ROM or storage 203 onto RAM and executing processing.

[0039] Storage 203 is an auxiliary storage device that stores programs and various data necessary for program execution, and includes non-volatile storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives). Storage 203 stores, for example, the OS (Operating System), which is the basic software that controls the entire learning device 200, and applications that run on the OS and provide various functions. Note that some of the programs and various data necessary for program execution may be stored in ROM.

[0040] The display device 204 is an image display device such as an LCD (Liquid Crystal Display), PDP (Plasma Display Panel), or organic EL (Electro-Luminescence) display, and displays various images according to the control of the processor 201. For example, the display device 204 displays images related to the trained model generation process on the display screen. The display device 204 may also be used in a configuration connected to the input / output interface 205.

[0041] The input / output interface 205 is an interface for connecting to input devices such as keyboards and mice that input operation signals, and output devices such as speakers that output audio data. For example, the input / output interface 205 takes in operation data entered by the operator via the input device and outputs data such as video and audio to the output device.

[0042] The communication interface 206 connects to a communication network and is an interface for the learning device 200 to communicate data with the parts counting device 100 and the storage device 400. The communication interface 206 includes, for example, a network board, a wireless LAN module, etc.

[0043] (Example of functional configuration of learning device 200) Next, the functional configuration of the learning device 200 will be described. As shown in Figure 3B, the learning device 200 functionally comprises a control unit 210, a storage unit 220, a display unit 230, an input / output unit 240, and a communication unit 250.

[0044] The control unit 210 is implemented, for example, by the process by which the processor 201 shown in Figure 3A executes a program that has been loaded from the storage 203 into the RAM of the memory 202. The storage unit 220 is implemented, for example, by the memory 202 and storage 203 shown in Figure 3A. The input / output unit 240 is implemented, for example, by the input / output interface 205 shown in Figure 3A. The communication unit 250 is implemented by the communication interface 206 shown in Figure 3A.

[0045] The control unit 210 controls the overall functions of the learning device 200. The control unit 210 includes a learning data acquisition unit 211 that acquires learning data, a trained model generation unit 212 that generates a trained model using the learning data, and a trained model output unit 213 that outputs the trained model.

[0046] The learning data acquisition unit 211 acquires data stored in the storage unit 120 of the parts counting device 100 as learning data from the parts counting device 100 via the communication unit 250. The learning data acquisition unit 211 acquires parts-related information, including, for example, captured images of parts, mass measurements, and count values ​​based on each of these, from the parts counting device 100 as learning data. Note that the learning data acquisition unit 211 is an example of a learning data acquisition means.

[0047] The trained model generation unit 212 generates a trained model through machine learning using part-related information. The trained model generation unit 212 generates a trained model that learns the relationship between the captured image and mass measurement values ​​of parts and their quantities by performing machine learning using the part-related information acquired as training data by the training data acquisition unit 211. Note that the trained model generation unit 212 is an example of a trained model generation means.

[0048] The learning algorithm used by the trained model generation unit 212 can be any known algorithm, such as supervised learning, unsupervised learning, or reinforcement learning. In this embodiment, the trained model generation unit 212 learns the quantity of parts corresponding to part-related information by so-called supervised learning, for example, according to a neural network model. Here, supervised learning is a learning method that derives the correct answer using training data that includes the correct answer data.

[0049] A neural network consists of an input layer, a hidden layer, and an output layer, each containing multiple neurons. Furthermore, by using so-called deep learning techniques that employ convolutional neural networks with three or more layers, it is possible to perform more effective learning.

[0050] For example, in the three-layer neural network shown in Figure 6, when multiple data points are input to input layers X1 to X3, each of these values ​​is multiplied by weights W1-11 to W1-32 and output to hidden layers Y1 and Y2. Each value input to hidden layers Y1 and Y2 is further multiplied by weights W2-11 to W2-23 and output from output layers Z1 to Z3. The output result will differ depending on the values ​​of the weights W1-11 to W1-32 and W2-11 to W2-23.

[0051] The trained model generation unit 212 learns the quantity of parts through supervised learning using a neural network, according to training data created based on combinations of captured images and mass measurements of parts acquired by the training data acquisition unit 211 and the counting results of the parts. In other words, the trained model generation unit 212 learns by inputting captured images of parts into the input layer of the neural network and adjusting the weights W1-11 to W1-32 and W2-11 to W2-23 so that the result output from the output layer approaches the counting results of the parts.

[0052] The trained model output unit 213 outputs the trained model generated by the trained model generation unit 212 to the storage device 400 via the communication unit 250 for storage. Note that the trained model output unit 213 is an example of a trained model output means.

[0053] The memory unit 220 stores control programs executed by the control unit 210 and various data. The display unit 230 displays various information according to the control of the control unit 210. The input / output unit 240 receives various data from the input device and outputs various data to the output device according to the control of the control unit 210. The communication unit 250 transmits and receives various data to and from the parts counting device 100, the inference device 300, and the storage device 400 via wireless LAN according to the control of the control unit 210. For example, the communication unit 250 receives training data from the parts counting device 100. The communication unit 250 also transmits, for example, a trained model generated based on the training data to the storage device 400.

[0054] (Example hardware configuration of inference device 300) Next, an example of the hardware configuration of the inference device 300 will be described. As shown in Figure 4A, the inference device 300 physically comprises a processor 301, memory 302, storage 303, display device 304, input / output interface (input / output I / F) 305, and communication interface (communication I / F) 306. Each of these components is electrically connected to the others via a bus line 307. Note that the example of the hardware configuration of the inference device 300 is the same as the example of the hardware configuration of the parts counting device 100 and the learning device 200, so a detailed explanation will be omitted.

[0055] (Example of the functional configuration of the inference device 300) Next, an example of the functional configuration of the inference device 300 will be described. As shown in Figure 4B, the inference device 300 functionally comprises a control unit 310, a storage unit 320, a display unit 330, an input / output unit 340, and a communication unit 350.

[0056] The control unit 310 is implemented, for example, by the process by which the processor 301 shown in Figure 4A executes a program that has been loaded from the storage 303 into the RAM of the memory 302. The storage unit 320 is implemented, for example, by the memory 302 and storage 303 shown in Figure 4A. The input / output unit 340 is implemented, for example, by the input / output interface 305 shown in Figure 4A. The communication unit 350 is implemented by the communication interface 306 shown in Figure 4A.

[0057] The control unit 310 controls the overall functions of the inference device 300. The control unit 310 includes an inference data acquisition unit 311 that acquires inference data, an inference unit 312 that infers the quantity of parts, and an inference result information output unit 313 that outputs inference result information.

[0058] The inference data acquisition unit 311 acquires data stored in the storage unit 120 of the parts counting device 100 as inference data via the communication unit 350. It acquires the imaged image and mass measurement values ​​of the parts to be inferred as inference data. For example, the inference data acquisition unit 311 acquires the imaged image and mass measurement values ​​of the parts to be inferred from the parts counting device 100 as inference data. Note that the inference data acquisition unit 311 is an example of an inference data acquisition means.

[0059] The inference unit 312 generates inference result information by inferring the quantity of parts using a trained model. First, the inference unit 312 reads the trained model from the storage device 400 via the communication unit 350. The inference unit 312 inputs the inference data acquired by the inference data acquisition unit 311 into the trained model, thereby generating inference result information that shows the inference result for the quantity of the parts to be inferred. The inference result information is information that can be represented in a table format, for example, as shown in Figure 5B, in which the "part drawing number" and the "inference result" showing the predicted quantity of each part are associated. Note that the inference unit 312 is an example of an inference means.

[0060] The inference result information output unit 313 outputs the inference result information generated by the inference unit 312. The inference result information output unit 313 transmits the inference result information to the parts counting device 100, for example, via the communication unit 350. The inference result information output unit 313 may also display the inference result information on the display unit 230. The inference result information output unit 313 is an example of an inference result information output means.

[0061] The memory unit 320 stores control programs executed by the control unit 310 and various data. The display unit 330 displays various information according to the control of the control unit 310. The input / output unit 340 receives various data from the input device and outputs various data to the output device according to the control of the control unit 310. The communication unit 350 transmits and receives various data to and from the parts counting device 100, the learning device 200, and the storage device 400 via wireless LAN according to the control of the control unit 310. For example, the communication unit 350 receives parts-related information from the parts counting device 100. Also, for example, the communication unit 350 receives learned models generated by the learning device 200 from the storage device 400.

[0062] Next, the operation of the parts counting system 1 having the above configuration will be described. The flowchart shown in Figure 7 is an example of the operation of the parts counting system 1, and is a flowchart relating to the trained model generation process executed by the learning device 200.

[0063] The control unit 210 of the learning device 200 starts the training model generation process in response to an operation input from the user of the parts counting system 1 to the parts counting device 100, for example, after the parts counting period by the parts counting device 100 has ended.

[0064] When the trained model generation process is started, the control unit 210 first acquires training data (step S101). The training data acquisition unit 211 of the control unit 210 acquires the training data necessary to generate a trained model by receiving component-related information, including captured images of components, mass measurements, and count values ​​based on each of these, transmitted from the component counting device 100 via the communication unit 250. The training data acquisition unit 211 may acquire the component-related information to be acquired as training data all at once, or it may acquire it in multiple stages.

[0065] Next, the trained model generation unit 212 generates a trained model by performing machine learning using the training information (step S102). The trained model generation unit 212 generates a trained model using all the training data acquired from the parts counting device 100.

[0066] Next, the trained model output unit 213 outputs the trained model generated by the trained model generation unit 212 to the storage device 400 (step S103). The storage device 400 stores the trained model input from the learning device 200, either as a new model or an updated model.

[0067] After executing the process in step S104, the control unit 210 terminates the trained model generation process. At this time, the control unit 210 transmits a signal to the parts counting device 100 via the communication unit 250 indicating that a trained model has been generated. Similarly, the control unit 210 may output a message via the display unit 230 or the input / output unit 240 indicating that a trained model has been generated.

[0068] Next, referring to the flowchart shown in Figure 8, we will describe an example of the operation of the parts counting system 1, specifically the inference result information generation process in which the inference device 300 generates inference result information. The control unit 310 of the inference device 300 starts the inference result information generation process in response to an operation input from the user of the parts counting system 1 to the parts counting device 100 instructing it to start inference, after the learning device 200 has generated a trained model.

[0069] First, the inference data acquisition unit 311 of the control unit 310 acquires inference data (step S201). The inference data acquisition unit 311 acquires inference data by receiving the captured image and mass measurement value of the component to be inferred transmitted from the component counting device 100 via the communication unit 350.

[0070] Next, the inference unit 312 inputs inference data into the trained model and generates inference result information (step S202). Specifically, the inference unit 312 reads the trained model stored in the storage device 400, inputs inference data into this trained model, and generates inference result information indicating the quantity of the component to be inferred based on the inference result output from the trained model.

[0071] Next, the inference result information output unit 313 outputs the inference result information generated by the inference unit 312 (step S203). The inference result information output unit 313 transmits the inference result information generated by the inference unit 312 to the parts counting device 100 via the communication unit 350. The inference result information output unit 313 may also display the inference result information generated by the inference unit 312, for example, via the display unit 330.

[0072] After executing the process in step S203, the control unit 310 terminates the inference result information generation process. After the termination of the inference result information generation process, the trained model may be updated by adding the inference data and inference result information verified in this step to the training data and running the trained model generation process again.

[0073] As described above, according to the parts counting system 1 of this embodiment, in the learning device 200, the learning data acquisition unit 211 acquires the captured image of the part, the mass measurement value, and the count value based on each of these as learning data. The trained model generation unit 212 then generates a trained model by machine learning using the learning data. In the inference device 300, the inference data acquisition unit 311 acquires the captured image and mass measurement value of the part to be inferred as inference data. The inference unit 312 then inputs the inference data into the trained model to generate inference result information, and the inference result information output unit 313 outputs the inference result information.

[0074] As a result, the learning device 200 can generate a trained model through machine learning using training data based on image analysis and mass measurements. In other words, the learning device 200 can learn the relationship between image analysis, mass measurements, and the quantity of parts, taking into account not only the image analysis of the parts but also the mass measurements of the parts. Therefore, the inference device 300 can predict the quantity of parts using a trained model that incorporates various elements such as captured images of parts and count values ​​based on mass measurements. As a result, the parts counting system 1 according to this embodiment 1 can verify the quantity of parts to be dispensed with greater accuracy than a parts counting system that does not generate a trained model through machine learning using count values ​​based on captured images and mass measurements, respectively.

[0075] This disclosure is not limited to the embodiments described above, and various modifications and applications are possible without departing from the spirit of this disclosure.

[0076] (modified version) In the above embodiment, the inference device 300 predicted the quantity of the component to be inferred using a trained model generated by the learning device 200. However, the trained model used by the inference device 300 may be a trained model created by a learning device other than the learning device 200, and the inference device 300 can output inference result information based on this trained model.

[0077] Furthermore, in the above embodiment, the parts counting device 100, the learning device 200, the inference device 300, and the storage device 400 were described as separate devices. However, some or all of these devices may be combined. For example, the learning device 200, the inference device 300, and the storage device 400 may be combined to form the same device.

[0078] Furthermore, even if the parts-related information management unit 113 of the parts counting device 100 adjusts the counting results by referring to the case information stored in the case information database 121, if the image count value based on the captured image and the mass count value based on the mass measurement value for the same part still show different values, and variability occurs in the training data, the learning device 200 may, for example, employ a learning algorithm that is robust to variable data, such as random forest regression or gradient boosting, to generate a trained model.

[0079] Furthermore, although the above embodiment describes the parts counting device 100, learning device 200, inference device 300, and storage device 400 as being capable of transmitting and receiving data via a wireless LAN, it is not limited to this. Some or all of these devices may transmit and receive data by being connected communicatively, for example, via a communication cable. Also, some or all of these devices may transmit and receive data via the internet, for example. Moreover, for example, the parts counting device 100, learning device 200, inference device 300, and storage device 400 may be logically implemented by cloud computing.

[0080] Furthermore, in the above embodiment, the learning device 200 pre-installed in the parts counting system 1 performed machine learning only on training data based on parts-related information acquired from the parts counting device 100 to generate a trained model, but this is not limited to this. For example, the learning device 200 of the parts counting system 1 may be a learning device that performed machine learning using training data generated based on information acquired from an information source provided in another parts counting system, and the trained model may be updated by retraining using training data acquired from the parts counting device 100.

[0081] In the above embodiment, for example, the control program executed by the processor 101 that implements the control unit 110 of the parts counting device 100 was mainly stored in storage 103 beforehand. However, this disclosure is not limited thereto, and the control program for executing the above-mentioned various processes may be implemented in an existing general-purpose computer, framework, workstation, etc., to function as a device equivalent to the parts counting device 100 according to the above embodiment. The same applies to the control program executed by the processor 201 that implements the control unit 210 of the learning device 200, and the control program executed by the processor 301 that implements the control unit 310 of the inference device 300.

[0082] The method of providing such programs is optional. For example, they may be distributed by storing them on a computer-readable storage medium (flexible disk, CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM), or they may be stored on network storage such as the Internet and provided for download.

[0083] Furthermore, if the above processing is performed through a division of labor between the OS (Operating System) and the application program, or through collaboration between the OS and the application program, only the application program may be stored on a recording medium, storage, etc. It is also possible to superimpose the program onto a carrier wave and distribute it over a network. For example, the above program may be posted on a bulletin board system (BBS) on a network and distributed over the network. The program may then be designed to execute the above processing by launching it and running it under the control of the OS, just like any other application program.

[0084] This disclosure allows for various embodiments and modifications without departing from the broad spirit and scope of this disclosure. Furthermore, the embodiments described above are for illustrative purposes only and do not limit the scope of this disclosure. In other words, the scope of this disclosure is indicated by the claims, not by the embodiments. Various modifications made within the scope of the claims and the equivalent significance of the disclosure are considered to be within the scope of this disclosure.

[0085] The various aspects of this disclosure are summarized below as an appendix. (Note 1) A parts counting system for counting the quantity of parts to be dispensed in a dispensing process in which parts necessary for product production are retrieved from a warehouse, A storage device that stores a trained model which has learned the relationship between the captured image and the mass measurement and the quantity of the part by machine learning using part-related information including the captured image of the part, the mass measurement, and the count value based on each of these, and example information showing examples related to the counting of the part. The system includes an inference device that applies the captured image and the mass measurement to the trained model to infer the quantity of the part, Parts counting system. (Note 2) The inference device is An inference data acquisition means for acquiring the captured image and the mass measurement value of the subject of inference as inference data, An inference means for inputting the inference data into the trained model and inferring the quantity of the parts, Includes an inference result information output means that outputs inference result information showing the inference result by the inference means, The parts counting system described in Appendix 1. (Note 3) The system further comprises a learning device for generating the aforementioned trained model, The learning device is A means for acquiring the aforementioned component-related information as training data, A trained model generation means generates the trained model by machine learning using the training data acquired by the training data acquisition means, The system includes a trained model output means that outputs the trained model generated by the trained model generation means, The parts counting system described in Appendix 1 or 2. (Note 4) The system further comprises a parts counting device that supplies the parts-related information to the learning device and supplies the captured image and mass measurement values ​​of the inference target to the inference device. The parts counting system described in Appendix 3. (Note 5) A learning device for a dispensing process in which parts necessary for product production are retrieved from a warehouse, which learns the relationship between the captured image and mass measurement of the parts to be dispensed and the quantity of the parts, A learning data acquisition means for acquiring part-related information, including the captured image, the mass measurement value, and the count value based on each of these, as learning data; A trained model generation means that generates a trained model by machine learning using the aforementioned training data, The system comprises a trained model output means that outputs the trained model generated by the trained model generation means, Learning device. (Note 6) In a dispensing process in which parts necessary for product production are retrieved from a warehouse, an inference device for inferring the quantity of parts to be dispensed, An inference data acquisition means for acquiring the captured image and mass measurement values ​​to be inferred as inference data, An inference means that uses machine learning with the captured image, the mass measurement value, and part-related information including count values ​​based on each thereof to learn the relationship between the captured image and the mass measurement value and the quantity of the part, to input the inference data acquired by the inference data acquisition means, and infers the quantity of the part; The system includes an inference result information output means that outputs inference result information showing the inference result obtained by the inference means, Reasoning device. (Note 7) A parts counting method performed by a parts counting system that counts the quantity of parts to be issued in a dispensing process in which parts necessary for product production are taken out of a warehouse, The computer included in the aforementioned parts counting system The captured image and mass measurements to be inferred are acquired as inference data. The inference data is input into a trained model that has learned the relationship between the captured image, the mass measurement, and the quantity of the part through machine learning using the captured image, the mass measurement, and part-related information including the count values ​​based on each of these, and the quantity of the part is inferred. Output inference result information showing the inference result. Parts counting method. (Note 8) Computers, An inference data acquisition means that acquires the captured image and mass measurement values ​​to be inferred as inference data. An inference means that uses machine learning with the captured image, the measured mass, and part-related information including count values ​​based on each thereof to learn the relationship between the captured image and the measured mass and the quantity of the part, inputs the inference data acquired by the inference data acquisition means into a trained model, and infers the quantity of the part. This means functions as an inference result information output means that outputs inference result information showing the inference result by the aforementioned inference means. program. [Explanation of Symbols]

[0086] 1...Parts counting system, 100...Parts counting device, 101...Processor, 102...Memory, 103...Storage, 104...Display device, 105...Input / output interface, 106...Communication interface, 107...Bus line, 110...Control unit, 111...Parts counting unit, 111a...Parts shape counting unit, 111b...Parts mass counting unit, 112...Case information management unit, 113...Parts-related information management unit, 114...Parts counting information management unit, 120...Storage unit, 121...Case information database, 122...Parts-related information database, 130...Display unit, 140...Input / output unit, 150...Communication unit, 200...Learning device, 201...Processor, 202...Memory, 203...Storage, 204...Display device, 205...Input / output Interface, 206…Communication interface, 207…Bus line, 210…Control unit, 211…Training data acquisition unit, 212…Trained model generation unit, 213…Trained model output unit, 220…Storage unit, 230…Display unit, 240…Input / output unit, 250…Communication unit, 300…Inference device, 301…Processor, 302…Memory, 303…Storage, 304…Display device, 305…Input / output interface, 306…Communication interface, 307…Bus line, 310…Control unit, 311…Inference data acquisition unit, 312…Inference unit, 313…Inference result information output unit, 320…Storage unit, 330…Display unit, 340…Input / output unit, 350…Communication unit, 400…Storage device, IE…Imaging device, WE…Weighing device

Claims

1. A parts counting system for counting the quantity of parts to be dispensed in a dispensing process in which parts necessary for product production are retrieved from a warehouse, A storage device that stores a trained model which has learned the relationship between the captured image and the mass measurement and the quantity of the part by machine learning using part-related information including the captured image of the part, the mass measurement, and the count value based on each of these, and example information showing examples related to the counting of the part. The system includes an inference device that applies the captured image and the mass measurement to the trained model to infer the quantity of the part, Parts counting system.

2. The inference device is An inference data acquisition means for acquiring the captured image and the mass measurement value of the subject of inference as inference data, An inference means for inputting the inference data into the trained model and inferring the quantity of the parts, Includes an inference result information output means that outputs inference result information showing the inference result by the inference means, The parts counting system according to claim 1.

3. The system further comprises a learning device for generating the aforementioned trained model, The learning device is A means for acquiring the aforementioned component-related information as training data, A trained model generation means generates the trained model by machine learning using the training data acquired by the training data acquisition means, The system includes a trained model output means that outputs the trained model generated by the trained model generation means, The parts counting system according to claim 1 or 2.

4. The system further comprises a parts counting device that supplies the parts-related information to the learning device and supplies the captured image and mass measurement values ​​of the inference target to the inference device. The parts counting system according to claim 3.

5. A learning device for a dispensing process in which parts necessary for product production are retrieved from a warehouse, which learns the relationship between the captured image and mass measurement of the parts to be dispensed and the quantity of the parts, A learning data acquisition means for acquiring part-related information, including the captured image, the mass measurement value, and the count value based on each of these, as learning data; A trained model generation means that generates a trained model by machine learning using the aforementioned training data, The system comprises a trained model output means that outputs the trained model generated by the trained model generation means, Learning device.

6. In a dispensing process in which parts necessary for product production are retrieved from a warehouse, an inference device for inferring the quantity of parts to be dispensed, An inference data acquisition means for acquiring the captured image and mass measurement values ​​to be inferred as inference data, An inference means that uses machine learning with the captured image, the mass measurement value, and part-related information including count values ​​based on each thereof to learn the relationship between the captured image and the mass measurement value and the quantity of the part, to input the inference data acquired by the inference data acquisition means, and infers the quantity of the part; The system includes an inference result information output means that outputs inference result information showing the inference result obtained by the inference means, Reasoning device.

7. A parts counting method performed by a parts counting system that counts the quantity of parts to be issued in a dispensing process in which parts necessary for product production are taken out of a warehouse, The computer included in the aforementioned parts counting system The captured image and mass measurements to be inferred are acquired as inference data. The inference data is input into a trained model that has learned the relationship between the captured image, the mass measurement, and the quantity of the part through machine learning using the captured image, the mass measurement, and part-related information including the count values ​​based on each of these, and the quantity of the part is inferred. Output inference result information showing the inference result. Parts counting method.

8. Computers, An inference data acquisition means that acquires the captured image and mass measurement values ​​to be inferred as inference data. An inference means that uses machine learning with the captured image, the measured mass, and part-related information including count values ​​based on each thereof to learn the relationship between the captured image and the measured mass and the quantity of the part, inputs the inference data acquired by the inference data acquisition means into a trained model, and infers the quantity of the part. This means functions as an inference result information output means that outputs inference result information showing the inference result by the aforementioned inference means. program.

Images