Product inspection support device, product inspection support method, and program

Abstract

To make it easy to send captured images of a product under inspection to a terminal(s) of an appropriate inspector(s), which was not easy with conventional technologies.SOLUTION: A product inspection support device 1 provided herein allows captured images of a product under inspection to be sent to a terminal of an appropriate inspector(s), the product inspection support device comprising a captured image acquisition unit 122 for acquiring captured images of the product under inspection, an inspector determination unit 125 configured to determine one or more inspectors who should receive the captured images from among two or more inspectors, and a captured image transmission unit 131 for sending the captured images to an inspector terminal(s) 2 of the one or more inspectors determined by the inspector determination unit 125.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a product inspection support device and the like that support product inspection by transmitting a photographed image of a product to an inspector terminal.

Background Art

[0002] Conventionally, there has been a technique of obtaining an inspection result regarding the appearance of a product by giving an appearance image of a product to a module that performs prediction processing of machine learning using a prediction model obtained by performing learning processing of machine learning and executing the module (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, it has not been assumed that two or more inspectors perform product inspection. As a result, it has been impossible to transmit a photographed image of a product to be inspected to an appropriate inspector's terminal.

[0005] In particular, in the prior art, it has not been assumed that two or more inspectors perform product inspection remotely, and it has not been done to transmit a photographed image to the terminals of two or more inspectors and receive inspection results from the inspectors.

Means for Solving the Problems

[0006] The product inspection support device of the present invention comprises: an image acquisition unit that acquires images of a product to be inspected; an inspector determination unit that determines one or more inspectors from among two or more inspectors to transmit the images; and an image transmission unit that transmits the images to the inspector terminals of the one or more inspectors determined by the inspector determination unit.

[0007] This configuration allows images of the product being inspected to be transmitted to the appropriate inspector's terminal.

[0008] Furthermore, the product inspection support device of this second invention, compared to the first invention, further comprises a trap image determination unit that determines whether or not each of two or more inspectors meets the trap image transmission conditions, which are the conditions for transmitting a trap image, and the captured image transmission unit transmits the trap image to the inspector terminal of each of the one or more inspectors that the trap image determination unit has determined to meet the trap image transmission conditions.

[0009] This configuration can increase the accuracy of the examiner's tests.

[0010] Furthermore, the product inspection support device of this third invention, compared to the second invention, includes an inspection result receiving unit that receives inspection results, which are responses to the transmission of trap images, from the inspector's terminal; a response determination unit that determines whether the inspection results are correct or not; and a captured image transmission unit, which performs processing related to the control of image transmission to the inspector when the response determination unit determines that the inspection results corresponding to the transmission of trap images are incorrect, and also performs processing for inappropriate persons.

[0011] This configuration can increase the accuracy of the examiner's tests.

[0012] Furthermore, the product inspection support device of the fourth invention further comprises an inspector information storage unit that stores two or more inspector information items, each having one or more inspector attribute values, for any one of the first to third inventions, and the inspector determination unit determines one or more inspectors to transmit the captured images using one or more inspector attribute values ​​possessed by each of the two or more inspector information items.

[0013] This configuration allows images of the product being inspected to be transmitted to the appropriate inspector's terminal.

[0014] Furthermore, the product inspection support device of the fifth invention further comprises a product information storage unit that stores product information having one or more product attribute values ​​which are attribute values ​​of a product or a captured image, and the inspector determination unit determines one or more inspectors to transmit the captured image using one or more inspector attribute values ​​which are possessed by two or more inspector information and one or more product attribute values ​​which are possessed by the product information.

[0015] This configuration allows images of the product being inspected to be transmitted to the terminal of the most appropriate inspector.

[0016] Furthermore, the product inspection support device of the sixth invention, compared to the fourth or fifth invention, further comprises an inspection result receiving unit that receives inspection results which are responses corresponding to the transmission of captured images and which are responses transmitted by the inspector terminals of two or more inspectors, and an inspector attribute value acquisition unit that acquires one or more inspector attribute values ​​for each of the two or more inspectors using the inspection results received by the inspection result receiving unit, wherein the one or more inspector attribute values ​​stored in the inspector information storage unit are the one or more inspector attribute values ​​acquired by the inspector attribute value acquisition unit.

[0017] This configuration allows images of the product being inspected to be transmitted to the appropriate inspector's terminal. Furthermore, this configuration enables the reception of inspection results from two or more remote inspectors. This allows for the effective utilization of the labor of cloud workers in product inspection.

[0018] In addition, the product inspection support device of the seventh invention performs a learning process of machine learning using two or more pieces of teacher data having a captured image and an inspection result for any one of the first to sixth inventions, and stores the obtained learning model in a learning model storage unit. The product inspection support device further includes a prediction unit that performs a prediction process of machine learning using the captured image acquired by the captured image acquisition unit and the learning model to obtain a prediction result. The captured image transmission unit is a product inspection support device that transmits the captured image and the prediction result to the inspection terminals of one or more inspectors determined by the inspector determination unit.

[0019] With such a configuration, the inspection by the inspector can be supported.

[0020] In addition, the product inspection support device of the eighth invention further includes an inspection result reception unit that receives, from the inspection terminal of the inspector, an inspection result that is a response corresponding to the transmission of the captured image and has a captured image identifier for identifying the captured image and position information for specifying the location of an error, and an inspection result output unit that outputs the inspection result received by the inspection result reception unit.

[0021] With such a configuration, the captured image of the product to be inspected can be transmitted to the terminal of an appropriate inspector, and the inspection result can be obtained from the inspector.

[0022] In addition, the product inspection support device of the ninth invention further includes a teacher data storage unit that stores two or more pieces of teacher data having a captured image and position information, a learning unit that performs a learning process of machine learning using the two or more pieces of teacher data to obtain a learning model, and a prediction unit that performs a prediction process of machine learning using the captured image acquired by the captured image acquisition unit and the learning model to obtain a prediction result. The captured image transmission unit is a product inspection support device that transmits the captured image and the prediction result to the inspection terminals of one or more inspectors determined by the inspector determination unit.

[0023] With such a configuration, the inspection by the inspector can be supported.

Effects of the Invention

[0024] According to the product inspection support device of the present invention, a captured image of a product to be inspected can be transmitted to a terminal of an appropriate inspector.

Brief Description of the Drawings

[0025] [Figure 1] Conceptual diagram of inspection system A in Embodiment 1 [Figure 2] Block diagram of the inspection system A [Figure 3] Block diagram of the product inspection support device 1 [Figure 4] Flowchart for explaining an operation example of the product inspection support device [Figure 5] Flowchart for explaining an example of the prediction process [Figure 6] Flowchart for explaining an example of the inspector determination process [Figure 7] Flowchart for explaining an example of the transmission information configuration process [Figure 8] Flowchart for explaining an example of the inspection result process [Figure 9] Flowchart for explaining an example of the transmission process of the trap image [Figure 10] Flowchart for explaining an operation example of the inspector terminal 2 [Figure 11] Diagram showing the product information management table [Figure 12] Diagram showing the inspector information management table [Figure 13] Diagram showing the trap image management table [Figure 14] Diagram showing the output example [Figure 15] Overview diagram of the computer system [Figure 16] Block diagram of the computer system

Modes for Carrying Out the Invention

[0026] The embodiments of the product inspection support device, etc., will be described below with reference to the drawings. In the embodiments, components that are denoted by the same reference numerals perform the same operation, and therefore, further explanation may be omitted.

[0027] (Embodiment 1) In this embodiment, an inspection system is described that includes a product inspection support device that transmits images of one or more products for product inspection to two or more inspector terminals. The products to be inspected are, for example, products produced on a production line. Preferably, the products to be inspected are products that flow through the production line one after another and are produced in succession. However, the production location of the products is not a factor. Furthermore, it is preferable that the inspection be performed while the products are in production.

[0028] Furthermore, in this embodiment, we will describe an inspection system that includes a product inspection support device that transmits a trap image to the inspector's terminal when the conditions for transmitting a trap image, for example, for determining whether or not the inspector is becoming tired, are met.

[0029] Furthermore, in this embodiment, we will describe an inspection system equipped with a product inspection support device that performs inappropriate person processing when the inspection result for the transmission of a trap image is incorrect. Inappropriate person processing refers to processing for an inspector who is deemed inappropriate to perform the inspection.

[0030] Furthermore, in this embodiment, we will describe an inspection system that includes a product inspection support device that determines the inspector terminal to transmit the captured image using one or more inspector attribute values.

[0031] Furthermore, in this embodiment, we will describe an inspection system that includes a product inspection support device that determines the inspector terminal to transmit the captured image using one or more product attribute values.

[0032] Furthermore, in this embodiment, we will describe an inspection system that includes a product inspection support device that acquires one or more inspector attribute values ​​using inspection results transmitted from the inspector terminal. This document describes an inspection system equipped with a product inspection support device.

[0033] Furthermore, this embodiment describes an inspection system that includes a product inspection support device that transmits, along with the captured image, the machine learning-based prediction results for the captured image to the inspector's terminal.

[0034] Furthermore, in this embodiment, we will describe an inspection system that includes a product inspection support device that receives inspection results having a captured image identifier and location information identifying the location of an error from an inspector's terminal.

[0035] Furthermore, in this embodiment, we will describe an inspection system that includes a product inspection support device that performs machine learning learning processing using two or more training data sets and constructs a learning model.

[0036] In this embodiment, the association of information X with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not limited. Information X and information Y may be linked, may exist in the same buffer, may information X be contained in information Y, or information Y may be contained in information X, and so on.

[0037] Figure 1 is a conceptual diagram of inspection system A in this embodiment. Inspection system A comprises a product inspection support device 1 and one or more inspector terminals 2.

[0038] Product inspection support device 1 transmits images of the product to be inspected to one or more inspector terminals 2 and receives inspection results from the one or more inspector terminals 2. Note that the device used to receive and utilize the inspection results may be a separate device from product inspection support device 1.

[0039] Product inspection support device 1 is, for example, a server that photographs a product, receives the captured image from a camera (not shown) that acquired the image, and transmits the captured image to one or more inspector terminals 2.

[0040] Product inspection support device 1 is an information processing device that, for example, has the function of photographing a product and acquiring the captured image, and transmits the captured image to one or more inspector terminals 2. The camera of product inspection support device 1 and the devices that realize other functions may be physically separate or integrated. Product inspection support device 1 may also receive the captured image transmitted from the device that photographed the product.

[0041] Product inspection support device 1 is, for example, a so-called server, such as a cloud server or ASP server, but the type is not limited. Product inspection support device 1 may also be, for example, a so-called personal computer with a camera function, a tablet terminal, a smartphone, etc.

[0042] The inspector terminal 2 receives captured images from the product inspection support device 1, outputs the captured images, and receives inspection results from the inspector based on the captured images. It is preferable for the inspector terminal 2 to transmit the inspection results to the product inspection support device 1 or other devices. However, the inspector terminal 2 may simply store or display the inspection results.

[0043] The examiner's terminal 2 can be, for example, a personal computer, tablet, or smartphone, and the type does not matter.

[0044] In Figure 1, the product inspection support device 1 is configured to include a camera, but the product inspection support device 1 does not necessarily have to include a camera. In Figure 1, B is a production line for producing products. 3 is the product to be produced. In Figure 1, product 3 is an iron and a microwave oven, but the type of product is not limited. In Figure 1, 4 is an inspector. Each inspector 4 usually operates one inspector terminal 2.

[0045] The product inspection support device 1 and one or more inspector terminals 2 can usually communicate via a network such as the internet or a LAN. In other words, it is preferable that the inspector terminals 2 are located remotely from the place where the product is manufactured. It is preferable that the inspectors are people who perform product inspections remotely. In other words, it is preferable that the inspectors are so-called cloud workers who perform product inspections remotely.

[0046] Figure 2 is a block diagram of inspection system A in this embodiment. Figure 3 is a block diagram of product inspection support device 1.

[0047] The product inspection support device 1 comprises a storage unit 11, a processing unit 12, a transmission unit 13, a receiving unit 14, and an output unit 15. The storage unit 11 comprises a training data storage unit 111, a learning model storage unit 112, a product information storage unit 113, an inspector information storage unit 114, a transmission condition storage unit 115, and a trap image storage unit 116. The processing unit 12 comprises a learning unit 121, a captured image acquisition unit 122, a prediction unit 123, an inspector attribute value acquisition unit 124, an inspector determination unit 125, a trap image judgment unit 126, and a response judgment unit 127. The transmission unit 13 comprises a captured image transmission unit 131. The receiving unit 14 comprises an inspection result receiving unit 141. The output unit 15 comprises an inspection result output unit 151.

[0048] The inspector terminal 2 comprises a terminal storage unit 21, a terminal receiving unit 22, a terminal processing unit 23, a terminal output unit 24, a terminal reception unit 25, and a terminal transmission unit 26.

[0049] The storage unit 11, which constitutes the product inspection support device 1, stores various types of information. These types of information include, for example, training data (described later), a learning model (described later), product information (described later), inspector information (described later), and transmission conditions (described later).

[0050] The training data storage unit 111 stores two or more training data. Training data is information that forms the basis for constructing a learning model. Training data is, for example, a positive example image or a negative example image. A positive example image is an image of a product without defects. A negative example image is an image of a product with defects. Note that the positive and negative examples may be reversed. Training data includes, for example, an image and an inspection result. The inspection result is the result of the inspection. The inspection result may be, for example, "defect present (e.g., "1")", "no defect (e.g., "0")", "location information", and "defect type identifier". Location information is information that identifies the location of the defect in the product. Location information may also be information that identifies the area of ​​the defect in the product. Location information is, for example, one or more relative coordinate values ​​(x,y) that identify the location. Location information is, for example, one or more coordinate values ​​that identify the relative location in the image. Location information consists of, for example, two coordinate values ​​that identify a rectangular area (e.g., the top-left coordinate value and the bottom-right coordinate value). Training data consists of, for example, captured images and location information. A defect type identifier is information that identifies the type of defect.

[0051] A defect is a flaw in a product. Examples of defects in resin products include foreign matter inclusion, contamination, white spots, black spots, color unevenness, cracks, and breaks. Examples of defects in metal products include scratches and dents. In other words, the defect type identifiers are, for example, "foreign matter inclusion," "contamination," "white spots," "black spots," "color unevenness," "cracks," and "breaks." However, the type and content of the defect are not considered.

[0052] Preferably, the training data storage unit 111 stores two or more training data for each product type. Preferably, each of the two or more training data is associated with a product type identifier. A product type identifier is information that identifies the type of product being tested. For example, a product type identifier could be "iron cassette" or "microwave oven," but it could also be information that identifies the product number of the product being tested.

[0053] The learning model storage unit 112 stores one or more learning models. It is preferable that the learning model storage unit 112 stores a learning model for each product type. It is preferable that each of the two or more learning models is associated with a product type identifier.

[0054] A learning model is information constructed through the learning process of machine learning using two or more training data sets, and is used for prediction processing in machine learning. The learning model is used in conjunction with captured images to obtain prediction results for product inspection within those images. A learning model can also be called a learner, classifier, or classification model.

[0055] The machine learning algorithm can be deep learning, random forest, decision tree, SVM, etc. Furthermore, various machine learning functions and existing libraries can be used, such as the TensorFlow library, the R language's random forest module, fastText, and TinySVM. The machine learning process is performed by the learning unit 121 (described later), and the prediction process is performed by the prediction unit 123.

[0056] The product information storage unit 113 stores one or more product information items. Product information refers to information about the product being inspected. Each product information item has one or more product attribute values. Product attribute values ​​are attribute values ​​of the product or a captured image. Examples of product attribute values ​​include difficulty of discrimination, product defect rate, and product type identifier. Difficulty of discrimination refers to the difficulty of determining whether or not a product has a defect. The difficulty of discrimination can be, for example, a five-level scale from "1" to "5", or one of "A", "B", or "C", but the specific information is not limited to these.

[0057] The inspector information storage unit 114 stores one or more inspector information. Typically, the inspector information storage unit 114 stores two or more inspector information.

[0058] Inspector information refers to information about the inspector. Inspector information has one or more inspector attribute values. For example, inspector information may include an inspector identifier and destination information.

[0059] Inspector attribute values ​​can be static or dynamic. Static attribute values ​​are those that do not change frequently. Examples of static attribute values ​​include the inspector's skill level, product type identifier of products the inspector has experience inspecting, gender, age, years of inspection experience, and unit price (remuneration for product inspection). Dynamic attribute values ​​are those that change frequently. Examples of dynamic attribute values ​​include the average response time over a specified period, the accuracy rate over a specified period, and the accuracy rate of trap images over a specified period. The specified period can be, for example, today, from the start of inspection work until yesterday, or from the start of inspection work until now; there is no specific timeframe.

[0060] The examiner identifier is information that identifies the examiner. For example, the examiner identifier may include the examiner's ID, email address, phone number, and destination information. The destination information is the information used to send information to examiner terminal 2. The destination information may include the IP address of examiner terminal 2, the MAC address of examiner terminal 2, the examiner's ID, the examiner's email address, and the examiner's phone number.

[0061] The trap image accuracy rate is the percentage of test results that were correct for a given trap image. A trap image is an image used for testing the examiner. For example, a trap image can be said to be an image used to check the examiner's condition. For example, a trap image can be said to be an image used to check whether the examiner is tired or not, or whether the examiner can perform the test appropriately.

[0062] The transmission condition storage unit 115 stores one or more transmission conditions. A transmission condition is a condition for transmitting an image for inspection. The image for inspection is either a captured image or a trap image. In other words, the transmission condition is, for example, a captured image transmission condition or a trap image transmission condition. The transmission conditions stored in the transmission condition storage unit 115 may also be embedded in the program.

[0063] The image transmission conditions are the conditions for transmitting captured images to one inspector terminal 2. These conditions include, for example, information indicating the time interval between image transmissions, conditions regarding the number of transmissions per unit period (e.g., 1 day, 4 hours), and time-related conditions. For example, one image transmission condition might indicate that one inspection should be performed every 10 seconds. A condition regarding the number of transmissions per unit period might be, for example, that the maximum number of images transmitted within the unit period has not been reached. A time-related condition might be that a predetermined amount of time has elapsed since the previous transmission. Another image transmission condition might be that the product type identifier of the product being inspected is included in one or more product type identifiers that indicate the inspector's past inspection experience.

[0064] Trap image transmission conditions are the conditions for transmitting trap images. Examples of trap image transmission conditions include conditions based on response time, conditions based on the number of captured images transmitted, and conditions based on inspection time.

[0065] Response time refers to information about the time it takes for the examiner to respond to a captured image. For example, response time is the time from when the captured image is sent to examiner terminal 2 until the examiner terminal 2 receives the test result. For example, response time is the time from when the captured image is output to examiner terminal 2 until examiner terminal 2 receives the test result. Conditions based on response time include, for example, "response time >= threshold", "response time > threshold", "change in response time (extension of response time) is greater than or equal to the threshold", and "change in response time (extension of response time) is greater than the threshold".

[0066] The condition based on the number of times captured images are transmitted is that the number of times captured images have been transmitted to one inspector terminal 2 after the start of the inspection work or after the trap image has been transmitted first reaches the threshold.

[0067] The conditions based on inspection time are the elapsed time since the start of the inspection and the elapsed time since the first trap image was transmitted. The conditions based on inspection time are when the elapsed time since the start of the inspection or the elapsed time since the first trap image was transmitted is equal to or greater than the threshold.

[0068] The trap image storage unit 116 stores one or more trap images. It is preferable that each of the one or more trap images in the trap image storage unit 116 is associated with correct answer information. It is also preferable that each trap image is associated with a trap image identifier.

[0069] Correct answer information refers to information about the correct response to a trouble image. For example, correct answer information may include "defect present," location information of the defect, and a defect type identifier. Preferably, the trouble image is an image of a defective product. Preferably, each of the one or more trouble images is associated with a product type identifier. Such a product type identifier is the product type identifier of the product from which the trouble image was taken.

[0070] A trouble image identifier is information that identifies a trouble image.

[0071] The processing unit 12 performs various processes. These various processes include, for example, those performed by the learning unit 121, the captured image acquisition unit 122, the prediction unit 123, the examiner attribute value acquisition unit 124, the examiner determination unit 125, the trap image judgment unit 126, and the response judgment unit 127.

[0072] The learning unit 121 uses two or more training data stored in the training data storage unit 111 to perform machine learning training and obtain a trained model. It is preferable for the learning unit 121 to obtain two or more training data corresponding to each product type identifier from the training data storage unit 111 for each of the one or more product type identifiers, and then use these two or more training data to perform machine learning training and obtain a trained model.

[0073] The learning unit 121 preferably stores the acquired learning model in the learning model storage unit 112. The learning unit 121 preferably stores the acquired learning model in the learning model storage unit 112 after associating it with a product type identifier. The algorithm for the machine learning learning process performed by the learning unit 121 is not limited, as described above.

[0074] The image acquisition unit 122 acquires images of the product to be inspected. The images are usually still images, but may also be videos with two or more still images (frames). The images may also be images obtained by applying predetermined processing to images captured by a camera. The images may also include images based on captured images. An image obtained by applying predetermined processing is, for example, an image obtained by applying resolution enhancement processing to an image captured by a camera. An image obtained by applying predetermined processing is, for example, an image obtained by making an image captured by a camera clearer using a machine learning algorithm such as GAN. Preferably, the image obtained by applying predetermined processing is an image obtained by processing an image captured by a camera to make it easier for the inspector to inspect.

[0075] The image acquisition unit 122 may acquire an image (which can also be called an image) that is a portion of the image taken of the product to be inspected. The area to be extracted may be a predetermined area, or it may be an area that is likely to have defects as a result of machine learning prediction processing.

[0076] The image acquisition unit 122, for example, photographs the product to be inspected and acquires the image. In this case, the image acquisition unit 122 is equipped with a camera function.

[0077] The image acquisition unit 122 receives, for example, an image transmitted from a camera that has photographed the product to be inspected.

[0078] The image acquisition unit 122 preferably constitutes transmission information that includes the captured image. The transmission information is information transmitted to the inspector's terminal. The transmission information preferably includes, for example, the captured image, as well as one or two of the following: a captured image identifier and a product type identifier. The transmission information preferably also includes, for example, the prediction results described later. The captured image identifier can be said to be the identifier of a single product being inspected.

[0079] A captured image identifier is information that identifies a captured image. A captured image identifier is information that identifies a single manufactured product. A product type identifier is information that indicates the type of product.

[0080] The prediction unit 123 uses the captured image acquired by the captured image acquisition unit 122 and the learning model in the learning model storage unit 112 to perform machine learning prediction processing and obtain prediction results. The prediction results may include, for example, whether or not there is a defect. The prediction results may include, for example, whether or not there is a defect and a score. The prediction results may include, for example, whether or not there is a defect and information identifying the area where there is a defect. The prediction results may also include, for example, information identifying the type of defect.

[0081] The prediction unit 123 obtains, for example, a learning model paired with a product type identifier corresponding to a captured image from the learning model storage unit 112. Next, the prediction unit 123 provides the captured image and the learning model to a machine learning prediction processing module, executes the module, and obtains the prediction result.

[0082] The inspector attribute value acquisition unit 124 acquires one or more inspector attribute values ​​for each of the two or more inspectors, using the inspection results received by the inspection result receiving unit 141. These one or more inspector attribute values ​​are usually dynamic attribute values. It is preferable for the inspector attribute value acquisition unit 124 to store the acquired one or more inspector attribute values ​​in the inspector information storage unit 114.

[0083] The inspector attribute value acquisition unit 124 may acquire one or more inspector attribute values ​​from the inspector information storage unit 114.

[0084] The examiner attribute value acquisition unit 124 acquires, for example, the response times corresponding to two or more test results received by the test result receiving unit 141 that are associated with the same examiner identifier, and obtains the average response time, which is the average of those response times. When acquiring the average response time, it is preferable to use only the response times from today, but response times from past days may also be used.

[0085] The examiner attribute value acquisition unit 124 acquires, for example, a response result indicating whether the received examination result is correct or not in response to the transmission of a trap image.

[0086] The examiner attribute value acquisition unit 124 acquires the response result (usually "correct" or "incorrect") for the inspection result received in response to the transmission of two or more trap images, and uses these two or more response results to acquire the accuracy rate.

[0087] The inspector selection unit 125 determines one or more inspectors from among two or more inspectors to transmit the captured images. The inspector selection unit 125 may also determine all candidate inspectors. All candidate inspectors are inspectors corresponding to each inspector information in the inspector information storage unit 114.

[0088] The inspector determination unit 125 preferably determines a predetermined number of inspectors for a single captured image. Furthermore, the inspector determination unit 125 preferably changes the number of inspectors determined for a single captured image according to the product attribute values. For example, the inspector determination unit 125 preferably determines a larger number of inspectors the greater the difficulty of discrimination, which is a product attribute value. Similarly, the inspector determination unit 125 preferably determines a larger number of inspectors the greater the defect rate, which is a product attribute value. The predetermined number is preferably two or more.

[0089] The inspector determination unit 125 determines, for example, one or more inspectors to transmit captured images to a single inspector terminal 2, such that there is a predetermined time interval between the transmission of captured images. In such cases, for example, the time when the most recent captured image was transmitted, or the elapsed time since the previous captured image was transmitted, is stored in association with the inspector identifier.

[0090] The examiner determination unit 125 preferably determines one or more examiners to transmit the captured images using one or more examiner attribute values ​​possessed by each of the two or more examiner information. The examiner attribute values ​​are stored in the examiner information storage unit 114. The examiner determination unit 125 determines one or more examiners to transmit the captured images, for example, using the average response time of the examiners. Typically, examiner terminals 2 of examiners with shorter average response times receive a larger number of captured images.

[0091] The examiner determination unit 125, for example, determines one or more examiners from among the examiners, excluding those whose answers correspond to incorrect answers in response to the transmission of a trap image. This process can be considered an example of inappropriate person processing.

[0092] The examiner determination unit 125, for example, obtains the examiner attribute value, which is the examiner's average response time, and the time when the captured image was first transmitted, for each examiner, and uses the average response time and the time to determine one or more examiners who have a large difference between the elapsed time from that time and the average response time.

[0093] The inspector determination unit 125 preferably determines one or more inspectors to transmit the captured images using one or more inspector attribute values ​​from each of the two or more inspector information sets and one or more product attribute values ​​from the product information set.

[0094] The inspector determination unit 125 may, for example, vary the number of inspectors to be determined depending on the product. The inspector determination unit 125 determines a number of inspectors corresponding to one or more attribute values ​​of the product. One or more attribute values ​​of the product are, for example, product importance, product defect rate, and difficulty of discrimination. The inspector determination unit 125 obtains, for example, the defect rate or difficulty of discrimination, which is a product attribute value, and obtains the number of inspectors corresponding to that defect rate or difficulty of discrimination. For example, the inspector determination unit 125 determines a larger number of inspectors the higher the defect rate or difficulty of discrimination. The inspector determination unit 125 obtains the number of inspectors corresponding to the obtained defect rate or difficulty of discrimination by referring to a correspondence table. The correspondence table has two or more correspondence pieces of information, which are pairs of conditions for the defect rate or difficulty of discrimination (for example, the range of the defect rate, the difficulty of discrimination) and the number of inspectors. The inspector determination unit 125 obtains the number of inspectors by using an increasing function with the defect rate or difficulty of discrimination as a parameter.

[0095] The inspector determination unit 125 determines, for example, one or more inspectors whose inspector information includes the same product type identifier as the product information of the product to be inspected. The product type identifier included in the inspector information is the product type identifier of a product that the inspector has previously inspected.

[0096] The inspector determination unit 125 determines, for example, one or more inspectors who meet the conditions for transmitting captured images.

[0097] The trap image determination unit 126 determines whether each of the two or more examiners meets the trap image transmission conditions. The trap image transmission conditions are stored in the transmission condition storage unit 115. It is preferable for the trap image determination unit 126 to determine whether one or more dynamic attribute values ​​of the examiner meet the trap image transmission conditions.

[0098] The response determination unit 127 determines whether the test result received by the test result receiving unit 141 is correct or not.

[0099] The response determination unit 127 makes a determination using, for example, the correct answer information associated with the transmitted trouble image and the inspection result received by the inspection result receiving unit 141. The response determination unit 127 determines, for example, whether the correct answer information associated with the transmitted trouble image and the inspection result received by the inspection result receiving unit 141 meet the correct answer conditions.

[0100] The correct answer condition is the condition for determining that the test result is correct. For example, the correct answer condition is that the correct answer information matches the test result, or that the difference between the location information included in the correct answer information and the location information included in the test result is below a threshold or less than a threshold.

[0101] The answer determination unit 127, for example, uses the inspection results transmitted from two or more inspector terminals 2 for a single captured image to determine whether each of the two or more inspection results is correct. The answer determination unit 127, for example, uses the two or more inspection results for a single captured image to determine whether each of the two or more inspection results meets the correct answer conditions. The correct answer conditions are, for example, that all inspection results are the same, or that the result is the one with the larger number among three or more inspection results.

[0102] The two or more inspection results for a single captured image are the inspection results entered by the inspectors at each inspector terminal 2 that received the single captured image.

[0103] The answer determination unit 127 determines, for example, that two or more inspection results for a single captured image are all the same, and that the two or more inspection results are correct. The answer determination unit 127 determines, for example, that if only one of three or more inspection results for a single captured image is different, that one inspection result is incorrect and the other inspection results are correct.

[0104] The answer determination unit 127, for example, if one or more of the two or more inspection results for a single captured image indicate that there is a "defect," it determines that the inspection result indicating "no defect" is incorrect.

[0105] The transmitting unit 13 transmits various types of information. These types of information include, for example, captured images and transmission information.

[0106] The captured image transmission unit 131 transmits the captured image to the examiner terminal 2 of each of the one or more examiners determined by the examiner determination unit 125. The captured image transmission unit 131 transmits, for example, the captured image and the prediction result to the examiner terminal 2 of each of the one or more examiners determined by the examiner determination unit 125. The prediction result is information acquired by the prediction unit 123. The captured image transmission unit 131 transmits, for example, the transmission information to the examiner terminal 2 of each of the one or more examiners determined by the examiner determination unit 125.

[0107] The captured image transmission unit 131 transmits a trap image to the inspector terminal 2 of one or more inspectors that the trap image determination unit 126 has determined to meet the trap image transmission conditions. Such a trap image is one of the trap images in the trap image storage unit 116. In this case, it is also preferable for the captured image transmission unit 131 to transmit a trap image identifier and a flag indicating that it corresponds to a trap image.

[0108] The captured image transmission unit 131 may transmit different trap images depending on the inspector. For example, it is preferable for the captured image transmission unit 131 to retrieve a trap image from the trap image storage unit 116 that corresponds to a product type identifier that was previously sent to one inspector, and transmit the said trap image to the inspector's terminal 2.

[0109] The image transmission unit 131 performs inappropriate person processing, which is processing related to image transmission control, when the response determination unit 127 determines that the inspection result received by the inspection result reception unit 141 is incorrect. Note that inappropriate person processing may also be performed by other components such as the processing unit 12.

[0110] Inappropriate user handling typically involves processing related to the control of image transmission. For example, inappropriate user handling might involve transmitting images of a product other than the one previously transmitted to the inspector terminal 2. For example, inappropriate user handling might involve refraining from transmitting images to the inspector terminal 2 for at least a predetermined period of time.

[0111] The captured image transmission unit 131 may transmit the captured image only if the prediction result obtained by the prediction unit 123 satisfies the captured image transmission conditions. The captured image transmission conditions here are, for example, that the prediction result is "defect found". The captured image transmission conditions are, for example, that the score of the prediction result is below or less than a threshold (when the likelihood of the prediction result is low). The captured image transmission conditions are, for example, that the prediction result is "defect found" or that the prediction result is "no defect" AND the score is below or less than a threshold. In addition, the captured image transmission conditions here can be any conditions based on the prediction result.

[0112] The receiving unit 14 receives various types of information. These types of information include, for example, inspection results and an identifier for the inspection results and captured image.

[0113] The inspection result receiving unit 141 receives inspection results, which are responses corresponding to the transmission of a single captured image and are responses transmitted by the inspector terminals 2 of two or more inspectors. The inspection results are, for example, associated with a captured image identifier that identifies a single captured image. The inspection results may also contain lot information and inspection date and time information for the product corresponding to the captured image.

[0114] The inspection result receiving unit 141 receives, for example, inspection results from the inspector's terminal 2, which are responses corresponding to the transmission of trap images. Such inspection results are associated with the identifier of the trap image.

[0115] The inspection result receiving unit 141 receives, for example, an inspection result from the inspector's terminal 2, which is a response to the transmission of the captured image, and includes a captured image identifier that identifies the captured image and location information that identifies the location of the error.

[0116] The output unit 15 outputs various types of information. These types of information include, for example, inspection results. Other types of information include, for example, inspection results and captured image identifiers.

[0117] Here, "output" is a concept that includes transmission to external devices, storage on recording media, transfer of processing results to other processing devices or other programs, display on a screen, projection using a projector, printing with a printer, and sound output.

[0118] The inspection result output unit 151 outputs the inspection results received by the inspection result receiving unit 141. The inspection result output unit 151 stores the inspection results received by the inspection result receiving unit 141, associating them with the captured image identifier. The inspection result output unit 151 stores the inspection results in, for example, the storage unit 11.

[0119] If the inspection result output unit 151 indicates that there is a "defect," it is preferable to display the information including the inspection result on a display or notify the administrator. Notification to the administrator may include sending to an administrator terminal (not shown) or displaying on a display viewed by the administrator. The information including the inspection result may include, for example, the inspection result, the captured image, and the captured image identifier.

[0120] Various types of information are stored in the terminal storage unit 21, which constitutes the inspector terminal 2. These types of information include, for example, an inspector identifier.

[0121] The terminal receiving unit 22 receives various types of information. These types of information include, for example, captured images and transmission information. The transmission information includes, for example, a flag indicating whether or not it corresponds to a trouble image.

[0122] The terminal processing unit 23 performs various processes. These processes include, for example, converting received information and instructions into information and instructions for transmission. Other processes include, for example, converting received information into information for output.

[0123] The terminal processing unit 23 obtains, for example, the response time. The terminal processing unit 23 obtains, for example, the response time which is the time from when the terminal output unit 24 outputs the captured image until the terminal reception unit 25 receives the test result.

[0124] The terminal processing unit 23 contains information to be transmitted to the product inspection support device 1, for example, and includes information including inspection results. This information includes, for example, one or more of the following: response time, captured image identifier, inspector identifier of the terminal storage unit 21, product lot information, inspection date and time information, and a flag indicating whether or not it corresponds to a trouble image.

[0125] The terminal output unit 24 outputs various types of information. These types of information include, for example, captured images, transmission information, and trap images.

[0126] The terminal reception unit 25 receives various types of information and instructions. These types of information and instructions include, for example, test results.

[0127] The means of inputting various types of information and instructions can be anything, such as a touch panel, keyboard, mouse, or menu screen.

[0128] The terminal transmission unit 26 transmits various information and instructions to the product inspection support device 1. These various information and instructions include, for example, inspection results, inspection results and captured image identifiers, and information generated by the terminal processing unit 23.

[0129] The storage unit 11, training data storage unit 111, learning model storage unit 112, product information storage unit 113, inspector information storage unit 114, transmission condition storage unit 115, trap image storage unit 116, and terminal storage unit 21 are preferably made of non-volatile recording media, but can also be made of volatile recording media.

[0130] The process by which information is stored in the storage unit 11, etc. is not relevant. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information input via an input device may be stored in the storage unit 11, etc.

[0131] The processing unit 12, learning unit 121, captured image acquisition unit 122, prediction unit 123, examiner attribute value acquisition unit 124, examiner determination unit 125, trap image judgment unit 126, response judgment unit 127, and terminal processing unit 23 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 12, etc., are usually implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0132] The transmitting unit 13, the captured image transmitting unit 131, and the terminal transmitting unit 26 are usually implemented by wireless or wired communication means, but may also be implemented by broadcasting means.

[0133] The receiving unit 14, the inspection result receiving unit 141, and the terminal receiving unit 22 are usually implemented by wireless or wired communication means, but they may also be implemented by means of receiving broadcasts.

[0134] The output unit 15 and the test result output unit 151 can be implemented, for example, by driver software for an output device or by driver software for an output device and an output device. The output unit 15 and the test result output unit 151 can be implemented, for example, by wireless or wired communication means. The output unit 15 and the test result output unit 151 can be implemented, for example, by a processor or memory.

[0135] The terminal output unit 24 may or may not be considered to include output devices such as a display or speakers. The terminal output unit 24 can be implemented using driver software for an output device, or a driver software for an output device and an output device.

[0136] The terminal reception unit 25 can be implemented using device drivers for input means such as touch panels and keyboards, or control software for menu screens, etc.

[0137] Next, we will explain an example of the operation of inspection system A. First, we will explain an example of the operation of the product inspection support device using the flowchart in Figure 4.

[0138] (Step S401) The image acquisition unit 122 determines whether or not it has acquired an image. If an image has been acquired, the process proceeds to step S402; otherwise, the process proceeds to step S410.

[0139] Acquiring captured images, for example, involves photographing the product or receiving images of the product taken by a camera (not shown). The camera that photographs the product is, for example, installed at the end of the production line that produces the product, and photographs the product after production is complete. Furthermore, it is preferable that the products of this application flow one after another along the production line and are produced sequentially.

[0140] Furthermore, it is preferable that the captured images obtained here include, in addition to the captured image, one or more of the following pieces of information: product type identifier, captured image identifier, lot information, and date and time.

[0141] (Step S402) The prediction unit 123 performs prediction processing. An example of prediction processing will be explained using the flowchart in Figure 5. Prediction processing is the process of obtaining the results of predictions regarding defects in the captured image acquired in step S401.

[0142] (Step S403) The inspector determination unit 125, etc., determines one or more inspectors to perform the inspection on the captured images acquired in step S401. An example of this inspector determination process will be explained using the flowchart in Figure 6.

[0143] (Step S404) The processing unit 12 configures the transmission information. An example of this transmission information configuration process will be explained using the flowchart in Figure 7. The transmission information is the information that is sent to the inspector terminal 2.

[0144] (Step S405) The captured image transmission unit 131 assigns 1 to counter i.

[0145] (Step S406) The captured image transmission unit 131 determines whether or not there is an i-th examiner among the one or more examiners determined in step S403. If there is an i-th examiner, the process proceeds to step S407; otherwise, the process returns to step S401.

[0146] (Step S407) The captured image transmission unit 131 obtains the destination information of the i-th examiner from the examiner information storage unit 114. The captured image transmission unit 131 transmits the transmission information configured in step S404 to the destination indicated by the destination information.

[0147] (Step S408) The processing unit 12 updates the examiner information of the i-th examiner in response to the transmission of information to the i-th examiner. For example, the processing unit 12 adds 1 to the examiner attribute value "Number of examinations" in the examiner information.

[0148] (Step S409) The captured image transmission unit 131 increments the counter i by 1. Return to step S406.

[0149] (Step S410) The inspection result receiving unit 141 determines whether or not it has received inspection results, etc. from the inspector terminal 2. If inspection results, etc. have been received, the process proceeds to step S411; otherwise, the process proceeds to step S415.

[0150] The inspection results, for example, include, in addition to the inspection results, one or more pieces of information from the following: a flag indicating whether or not it corresponds to a trouble image, a product type identifier, a captured image identifier or trouble image identifier, an inspector identifier, response time, and date and time.

[0151] (Step S411) The trap image determination unit 126 determines whether the inspection result etc. received in step S410 corresponds to the trap image. If it corresponds to the trap image, the unit proceeds to step S412; otherwise, the unit proceeds to step S414.

[0152] (Step S412) The trap image determination unit 126 obtains correct information corresponding to the trap image corresponding to the received inspection result, etc., from the trap image storage unit 116. The trap image determination unit 126 uses this correct information and the inspection result received in step S410 to determine whether the inspection result received in step S410 is correct or not. If the inspection result is correct, the unit returns to step S401; if the inspection result is incorrect, the unit proceeds to step S413.

[0153] (Step S413) The captured image transmission unit 131 performs inappropriate person processing on the inspector who has transmitted the inspection results, etc. Return to step S401.

[0154] Furthermore, the handling of inappropriate individuals involves sending a request to the inspector to take a break for a specified period (e.g., 10 minutes), refraining from sending captured images for that period, and sending captured images corresponding to a different product type identifier than the product type identifier of the product that was being inspected immediately before.

[0155] (Step S414) The processing unit 12 performs inspection result processing. The process returns to step S401. Note that inspection result processing is processing using the received inspection results. An example of inspection result processing will be explained using the flowchart in Figure 8.

[0156] (Step S415) The processing unit 12 determines whether or not it is time to send the trap image. If it is time to send the trap image, the unit proceeds to step S416; otherwise, it proceeds to step S420.

[0157] The timing of sending trap images can be continuous or at predetermined intervals. The timing of sending trap images is not specified.

[0158] (Step S416) The processing unit 12 assigns 1 to counter i.

[0159] (Step S417) The processing unit 12 determines whether or not the i-th examiner exists. If the i-th examiner exists, the process goes to step S418; otherwise, it returns to step S401.

[0160] (Step S418) The processing unit 12, etc., performs the trap image transmission process. An example of the trap image transmission process will be explained using the flowchart in Figure 9.

[0161] (Step S419) The processing unit 12 increments counter i by 1. The process returns to step S417.

[0162] (Step S420) The receiving unit 14 determines whether or not it has received the training data. If it has received the training data, it proceeds to step S421; otherwise, it proceeds to step S422. It is preferable that the receiving unit 14 receives both the training data and the product type identifier.

[0163] (Step S421) The processing unit 12 stores the training data received in step S420 in the training data storage unit 111, associating it with the product type identifier. The process returns to step S401.

[0164] (Step S422) The processing unit 12 determines whether or not to perform the learning process. If it decides to perform the learning process, it proceeds to step S423; otherwise, it returns to step S401. For example, if the receiving unit 14 receives a learning instruction, the processing unit 12 decides to perform the learning process. However, the timing of the learning process is not specified.

[0165] (Step S423) The learning unit 121 retrieves two or more training data stored in the training data storage unit 111 for each product type identifier. Next, the learning unit 121 uses the two or more training data for each product type identifier to perform machine learning training and obtain a training model.

[0166] (Step S424) The learning unit 121 stores the learning model acquired in step S423 in the learning model storage unit 112, associating it with each product type identifier. The process returns to step S401.

[0167] In the flowchart shown in Figure 4, processing is terminated by power-off or processing termination interrupts.

[0168] Next, an example of the prediction process in step S402 will be explained using the flowchart in Figure 5.

[0169] (Step S501) The prediction unit 123 obtains a product type identifier corresponding to the received captured image. The prediction unit 123 obtains a learning model that is paired with the product type identifier from the learning model storage unit 112.

[0170] (Step S502) The prediction unit 123 acquires the received captured image.

[0171] (Step S503) The prediction unit 123 provides the learning model acquired in step S501 and the captured images acquired in step S502 to a machine learning prediction processing module, executes the module, and obtains the prediction result. It then returns to the higher-level processing.

[0172] Next, an example of the inspector determination process in step S403 will be explained using the flowchart in Figure 6.

[0173] (Step S601) The inspector determination unit 125 obtains one or more product attribute values ​​from the product information storage unit 113 that are paired with a product type identifier corresponding to the captured image.

[0174] (Step S602) The inspector determination unit 125 assigns 1 to counter i.

[0175] (Step S603) The examiner attribute value acquisition unit 124 determines whether or not the i-th examiner exists. If the i-th examiner exists, the unit proceeds to step S604; otherwise, the unit proceeds to step S608.

[0176] (Step S604) The examiner attribute value acquisition unit 124 acquires one or more examiner attribute values ​​for the i-th examiner from the examiner information storage unit 114.

[0177] (Step S605) The inspector determination unit 125 uses one or more inspector attribute values ​​obtained in step S604 and one or more product attribute values ​​obtained in step S601 to determine whether the i-th inspector meets the captured image transmission conditions. If the captured image transmission conditions are met, the process proceeds to step S606; otherwise, the process proceeds to step S607.

[0178] (Step S606) The inspector determination unit 125 obtains the inspector identifier of the i-th inspector and temporarily stores the inspector identifier in a buffer (not shown).

[0179] (Step S607) The examiner determination unit 125 increments the counter i by 1. Return to step S603.

[0180] (Step S608) The examiner determination unit 125 obtains one or more examiner identifiers from the one or more examiner identifiers temporarily stored in a buffer (not shown) in step S606. The examiner determination unit 125 obtains, for example, all destination information that is paired with each of the one or more examiner identifiers temporarily stored in the buffer (not shown) in step S606 from the examiner information storage unit 114. The examiner determination unit 125 obtains, for example, a predetermined number of examiner identifiers randomly from the one or more examiner identifiers temporarily stored in the buffer (not shown) in step S606, and obtains destination information that is paired with each of the one or more examiner identifiers from the examiner information storage unit 114. The examiner determination unit 125 obtains, for example, a predetermined number of examiner identifiers that are paired with a high accuracy rate from the one or more examiner identifiers temporarily stored in the buffer (not shown) in step S606, and obtains destination information that is paired with each of the one or more examiner identifiers from the examiner information storage unit 114. The inspector determination unit 125, for example, obtains a predetermined number of inspector identifiers that are paired with the lowest unit price (remuneration) from among the one or more inspector identifiers temporarily stored in a buffer (not shown) in step S606, and obtains destination information paired with each of the one or more inspector identifiers from the inspector information storage unit 114.

[0181] Next, an example of the transmission information configuration process in step S404 will be explained using the flowchart in Figure 7.

[0182] (Step S701) The processing unit 12 acquires a captured image from among the captured images acquired in step S401.

[0183] (Step S702) The processing unit 12 obtains the captured image identifier that is paired with the captured image.

[0184] (Step S703) The processing unit 12 obtains the product type identifier that is paired with the captured image.

[0185] (Step S704) The processing unit 12 acquires other information that corresponds to the captured image. Other information may include, for example, lot information, date and time, and product attribute values ​​(for example, defect rate).

[0186] (Step S705) The processing unit 12 configures the transmission information that has been obtained in steps S701 to S704. It returns to the higher-level processing unit.

[0187] Next, an example of the inspection result processing in step S414 will be explained using the flowchart in Figure 8.

[0188] (Step S801) The response determination unit 127 obtains the captured image identifier corresponding to the inspection result, etc., received in step S410.

[0189] (Step S802) The response determination unit 127 temporarily stores the test results, etc., received in step S410 in a buffer (not shown).

[0190] (Step S803) The response determination unit 127 determines whether there are as many inspection results corresponding to the captured image identifier obtained in step S801 as there are inspector terminals 2 from which the captured image was transmitted (whether or not inspection results have been received from all inspectors). If there are as many inspection results as there are inspector terminals 2 (final determination made), the unit proceeds to step S804; if the inspection results from all inspectors are not yet complete (no determination made), the unit returns to the higher-level process.

[0191] (Step S804) The response determination unit 127 obtains all inspection results corresponding to the captured image identifier obtained in step S801 from a buffer (not shown).

[0192] (Step S805) The answer determination unit 127 uses one or more inspection results obtained in step S804 to obtain the final inspection result (an inspection result that can be presumed to be correct). For example, if even one inspection result indicating a defect exists, the answer determination unit 127 sets the final inspection result to "defective". For example, the answer determination unit 127 sets the inspection result with the most votes as the final inspection result. Otherwise, the algorithm for determining the final inspection result from two or more inspection results is not limited.

[0193] (Step S806) The answer determination unit 127 assigns 1 to counter i.

[0194] (Step S807) The response determination unit 127 determines whether or not there is an i-th examiner who has sent the test result. If there is an i-th examiner, the unit proceeds to step S808; otherwise, the unit proceeds to step S811.

[0195] (Step S808) The answer determination unit 127 uses the test result of the i-th examiner and the final test result obtained in step S805 to obtain correct answer information for the test result of the i-th examiner. The correct answer information is, for example, "correct" or "incorrect". The correct answer information is, for example, the test result of the i-th examiner or "incorrect".

[0196] (Step S809) The answer determination unit 127 updates the examiner information of the i-th examiner (for example, accuracy rate, number of correct answers, number of errors) using the correct answer information obtained in step S808.

[0197] (Step S810) The answer determination unit 127 increments the counter i by 1. Return to step S807.

[0198] (Step S811) The inspection result output unit 151 outputs the final inspection result obtained in step S805, associating it with the captured image identifier. It then returns to the higher-level processing. If the final inspection result corresponds to a "defect", it is preferable for the inspection result output unit 151 to notify the administrator of the final inspection result, for example.

[0199] Next, an example of the trap image transmission process in step S418 will be explained using the flowchart in Figure 9.

[0200] (Step S901) The trap image determination unit 126 obtains one or more examiner attribute values ​​of the examiner to be determined from the examiner information storage unit 114. It is preferable that the one or more examiner attribute values ​​include dynamic attribute values ​​such as response time and accuracy rate.

[0201] (Step S902) The trap image determination unit 126 uses one or more examiner attribute values ​​obtained in step S901 to determine whether the examiner to be determined meets the trap image transmission conditions. If the trap image transmission conditions are met, the process proceeds to step S903; otherwise, the process returns to the higher-level processing.

[0202] (Step S903) The processing unit 12 obtains the product type identifier of the product corresponding to the inspection being performed by the inspector whose product is being judged.

[0203] (Step S904) The processing unit 12 obtains a trap image from the trap image storage unit 116 that corresponds to the product type identifier obtained in step S903.

[0204] (Step S905) The captured image transmission unit 131 obtains the destination information of the examiner to be judged from the examiner information storage unit 114.

[0205] (Step S906) The captured image transmission unit 131 transmits the trap image acquired in step S904 to the destination indicated by the destination information acquired in step S905. The process returns to the higher level.

[0206] Next, an example of the operation of the inspector terminal 2 will be explained using the flowchart in Figure 10.

[0207] (Step S1001) The terminal receiving unit 22 determines whether or not it has received an image or the like from the product inspection support device 1. If an image or the like has been received, it proceeds to step S1002; otherwise, it returns to step S1001. The image or the like includes an image, and the image is either a captured image or a trap image. The image or the like includes, for example, a captured image identifier or a trap image identifier, and a flag indicating whether or not it is a trap image.

[0208] (Step S1002) The terminal processing unit 23 uses the image etc. received in step S1001 to construct the image etc. to be output. The terminal output unit 24 outputs the image etc.

[0209] (Step S1003) The terminal reception unit 25 determines whether or not it has received the inspection result for the output image. If it has received the inspection result, it proceeds to step S1004; if it has not received the inspection result, it proceeds to step S1003.

[0210] (Step S1004) The terminal processing unit 23 obtains the response time, etc. The terminal processing unit 23 obtains the response time using, for example, a timer (not shown). The response time is, for example, the time from receiving an image, etc. in step S1001 to receiving the test result in step S1003.

[0211] (Step S1005) The terminal processing unit 23 configures information including the inspection result received in step S1003 and the response time obtained in step S1004. The information including the response time includes, for example, a captured image identifier or trap image identifier, an inspector identifier, and a flag indicating whether or not it is a trap image.

[0212] (Step S1006) The terminal transmission unit 26 transmits the information configured in step S1005 to the product inspection support device 1. Return to step S1001.

[0213] In the flowchart shown in Figure 10, processing is terminated by power-off or processing termination interrupts.

[0214] The following describes a specific example of the operation of inspection system A in this embodiment. As shown in Figure 1, two or more products, such as an iron ("Iron X") and a microwave oven ("Microwave Oven E"), are manufactured on the production line. A camera is installed after the final stage of production for each of the two or more products to photograph the product, and the images captured by each camera are successively transmitted to the product inspection support device 1.

[0215] Furthermore, the learning model storage unit 112 of the product inspection support device 1 stores a learning model identified as "Learning Model X" associated with the product type identifier "Iron X". In addition, the learning model storage unit 112 stores a learning model identified as "Learning Model E" associated with the product type identifier "Microwave Oven E".

[0216] "Learning Model X" is data stored in the learning model storage unit 112 after the learning unit 121 performs a learning process using two or more training data sets that have "captured images" and "correct answer information (presence or absence of defects, location information, defect type identifier)". "Learning Model E" is data stored in the learning model storage unit 112 after the learning unit 121 performs a learning process using one or more positive examples (captured images of microwave oven E with defects) and one or more negative examples (captured images of microwave oven E without defects).

[0217] Furthermore, the product information storage unit 113 stores the product information management table shown in Figure 11. The product information management table is a table that manages information about the product to be inspected. The product information management table manages one or more records that have an "ID" and a "dynamic attribute value". The "ID" is information that identifies the record. The "dynamic attribute value" has a "product type identifier", "difficulty of discrimination", and "defect rate (%)". The "difficulty of discrimination" is the difficulty of the inspection, and can take any of the five values ​​from "1" to "5".

[0218] Assume that the examiner information storage unit 114 stores an examiner information management table having the structure shown in Figure 12. The examiner information management table manages one or more records having "ID", "examiner identifier", "static attribute value", and "dynamic attribute value". The "static attribute value" here includes "gender", "skill level", "unit price", and "years of experience". The "dynamic attribute value" here includes "number of responses", "average response time (S)", and "trap image accuracy". Here, "number of responses", "average response time (S)", and "trap image accuracy" are assumed to be the examiner's dynamic attribute values ​​for today. Although not shown in the figure, each of the one or more records (examiner information) also has destination information. Furthermore, the examiner here is a registered cloud worker.

[0219] The transmission condition storage unit 115 stores the captured image transmission conditions "skill level >= difficulty of discrimination" and "number of inspectors <= 5". In other words, captured images are sent only to inspectors with a skill level equal to or greater than the difficulty of discrimination of the product being inspected. The maximum number of inspectors who can inspect a single captured image is assumed to be 5. The transmission condition storage unit 115 also stores the trap image transmission condition "response time - average response time >= threshold (here, for example, "20 seconds")". In other words, if the response time is longer than the threshold (20 seconds) based on the average response time for the day, the inspector is judged to be tired, and a trap image is sent to the inspector.

[0220] Furthermore, the trap image storage unit 116 stores the trap image management table shown in Figure 13. The trap image management table is a table for managing trap images. The trap image management table manages one or more records having "ID", "trap image", "trap image identifier", "product type identifier", and "correct answer information". In this case, "correct answer information" has "defect", "location information", and "defect type identifier". Note that "correct answer information" may be only "defect". "Defect" is either "present (defect exists)" or "absent (not a defect)". "Location information" is information that identifies the location (or area) of the defect. If "location information" has two coordinate values, it indicates the area with the defect. If "location information" has one coordinate value, it indicates the centroid coordinate of the area with the defect. "Defect type identifier" is, for example, "contamination", "scratch", or "crack".

[0221] In this situation, the following two specific examples will be explained. Specific example 1 is a case where the examiner of the captured image is determined, and the captured image is transmitted only to the determined examiner. Specific example 2 is a case where, upon receiving the examination results from the examiner, a decision is made as to whether or not to transmit a trap image, and the trap image is transmitted. Furthermore, in Specific example 2, inappropriate actions are taken depending on the examination results corresponding to the transmission of the trap image. Inappropriate actions here refer to "sending an instruction to take a break for a predetermined time" and "not transmitting the captured image for a predetermined time" to the examiner in question.

[0222] (Specific example 1) Assume that production of "Iron X" has begun on the production line. Next, a camera (not shown) photographs "Iron X," acquires the image, and transmits the image, paired with the product type identifier "Iron X" and the image identifier "X0001," to the product inspection support device 1.

[0223] For example, let's assume that the camera is included in the product inspection support device 1. The camera periodically photographs the finished products in accordance with the movement of the production line. The product inspection support device 1, including the camera, generates a unique image identifier each time it takes a photograph. The communication means of the product inspection support device 1 transmits the product type identifier "Iron X", the image identifier "X0001", and the photographed image to a server included in the product inspection support device 1.

[0224] Next, the image acquisition unit 122 of the product inspection support device 1 receives the captured image, etc. Next, the prediction unit 123 obtains the learning model "Learning Model X" which is paired with the product type identifier "Iron X" from the learning model storage unit 112. Next, the prediction unit 123 provides the "Learning Model X" and the received captured image to a machine learning prediction processing module, executes the module, and obtains a prediction result of "No defects (e.g., "0")". Note that the prediction unit 123 may also obtain an incorrect prediction result.

[0225] Next, the examiner determination unit 125 determines the examiner to transmit the captured images, etc., as follows. Specifically, the examiner determination unit 125 obtains the captured image transmission conditions "skill level >= difficulty of discrimination" and "number of examiners <= 5" from the transmission condition storage unit 115.

[0226] Next, the inspector determination unit 125 obtains the product type identifier "Iron X" corresponding to the captured image. Then, based on the captured image transmission condition "Skill level >= Discrimination difficulty", the inspector determination unit 125 obtains the discrimination difficulty "4" which is paired with the product type identifier "Iron X" from the product information management table (Figure 11).

[0227] Next, the examiner determination unit 125 refers to the examiner information management table in Figure 12 and obtains examiner identifiers "I001", "I002", "I025", "I038", "I055", and "I103" that are paired with skill levels that satisfy the condition "Skill level >= discrimination difficulty "4".

[0228] Next, the examiner determination unit 125, in order to satisfy the image transmission condition "number of examiners <= 5", randomly selected five examiner identifiers (here, "I001", "I002", "I025", "I038", and "I055") from the six examiner identifiers obtained. Note that the algorithm used to narrow down the examiner identifiers is irrelevant.

[0229] Through the above process, five examiners were determined for the captured image identified by the image identifier "X0001".

[0230] Next, the processing unit 12 acquires the received captured image, the product type identifier "Iron X", the captured image identifier "X0001", the current date and time, and the prediction result "No defects", and constructs transmission information containing this information.

[0231] Next, the captured image transmission unit 131 obtains the destination information corresponding to each of the five inspector identifiers "I001", "I002", "I025", "I038", and "I055" from the inspector information management table.

[0232] Next, the captured image transmission unit 131 transmits the configured transmission information to the inspector terminal 2 specified by each of the five destination pieces of information.

[0233] Next, the terminal receiving unit 22 of each of the five inspector terminals 2 receives transmission information from the product inspection support device 1. Then, the terminal processing unit 23 of each of the five inspector terminals 2 uses the received transmission information to construct the image, etc. to be output. The terminal output unit 24 outputs the image, etc. An example of such output is shown in Figure 14. In Figure 14, the captured image 1401 and the prediction result 1402 are output.

[0234] Next, suppose an inspector identified as "I001" looks at the captured image 1401, checks the checkboxes for "defective" and "contamination" in the inspection results, and inputs a graphic to identify the contaminated area 1403. Then, suppose the inspector presses the send button 1404.

[0235] The terminal reception unit 25 then receives the inspection result for the output image: "<Defect> present <Defect type identifier> Contamination <Location information>(X11,Y11) <Image> Defect image file.jpeg". Here, the location information (X11,Y11) is assumed to be the center point of the 1403 figure.

[0236] Next, the terminal processing unit 23 obtains the response time "21 seconds". Then, the terminal processing unit 23 constructs information including the inspection result, the response time "21 seconds", the inspector identifier "I001", the product type identifier "Iron X", the captured image identifier "X0001", and the prediction result "No defects".

[0237] Next, the terminal transmission unit 26 transmits the information configured by the terminal processing unit 23 to the product inspection support device 1.

[0238] Furthermore, suppose the other four inspectors, for example, enter the inspection result "No defects" and press the send button 1404. Then, from the inspector terminals 2 of the other four inspectors, information including the inspection result "No defects", response time, inspector identifier "I001", product type identifier "Iron X", captured image identifier "X0001", and prediction result "No defects" is transmitted to the product inspection support device 1.

[0239] Next, the inspection result receiving unit 141 of the product inspection support device 1 receives inspection results and other information from each of the five inspector terminals 2.

[0240] Next, the response determination unit 127 processes the inspection results using the five inspection results. Specifically, the response determination unit 127 detects if one of the five inspection results contains "<defect> present" and obtains the final inspection result "<defect> present". In this case, to ensure the superior quality of the product, if even one inspection result contains "<defect> present", the final inspection result will be "<defect> present".

[0241] Next, the inspection result output unit 151 sends the inspection result corresponding to "<Defect> found" "<Defect> found <Defect type identifier>Contamination <Location information>(X11,Y11) <Image>Defect image file.jpeg" to the administrator's terminal (not shown).

[0242] Next, the administrator's terminal receives and outputs the inspection results. As a result, the production line manager can identify defective products and dispose of them. This helps maintain high product quality.

[0243] Furthermore, the response determination unit 127 updates the dynamic attribute values ​​of five examiners by increasing the accuracy rate and number of correct answers (not shown) of examiner identifier "I001" in the examiner management table in Figure 12, and decreasing the accuracy rate (not shown) and increasing the number of errors (not shown) of examiners with examiner identifiers "I002", "I025", "I038", and "I055".

[0244] (Specific example 2) After more than one hour had passed since the start of the "Iron X" test, the average response time for the examiner identified by examiner identifier "I002" had increased from "15 seconds" to "50 seconds" due to fatigue or blurred vision, etc.

[0245] In other words, the inspection result receiving unit 141 of the product inspection support device 1 receives an inspection result, etc., which includes the inspector identifier "I002", the inspection result, and the response time "50 seconds". Then, the inspection result processing described in Specific Example 1 is performed.

[0246] Next, the trap image determination unit 126 obtains the trap image transmission condition "response time - average response time >= 20 seconds" from the transmission condition storage unit 115.

[0247] Next, the trap image determination unit 126 obtains the average response time "15" which is paired with the examiner identifier "I002" from the examiner information management table (Figure 12). The trap image determination unit 126 also obtains the received response time "50 seconds". Then, the trap image determination unit 126 determines that "50 - 15 >= 20 seconds". In other words, the trap image determination unit 126 determines that the response time of examiner identifier "I002" satisfies the trap image transmission condition.

[0248] Next, the processing unit 12 obtains the product type identifier "Iron X" for the product corresponding to the inspection being performed by the inspector whose product is being judged.

[0249] Next, the processing unit 12 retrieves the trap image that corresponds to the product type identifier "Iron X" from the trap image management table (Figure 13). Here, it is assumed that one trap image (for example, the trap image with "ID=1") is arbitrarily retrieved from the three trap images that correspond to "Iron X". The processing unit 12 also retrieves the trap image identifier "T01".

[0250] Next, the captured image transmission unit 131 obtains the destination information of the inspector with inspector identifier "I002" from the inspector information management table. Then, the captured image transmission unit 131 transmits the trap image with "ID=1" and the trap image identifier "T01", etc., to the destination indicated by the obtained destination information.

[0251] Next, inspector terminal 2, with inspector identifier "I002", receives and outputs the trap image with "ID=1" and the trap image identifier "T01", etc.

[0252] Next, let's assume that the inspector with inspector identifier "I002" entered the inspection result "No defects" into inspector terminal 2.

[0253] The inspector terminal 2 then receives the inspection result "No defects". Next, the terminal processing unit 23 acquires the response time, etc. Next, the terminal processing unit 23 constructs information including the received inspection result "No defects", response time, trap image identifier "T01", inspector identifier "I002", etc. Next, the terminal transmission unit 26 transmits the constructed information to the product inspection support device 1.

[0254] Next, the inspection result receiving unit 141 of the product inspection support device 1 receives the inspection results, etc., from the inspector terminal 2 with inspector identifier "I002".

[0255] Next, the trap image determination unit 126 determines from the received trap image identifier "T01" that the inspection result, etc. corresponds to the trap image.

[0256] Next, the trap image determination unit 126 determines the correct answer information corresponding to the trap image corresponding to the received inspection result, etc., "<Defect> detected <Location information>(x 11 ,y 11 )(x 12 ,y 12 The trap image management table (Figure 13) is obtained with the <defect type identifier> "contamination". Next, the trap image determination unit 126 uses the correct information and the received inspection result "no defects" to determine that the received inspection result "no defects" is incorrect.

[0257] Next, the image transmission unit 131 performs inappropriate user processing on the inspector who has transmitted the inspection results, etc. Specifically, the image transmission unit 131 sends a warning message, "You appear tired. Please take a 30-minute break," to the inspector terminal 2 with inspector identifier "I002." The image transmission unit 131 also refrains from transmitting images to the inspector terminal 2 with inspector identifier "I002" for the next 30 minutes. The warning message used for inappropriate user processing is stored in the storage unit 11.

[0258] Next, examiner terminal 2, with examiner identifier "I002", receives and outputs the message, "You seem tired. Please take a 30-minute break."

[0259] Through the above processing, trap images can be transmitted appropriately, and the transmission of captured images can be controlled appropriately according to the inspection results corresponding to the trap images.

[0260] As described above, according to this embodiment, images of the product to be inspected can be transmitted to the terminal of the appropriate inspector.

[0261] Furthermore, according to this embodiment, the examiner's condition can be understood by using the trap image.

[0262] Furthermore, according to this embodiment, the accuracy of the inspection can be increased by performing appropriate actions to identify inappropriate individuals as needed.

[0263] Furthermore, according to this embodiment, the labor of cloud workers can be effectively utilized for product inspection.

[0264] Furthermore, according to this embodiment, the inspector's inspection can be supported by performing machine learning prediction processing.

[0265] In this embodiment, the inspector determination unit 125 may always determine all inspectors. In this case, the product inspection support device is a product inspection support device comprising an image acquisition unit that acquires images of the product to be inspected, and an image transmission unit that transmits the images to the inspector terminals of one or more predetermined inspectors.

[0266] Furthermore, the processing in this embodiment may be implemented by software. This software may be distributed by software download or the like. Alternatively, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that implements the product inspection support device 1 in this embodiment is the following program. In other words, this program causes the computer to function as an image acquisition unit that acquires images of the product to be inspected, an inspector determination unit that determines one or more inspectors from among two or more inspectors to transmit the images, and an image transmission unit that transmits the images to the inspector terminals of each of the one or more inspectors determined by the inspector determination unit.

[0267] Figure 15 also shows the appearance of a computer that runs the program described herein to realize the product inspection support device 1 and inspector terminal 2 of the various embodiments described above. The embodiments described above can be realized with computer hardware and computer programs executed thereon. Figure 15 is an overview of this computer system 300, and Figure 16 is a block diagram of the system 300.

[0268] In Figure 15, the computer system 300 includes a computer 301 with a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0269] In Figure 16, the computer 301 includes, in addition to the CD-ROM drive 3012, an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, a ROM 3015 for storing programs such as boot-up programs, a RAM 3016 connected to the MPU 3013 for temporarily storing application program instructions and providing temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card for providing connectivity to a LAN.

[0270] The program that causes the computer system 300 to execute the functions of the product inspection support device 1, etc., as described above, may be stored on the CD-ROM 3101, inserted into the CD-ROM drive 3012, and then transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored on the hard disk 3017. The program is loaded into the RAM 3016 when executed. The program may also be loaded directly from the CD-ROM 3101 or the network.

[0271] The program does not necessarily have to include an operating system (OS) or third-party program that causes the computer 301 to execute functions such as the product inspection support device 1 of the above-described embodiment. The program only needs to include the instruction portion that calls the appropriate function (module) in a controlled manner and obtains the desired result. How the computer system 300 operates is well known, so a detailed explanation is omitted.

[0272] In the above program, steps such as sending information and receiving information do not include hardware-based processing, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0273] Also, the computer that executes the above program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

[0274] Also, in each of the above embodiments, it is needless to say that two or more communication means existing in one device may be physically realized by one medium.

[0275] Also, in each of the above embodiments, each process may be realized by being centrally processed by a single device, or may be realized by being distributedly processed by a plurality of devices.

[0276] Needless to say, the present invention is not limited to the above embodiments, and various modifications are possible and are also included within the scope of the present invention.

Industrial Applicability

[0277] As described above, the product inspection support device 1 according to the present invention has the effect of being able to transmit a photographed image of a product to be inspected to an appropriate inspector's terminal, and is useful as a server or the like for supporting product inspection.

Explanation of Signs

[0278] 1 Product inspection support device 2 Inspector terminal 11 Storage unit 12 Processing unit 13 Transmission unit 14 Reception unit 15 Output unit 21 Terminal storage unit 22 Terminal reception unit 23 Terminal processing unit 24 Terminal output unit 25 Terminal reception section 26 Terminal transmission unit 111 Teacher data storage unit 112 Learning model storage unit 113 Product information storage unit 114 Inspector Information Storage Unit 115 Transmission Condition Storage Unit 116 Trap Image Storage Unit 121 Learning Department 122 Image acquisition unit 123 Prediction Section 124 Inspector attribute value acquisition unit 125 Inspector Selection Department 126 Trap Image Judgment Unit 127 Answer Judgment Department 131 Image transmission unit 141 Inspection Result Receiving Unit 151 Inspection Result Output Unit

Claims

1. An inspector information storage unit that stores two or more inspector information items having one or more inspector attribute values, An image acquisition unit that acquires images of the product to be inspected, An inspector determination unit that determines one or more inspectors from among two or more inspectors to transmit the captured images, A captured image transmission unit transmits the captured images to the examiner terminal of one or more examiners determined by the examiner determination unit, The system includes a trap image determination unit that determines whether the one or more dynamic attribute values ​​of each of the two or more examiners match the trap image transmission conditions, which are the conditions for transmitting a trap image. The aforementioned image transmission unit is A product inspection support device that transmits the trap image to the inspector terminal of one or more inspectors that the trap image determination unit determines to meet the trap image transmission conditions.

2. An inspection result receiving unit receives the inspection result, which is the response to the transmission of the aforementioned trap image, from the inspector's terminal. A response determination unit that determines whether the aforementioned test result is correct or not, The aforementioned image transmission unit is The product inspection support device according to claim 1, wherein if the response determination unit determines that the inspection result corresponding to the transmission of the trap image is incorrect, it performs processing related to the control of image transmission to the inspector, and processing for inappropriate persons.

3. An image acquisition unit that acquires an image of the product to be inspected, An inspector determination unit that determines one or more inspectors from among two or more inspectors to transmit the captured images, A captured image transmission unit transmits the captured images to the examiner terminal of one or more examiners determined by the examiner determination unit, An inspector information storage unit that stores two or more inspector information items, each having one or more inspector attribute values, The system comprises a product information storage unit which stores product information having one or more product attribute values ​​that are attribute values ​​of a product or a photographed image, The aforementioned inspector determination unit, A product inspection support device that determines the one or more inspectors who transmit the captured images, using one or more inspector attribute values ​​possessed by each of the two or more inspector information and one or more product attribute values ​​possessed by the product information.

4. An inspection result receiving unit that receives inspection results which are responses corresponding to the transmission of the aforementioned captured images and which are responses transmitted by the inspector terminals of each of the two or more inspectors, The system further comprises an inspector attribute value acquisition unit that acquires one or more inspector attribute values ​​for each of the two or more inspectors using the inspection results received by the inspection result receiving unit, The product inspection support device according to claim 3, wherein the one or more inspector attribute values ​​stored in the inspector information storage unit are one or more inspector attribute values ​​acquired by the inspector attribute value acquisition unit.

5. A learning model storage unit stores the learned model obtained by performing machine learning training using two or more training data sets, which consist of captured images and inspection results. The system further comprises a prediction unit that uses the captured image acquired by the image acquisition unit and the learning model to perform machine learning prediction processing and acquire the prediction result, The aforementioned image transmission unit is The product inspection support device according to claim 1, which transmits the captured image and the prediction result to the inspector terminal of each of the one or more inspectors determined by the inspector determination unit.

6. An inspection result receiving unit receives an inspection result from the inspector's terminal, which is a response to the transmission of the aforementioned captured image and includes an image identifier that identifies the captured image and location information that identifies the location of the error. The product inspection support device according to claim 1, further comprising: an inspection result output unit that outputs the inspection results received by the inspection result receiving unit.

7. A training data storage unit that stores two or more training data sets having captured images and location information, A learning unit that uses the two or more training data sets mentioned above to perform machine learning training and obtain a learning model, The system further comprises a prediction unit that uses the captured image acquired by the image acquisition unit and the learning model to perform machine learning prediction processing and acquire the prediction result, The aforementioned image transmission unit is The product inspection support device according to claim 6, which transmits the captured image and the prediction result to the inspector terminal of each of the one or more inspectors determined by the inspector determination unit.

8. A product inspection support method for causing a computer to perform all the processing performed by the product inspection support device described in any one of Claims 1 to 7.

9. Computers, A program for causing a product inspection support device to function as described in any one of claims 1 to 7.

Images