Inspection equipment, inspection method, and program

The inspection apparatus enhances image recognition accuracy by generating variation images and further training the model, overcoming the challenge of limited data availability for abnormal states.

JP2026181649APending Publication Date: 2026-11-19RICOH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2025077105
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-07
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Conventional image recognition AI models for visual inspection face challenges in improving accuracy due to the difficulty in collecting a large volume of image data, particularly for abnormal states, which have low occurrence frequencies.

Method used

An inspection apparatus that utilizes an image generation model to generate multiple variation images based on a small set of sample images, followed by additional training of an image recognition model to enhance its accuracy without requiring a large dataset.

Benefits of technology

Improves the accuracy of visual inspection using image recognition without the need for a large amount of training data, specifically addressing the scarcity of abnormal state data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2026181649000001_ABST
    Figure 2026181649000001_ABST
Patent Text Reader

Abstract

To provide an inspection device that improves the accuracy of visual inspection using image recognition without having to collect a large amount of image data for training. [Solution] An inspection device 5 for inspecting the state of an object based on an image of the object, comprising: a reception unit that receives information of the object to be inspected, which is the state of the object, and a sample image of the object; a first additional learning unit that causes an image generation model to be further trained so that an image in which the state of the object indicated by the information of the object to be inspected is visually represented based on the information of the object to be inspected and the sample image; an image generation unit that uses the additionally trained image generation model to generate a plurality of variation images of the object in which the state of the object indicated by the information of the object to be inspected is visually represented for the object; a second additional learning unit that causes an image recognition model to be further trained based on the variation images and the information of the object to be inspected; and an image recognition unit that uses the additionally trained image recognition model to recognize an image of the object to be inspected.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an inspection apparatus, an inspection method, and a program.

Background Art

[0002] In recent years, at sites for equipment inspection such as in factories, visual inspection (inspection for the presence or absence of equipment anomalies, etc.) by image recognition AI using an AI (Artificial Intelligence) model has been performed. The AI model used for this visual inspection is constructed by subjecting a diverse set of image data (image set) including image data indicating the normal state of the inspection target and image data indicating the abnormal state thereof to machine learning. In general, in order to improve the accuracy of visual inspection by such image recognition AI, it is necessary to make the AI model learn using a large volume of image data.

[0003] Patent Document 1 discloses an AI platform system that realizes an environment in which users without specialized knowledge can create and test AI models that meet their respective needs.

Summary of Invention

Problem to be Solved by the Invention

[0004] However, according to the conventional technology, when it is difficult to collect a large volume of image data used for learning of the AI model, the AI model cannot be made to learn sufficiently, and there is a problem that the accuracy of visual inspection by image recognition cannot be improved. In particular, image data indicating the abnormal state of the inspection target has a low occurrence frequency, and it has been difficult to secure a sufficient amount of image data required for learning.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an inspection apparatus capable of improving the accuracy of visual inspection by image recognition without collecting a large volume of image data used for learning.

Means for Solving the Problem

[0006] To solve the above-mentioned problems and achieve the objective, the present invention provides an inspection device for inspecting the state of an object based on an image of the object, comprising: a reception unit that receives information of an object to be inspected, which is the state of the object, and a sample image of the object; a first additional learning unit that causes an image generation model to be further trained based on the information of the object to be inspected and the sample image received by the reception unit, so that an image in which the state of the object as indicated by the information of the object to be inspected is visually represented; an image generation unit that generates a plurality of variation images of the object in which the state of the object as indicated by the information of the object to be inspected is visually represented for the object, using the additionally trained image generation model further trained by the first additional learning unit; a second additional learning unit that causes an image recognition model to be further trained based on the variation images generated by the image generation unit and the information of the object to be inspected; and an image recognition unit that performs image recognition of the object relating to the object to be inspected using the additionally trained image recognition model further trained by the second additional learning unit. [Effects of the Invention]

[0007] According to the present invention, the accuracy of visual inspection using image recognition can be improved without having to collect a large amount of image data for training. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of the configuration of the inspection system according to the first embodiment. [Figure 2] Figure 2 shows an example of the hardware configuration of the inspection device according to the first embodiment. [Figure 3] Figure 3 shows an example of the functional configuration of the inspection device according to the first embodiment. [Figure 4] Figure 4 is a diagram showing the processing flow of the inspection apparatus according to the first embodiment. [Figure 5] Figure 5 shows the processing flow of the first additional learning unit. [Figure 6]Figure 6 shows the processing flow of the image generation unit and the second additional learning unit. [Figure 7] Figure 7 is a flowchart showing an example of additional training processing for the image generation model according to the first embodiment. [Figure 8] Figure 8 is a flowchart showing an example of the additional training process for the image recognition model according to the first embodiment. [Figure 9] Figure 9 shows an example of the functional configuration of the inspection device according to the second embodiment. [Figure 10] Figure 10 is a flowchart showing an example of the additional training process for the image recognition model according to the second embodiment. [Figure 11] Figure 11 shows an example of the functional configuration of the inspection device according to the third embodiment. [Figure 12] Figure 12 illustrates an example of additional training of an image recognition model according to the third embodiment. [Figure 13] Figure 13 is a flowchart showing an example of the additional training process for an image recognition model according to the third embodiment. [Modes for carrying out the invention]

[0009] The embodiments of the inspection apparatus, inspection method, and program will be described in detail below with reference to the attached drawings.

[0010] (First Embodiment) Figure 1 shows an example of the configuration of the inspection system 1 according to the first embodiment. As shown in Figure 1, the inspection system 1 according to this embodiment includes an electronic whiteboard 2, an inkjet printer 3, a smartphone 4, an inspection device 5, a 360-degree imaging device 6, a video conferencing terminal 7, a projector 8, and an MFP (Multi-Function Peripheral) 9, and each device is connected to communicate with each other via a communication network 100. The inspection device 5 is an information processing device such as a PC (Personal Computer) or a server.

[0011] Figure 2 shows an example of the hardware configuration of the inspection device 5 according to the first embodiment. Here, the hardware configuration will be described assuming that the inspection device 5 is a server.

[0012] As shown in Figure 2, the inspection device 5 is built by a computer and includes a CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, HDD (Hard Disk Drive) 504, HDD (Hard Disk Drive) controller 505, display 506, external device connection I / F (Interface) 508, network I / F 509, bus line (data bus) 510, keyboard 511, pointing device 512, DVD-RW (Digital Versatile Disk Rewritable) drive 514, and media I / F 516.

[0013] Of these, the CPU 501 controls the operation of the entire inspection device 5. The ROM 502 stores programs used to drive the CPU 501, such as the IPL (Initial Program Loader). The RAM 503 is used as the work area for the CPU 501. The HDD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data to the HDD 504 according to the control of the CPU 501. The display 506 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. The network I / F 509 is an interface for data communication using the communication network 100. The bus line 510 is an address bus, data bus, etc., for electrically connecting each component such as the CPU 501 shown in Figure 1.

[0014] Further, the keyboard 511 is a type of input unit provided with a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 512 is a type of input unit that performs selection and execution of various instructions, selection of a processing target, movement of a cursor, and the like. The DVD-RW drive 514 controls reading or writing of various data with respect to a DVD-RW 513 as an example of a detachable recording medium. Note that the recording medium is not limited to a DVD-RW, and may be a DVD-R or the like. The media I / F 516 controls reading or writing (storage) of data with respect to a recording medium 515 such as a flash memory.

[0015] Figure 3 is a diagram illustrating an example of the functional configuration of the inspection apparatus 5 according to the first embodiment. The inspection apparatus 5 inspects the state of a target object such as a facility on the basis of an image obtained by capturing the target object. Here, among the functions exhibited by the inspection apparatus 5, the characteristic functions in the present embodiment will be described.

[0016] As shown in Figure 3, the inspection apparatus 5 includes a reception unit 50, a first additional learning unit 51, an image generation unit 52, a second additional learning unit 53, and an image recognition unit 54. In the inspection apparatus 5 according to the present embodiment, the functions of the reception unit 50, the first additional learning unit 51, the image generation unit 52, the second additional learning unit 53, and the image recognition unit 54 are implemented by the CPU 501 executing a program stored in the ROM 502 using the RAM 503 or the like as a work area. Further, the inspection apparatus 5 includes an image generation model 60 and an image recognition model 61 constructed by the RAM 503 and the HDD 504.

[0017] The reception unit 50 receives user information, inspection target information, and sample images. The user information is, for example, a customer ID (Identification) indicating a customer (user) who requests a visual inspection (inspection for the presence or absence of equipment abnormality, etc.). The user information may be any information that can be identified by the inspection device 5, such as numerical values, character strings such as company names, and the like. The inspection target is, for example, the state of equipment or the like that is an object of image recognition. The inspection target information is an inspection target item indicating the state of the object, and is represented by an inspection target ID. The state of the object is a state where the object has an abnormality (rust, crack, etc.) or a normal state with no abnormality. The inspection target information may be any information that can be identified by the inspection device 5, such as numerical values, character strings such as state names, and the like.

[0018] The sample image is, for example, at least one image obtained by photographing an object such as equipment. For example, when a user requests an inspection for the presence or absence of an abnormality (rust, crack, etc.) in equipment, the sample image is an image of the equipment in which the abnormality has occurred. Note that the sample image may be an image of equipment in which no abnormality has occurred (normal). Further, the number of sample images may be one or a plurality.

[0019] The first additional learning unit 51 causes the image generation model 60 to perform additional learning based on the user information, the inspection target information, and the sample images received by the reception unit 50. More specifically, the first additional learning unit 51 causes the image generation model 60 to perform additional learning so that an image in which the state of the object indicated by the inspection target information is visually represented can be generated.

[0020] The image generation unit 52 uses the additionally trained image generation model 60, which has been further trained by the first additional training unit 51, to generate an image set containing multiple images that visually represent the state of the object indicated by the information of the object to be inspected. These multiple images are used to further train the image recognition model 61. The multiple images included in the generated image set are images (variation images) that are similar to the sample image. For example, if the sample image is an image of rust of a certain shape, the images included in the image set are images of rust of various shapes similar to that shape. The image generation unit 52 generates as many such images as are possible for further training by the image generation unit 52.

[0021] The second additional learning unit 53 causes the image recognition model 61 to undergo additional training based on the image set generated by the image generation unit 52, user information, and information about the object to be inspected.

[0022] The image recognition unit 54 uses the additionally trained image recognition model 61, which has been further trained by the second additional training unit 53, to recognize images related to the user's inspection target.

[0023] Image generation model 60 is an AI (Artificial Intelligence) model (pre-trained image generation model) that generates images of desired inspection targets by pre-training it with images of abnormal and normal equipment from various users.

[0024] Image recognition model 61 is an AI model (pre-trained image recognition model) that recognizes images of desired inspection targets by pre-training it with images of abnormal and normal equipment from various users.

[0025] The learning (machine learning) used to generate AI models (trained models) is a technique for enabling computers to acquire human-like learning abilities. It refers to a technique in which a computer autonomously generates algorithms necessary for judgments such as data identification from pre-incorporated training data, and applies these algorithms to new data to make predictions. The learning method for machine learning can be supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, or deep learning, or a combination of these learning methods; the learning method for machine learning is not restricted.

[0026] Figure 4 shows the processing flow of the inspection device according to the first embodiment. First, the user inputs the customer ID, the inspection target ID, and a sample image into the inspection device (step S1). Here, the image generation model 60 and the image recognition model 61 have not yet been trained on the user's inspection target. In other words, the user's inspection target is a new inspection target for which the user will be trained.

[0027] Next, the first additional learning unit 51 refers to the additional learning dataset for the image generation model 60 (input customer ID, inspection target ID, and sample image) and performs additional learning (step S2). The first additional learning unit 51 uses the pre-trained image generation model 60a as the initial AI model and performs additional learning on it. The AI ​​model after the additional learning is completed becomes the additionally trained image generation model 60b.

[0028] Furthermore, the first additional learning unit 51 may discard the sample images after the completion of additional learning. Discarding the sample images reduces the amount of data stored on the HDD 404, etc. Also, if the sample images contain confidential information such as the user's equipment, discarding the sample images can prevent information leakage.

[0029] Furthermore, the first additional learning unit 51 may copy the additionally trained image generation model 60b, which has completed additional training, to the pre-trained image generation model 60a. This allows the additionally trained image generation model 60b to be used as the initial AI model for further training during the next additional training session.

[0030] Next, the image generation unit 52 generates a dataset for further training of the image recognition model 61 using the additionally trained image generation model 60b (step S3).

[0031] Next, the second additional learning unit 53 refers to the additional training dataset for the image recognition model 61 and performs additional training (step S4). The second additional learning unit 53 uses the pre-trained image recognition model 61a as the initial AI model and performs additional training on it. The AI ​​model after the additional training is completed becomes the additionally trained image recognition model 61b.

[0032] Finally, the image recognition unit 54 refers to an image of the user's equipment (an image of the object) (step S5) and notifies the user of the recognition result (inspection result) indicating whether or not there is an abnormality in the equipment (step S6). In this embodiment, with just one sample image, the image recognition model 61 can be trained for a new inspection target of the user, and accurate image recognition (visual inspection) can be performed.

[0033] The second additional learning unit 53 may also copy the newly trained image recognition model 61b, which has completed additional training, to the pre-trained image recognition model 61a. This allows the newly trained image recognition model 61b to be used as the initial AI model for further training during the next additional training session.

[0034] Figure 5 shows the processing flow of the first additional learning unit 51. The processing shown in Figure 5 corresponds to the processing in step S2 of Figure 4.

[0035] First, the first additional learning unit 51 converts the input customer ID, inspection target ID, and sample image into strings (steps S21-S23). In the example in Figure 5, the customer ID number 5278 is converted into the string "C-5278", and the inspection target ID number 0031 is converted into the string "T-0031". Also, the sample image is converted into the string "This image shows a close-up photo of the equipment" by the image depiction AI. The conversion from ID to string is performed by adding symbols and codes based on predetermined rules. The image depiction AI is an AI that implements a trained model that has been trained to depict the content of images using various images.

[0036] Next, the first additional learning unit 51 generates an annotation string in which the converted strings are merged (step S24). In the example in Figure 5, the annotation string is the string "This image shows a close-up photo of the equipment with <t-0031>at the site of <c-5278>" is being generated.

[0037] Next, the first additional learning unit 51 converts the annotation string into a set of numbers and generates an embedding vector (step S25). The embedding vector is a vector that represents (embeds) words, symbols, sentences, etc., as points (vectors) in a multidimensional space by quantifying them. In the example in Figure 5, constituent units such as words are extracted from the annotation string by the tokenizer, and each constituent unit is converted into a vector. In this way, for example, an embedding vector (0.30, 0.73, -0.64, 0.64) corresponding to the customer ID is generated.

[0038] Then, the first additional learning unit 51 causes the pre-trained image generation model 60a to undergo additional training based on the generated embedding vectors and sample images (S26). In the example in Figure 5, additional training is performed by repeatedly comparing the image generated by the pre-trained image generation model 60a using the embedding vectors with the sample image and updating the parameters of the AI ​​model to minimize the output of the cost function, etc. Note that the embedding vectors corresponding to the customer ID and the inspection target ID are used for optimization by the cost function, etc., for the output of the cost function, but the other embedding vectors are predetermined vectors and are not used for such optimization.

[0039] Figure 6 shows the processing flow of the image generation unit 52 and the second additional learning unit 53. The processing shown in Figure 6 corresponds to the processing in steps S3 and S4 of Figure 4.

[0040] First, the image generation unit 52 converts the input customer ID and inspection target ID into strings (steps S31, S32), and generates an annotation string by integrating the converted strings with an arbitrary string (step S33). Here, the arbitrary string can be any string that the AI ​​implemented in the image generation unit 52 can use to generate an image. The arbitrary string can also be a string entered by the user, a string pre-stored in the HDD 404, etc. (for example, the annotation string generated in step S24 of Figure 5).

[0041] Next, the image generation unit 52 converts the generated annotation string into a set of numbers to generate an embedding vector (step S34), and generates an image using the embedding vector and the additionally trained image generation model 60b (step S35).

[0042] Then, the second additional learning unit 53 refers to the image set including the generated images and the ID of the object to be inspected (steps S41, S42), and causes the pre-trained image recognition model 61a to undergo additional training (step S43). In the example in Figure 6, additional learning is performed by repeatedly comparing the recognition result of recognizing an image in the image set with the ID of the object to be inspected, and updating the parameters of the AI ​​model to minimize the output of the cost function, etc.

[0043] Figure 7 is a flowchart showing an example of additional learning processing for the image generation model 60 according to the first embodiment. First, the reception unit 50 receives the customer ID, the inspection target ID, and a sample image (step S100), and the first additional learning unit 51 converts each of the received data into a string (step S101).

[0044] Next, the first additional learning unit 51 generates an annotation string from the converted string (step S102) and generates an embedding vector (step S103).

[0045] The first additional learning unit 51 generates an image based on the embedded vector (step S104), compares the generated image with the sample image, and if it determines that learning is complete (step S105: Yes), copies the additionally trained image generation model to the pre-trained image generation model (step S107).

[0046] On the other hand, if it is not determined that learning is complete (step S105: No), the first additional learning unit 51 updates the parameters of the pre-trained image generation model (AI model) (step S106), and the process returns to step S104. The determination of whether learning is complete or not is made, for example, by whether the output of the cost function is below a predetermined threshold. Here, the predetermined threshold is a value determined in advance through experiments or the like.

[0047] Figure 8 is a flowchart showing an example of additional training processing for the image recognition model 61 according to the first embodiment. First, the reception unit 50 receives a customer ID, an inspection target ID, and an arbitrary string (step S150), and the image generation unit 52 converts each of the received IDs into a string (step S151).

[0048] Next, the image generation unit 52 generates an annotation string from the converted string (step S152), generates an embedding vector (step S153), and generates an image set based on the embedding vector (step S154).

[0049] Next, the second additional learning unit 53 performs the processing in steps S155 to S157 for each image in the generated image set, causing the pre-trained image recognition model 61a to undergo additional training. Specifically, the second additional learning unit 53 outputs the recognition result of the generated image (step S155), compares the recognition result with the ID of the object to be inspected, and if it determines that training is complete (step S156: Yes), copies the additionally trained image recognition model to the pre-trained image recognition model (step S158).

[0050] On the other hand, if it is not determined that learning is complete (step S156: No), the second additional learning unit 53 updates the parameters of the pre-trained image recognition model (AI model) (step S157), and the process returns to step S155. The determination of whether learning is complete or not is made, for example, by whether the output of the cost function is below a predetermined threshold. Here, the predetermined threshold is a value determined in advance through experiments or the like.

[0051] Thus, according to this embodiment, the accuracy of visual inspection using image recognition can be improved without collecting a large amount of image data for training.

[0052] (Second Embodiment) The second embodiment allows the user to select whether to use images generated by the image generation unit 52 or sample images as images for further training of the image recognition model 61. In the following description of the second embodiment, the parts that overlap with the first embodiment will be omitted, and the parts that differ from the first embodiment will be described.

[0053] Figure 9 shows an example of the functional configuration of the inspection device 5 according to the second embodiment. The difference from Figure 3 is that the inspection device 5 further includes a learning data selection unit 55. Here, we will describe the characteristic functions of the inspection device 5 in this embodiment.

[0054] The training data selection unit 55 selects whether to use images generated by the image generation unit 52 or sample images as images to be used for additional training of the image recognition model 61. The training data selection unit 55 makes the above selection based, for example, on the selection result entered by the user from a selection screen (not shown) using the keyboard 511 or pointing device 512.

[0055] This allows, for example, if a user can prepare a sufficient number of sample images for training the image recognition model 61, the image recognition model 61 to be further trained based on the sample images.

[0056] Figure 10 is a flowchart showing an example of additional learning processing for the image recognition model 61 according to the second embodiment. The difference from Figure 8 is that a step S200 is added in which the learning data selection unit 55 performs selection, and in step S206, the second additional learning unit 53 can output recognition results based on the sample image. The processing in steps S201 to S205 and S207 to S209 in Figure 10 is the same as the processing in steps S150 to S154 and S156 to S158 in Figure 8, so the explanation is omitted.

[0057] If it is selected to train using sample images (step S200: Yes), the training data selection unit 55 selects the sample images as images to be used for additional training of the image recognition model 61, and the process proceeds to step S206. On the other hand, if it is not selected to train using sample images (step S200: No), the process proceeds to step S201, and the training data selection unit 55 selects the images generated by the image generation unit 52 in step S205 as images to be used for additional training of the image recognition model 61.

[0058] The second additional learning unit 53 outputs the recognition result of the image selected by the learning data selection unit 55 (step S206).

[0059] Thus, according to this embodiment, the accuracy of visual inspection by image recognition can be improved without collecting a large amount of image data for training. Furthermore, it is possible to select whether to use images generated by the image generation unit 52 or sample images as images to be used for additional training of the image recognition model 61.

[0060] (Third embodiment) The third embodiment involves having the image recognition model 61 perform additional training on a user for an inspection target that is different from the inspection target received from the user, but has already been further trained on by another user. In the following description of the third embodiment, the parts that overlap with the first and second embodiments will be omitted, and the parts that differ from the first and second embodiments will be described.

[0061] Figure 11 shows an example of the functional configuration of the inspection device 5 according to the third embodiment. The difference from Figure 3 is that the inspection device 5 further includes an inspection target selection unit 56. Here, we will describe the characteristic functions of the inspection device 5 in this embodiment.

[0062] The inspection target selection unit 56 selects whether or not to allow the user to perform additional training on an inspection target that has already been trained by another user, which is different from the inspection target received by the reception unit 50 from the user. The inspection target selection unit 56 makes the above selection based, for example, on the selection result entered by the user from a selection screen (not shown) using the keyboard 511 or pointing device 512.

[0063] Figure 12 illustrates an example of additional training of the image recognition model 61 according to the third embodiment. Hereafter, inspection targets that have been further trained by other users, different from the inspection target that was initially accepted, may simply be referred to as "other inspection targets," etc. In Figure 12(a), the image generation model 60 and the image recognition model 61 are assumed to have been further trained based on sample images input by each customer for inspection targets α and γ of customer A and inspection target β of customer B. The inspection device 5 is assumed to have received inspection target α and sample images related to α from a new customer C.

[0064] If the inspection target selection unit 56 chooses not to further train other inspection targets, as shown in Figure 12(b), the image generation model 60 and the image recognition model 61 will only perform additional training on inspection target α of customer C.

[0065] On the other hand, if the inspection target selection unit 56 chooses to further train other inspection targets, as shown in Figure 12(c), the image recognition model 61 will be further trained not only on inspection target α of customer C, but also on inspection targets β and γ of customer C. Since the image generation model 60 has already been further trained on inspection targets β and γ of customers A and B, it can use the embedded vectors corresponding to inspection targets β and γ to generate an image set for further training of the image recognition model 61 on inspection targets β and γ of customer C.

[0066] Figure 13 is a flowchart showing an example of the additional training process for the image recognition model 61 according to the third embodiment. First, the reception unit 50 receives the customer ID, the inspection target ID, and a sample image (step S300), and additional training of the image recognition model is performed (step S201). In step S301, the processes of steps S101 to S107 in Figure 7 and steps S150 to S158 in Figure 8 are performed.

[0067] Next, if the inspection target selection unit 56 chooses to train with other inspection target IDs (step S302: Yes), additional training of the image recognition model is performed for other inspection target IDs that have already been trained by other users (step S303). In step S303, the processes of steps S150 to S158 in Figure 8 are performed, but other inspection target IDs are used as inspection target IDs. That is, the image generation unit 52 generates an image set to allow the image recognition model 61 to further train in image recognition recognition related to other inspection target IDs. The second additional training unit 53 then causes the image recognition model 61 to perform additional training based on the image set generated using other inspection target IDs.

[0068] On the other hand, if the inspection target selection unit 56 does not choose to train with other inspection target IDs (step S302: No), or if training is completed for all inspection target IDs (step S304: Yes), the process ends. Also, if training is not completed for all inspection target IDs that have been additionally trained by other users (step S304: No), the process returns to step S303, and additional training of the image recognition model is performed for the other inspection target IDs for which training is not completed (step S303).

[0069] In addition, the above description explains an example in which all test targets that have already been further trained by other users are trained. However, the test target selection unit 56 may also be configured to select only some of the test targets that have already been further trained by other users as test targets to be further trained.

[0070] Thus, according to this embodiment, the accuracy of visual inspection by image recognition can be improved without collecting a large amount of image data for training. Furthermore, the image recognition model 61 can be further trained for a user using inspection targets that are different from the inspection target received from the user, but have already been trained on by other users.

[0071] The programs executed by the inspection device 5 in each embodiment described above are provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0072] Furthermore, the program executed by the inspection device 5 of each embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the program executed by the inspection device 5 of each embodiment may be provided or distributed via a network such as the Internet.

[0073] Furthermore, the programs for each embodiment may be pre-installed and provided in a ROM or the like.

[0074] The program executed by the inspection device 5 in each embodiment has a modular configuration that includes the above-mentioned parts (reception unit 50, first additional learning unit 51, etc.), and in actual hardware, the CPU (processor) reads the program from the recording medium and executes it, thereby loading the above-mentioned parts onto the main memory and generating them.

[0075] Each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each function described above.

[0076] Furthermore, the apparatus described in each embodiment represents only one of several computing environments for carrying out each embodiment disclosed herein.

[0077] In one embodiment, the inspection apparatus 5 includes a plurality of computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network or shared memory, and perform the processing disclosed herein.

[0078] The inspection device 5 may be any device equipped with communication capabilities, such as a PJ (Projector), IWB (Interactive White Board), digital signage output device, HUD (Head Up Display) device, industrial machinery, imaging device, sound collection device, medical equipment, networked home appliance, automobile (Connected Car), notebook PC (Personal Computer), mobile phone, smartphone, tablet device, game console, PDA (Personal Digital Assistant), digital camera, wearable PC or desktop PC, etc.

[0079] Although various embodiments of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These novel embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. Furthermore, components from different embodiments and modifications may be combined as appropriate.

[0080] Examples of the present invention are as follows: <1> An inspection device for inspecting the condition of an object based on an image of the object, A reception unit that receives information about the object to be inspected, which is the state of the object, and a sample image of the object taken from the object, A first additional learning unit, based on the information of the object to be inspected and the sample image received by the reception unit, causes the image generation model to undergo additional training so that an image can be generated that visually represents the state of the object as indicated by the information of the object to be inspected; An image generation unit generates multiple variation images that visually represent the state of the object as indicated by the information of the object to be inspected, using an additionally trained image generation model further trained by the first additional learning unit; A second additional learning unit causes the image recognition model to undergo additional training based on the variation images generated by the image generation unit and the information of the object to be inspected, An image recognition unit that uses an additionally trained image recognition model further trained by the second additional learning unit to recognize an image of the object that is to be inspected, This inspection device is characterized by being equipped with [a specific feature]. <2> The first additional learning unit generates an embedded vector based on the user information, the information of the object to be inspected, and the sample image, and further trains the image generation model based on the embedded vector and the sample image. <1> This is the inspection device described in [reference]. <3> The first additional learning unit discards the sample image after further training the image generation model. <1> or <2> This is the inspection device described in [reference]. <4> The first additional learning unit performs additional training on the image generation model and then copies the additionally trained image generation model to the image generation model. <1> ~ <3> The inspection device is one of the devices described in any one of the following lists. <5> The second additional learning unit, after performing additional training on the image recognition model, copies the additionally trained image recognition model to the image recognition model. <1> ~ <4> The inspection device is one of the devices described in any one of the following lists. <6> The image recognition model is further provided with a training data selection unit that selects whether to use images generated by the image generation unit or sample images as images for additional training of the image recognition model. <1> ~ <5> The inspection device is one of the devices described in any one of the following lists. <7> The image generation unit generates an image set for the image recognition model to perform additional training on the user using an object that is different from the object being inspected and has been further trained by another user. <2> ~ <6> The inspection device is one of the devices described in any one of the following lists. <8> The image recognition model further includes an inspection target selection unit that selects whether or not to allow the user to further train an inspection target that is different from the inspection target and has been further trained by another user. <2> ~ <7> The inspection device is one of the devices described in any one of the following lists. <9> An inspection method performed by an inspection device that inspects the condition of an object based on an image of the object, A receiving process that receives information about the object to be inspected, which is the state of the object, and a sample image of the object taken, A first additional learning step involves further training an image generation model so that, based on the information of the object to be inspected and the sample image received in the aforementioned reception step, an image generation model can be further trained so that an image can be generated that visually represents the state of the object as indicated by the information of the object to be inspected. An image generation step that generates multiple variation images in which the state of the object as indicated by the information of the object to be inspected is visually represented for the object, using the additionally trained image generation model that has been further trained in the first additional training step, A second additional learning step involves further training the image recognition model based on the variation images generated in the image generation step and the information of the object to be inspected, An image recognition step in which an image relating to the object to be inspected is recognized using the additionally trained image recognition model that has been further trained in the second additional training step, This is an inspection method characterized by including [a specific element]. <10> In an inspection device that inspects the condition of an object based on an image of the object, Computers, A receiving means for receiving information about the object to be inspected, which is the state of the object, and a sample image of the object taken, A first additional learning means causes an image generation model to undergo additional training based on the information of the object to be inspected and the sample image received by the reception means, so that an image can be generated in which the state of the object as indicated by the information of the object to be inspected is visually represented. An image generation means that generates a plurality of variation images in which the state of the object as indicated by the information of the object to be inspected is visually represented for the object, using an additionally trained image generation model further trained by the first additional training means, A second additional learning means for causing an image recognition model to undergo additional training based on the variation images generated by the image generation means and the information of the object to be inspected, An image recognition means that performs image recognition of the object relating to the object to be inspected using the additionally trained image recognition model further trained by the second additional training means, This program is characterized by its ability to function as such. [Explanation of symbols]

[0081] 1. Inspection System 5. Inspection equipment 50 Reception Department 51. First Additional Learning Section 52 Image Generation Unit 53 Second Additional Learning Section 54 Image Recognition Unit 55. Training Data Selection Section 56. Selection section for the subject of inspection 60 Image Generation Models 100 Communication Networks [Prior art documents] [Patent Documents]

[0082] [Patent Document 1] Japanese Patent Publication No. 2022-191637

Claims

1. An inspection device for inspecting the condition of an object based on an image of the object, A reception unit that receives information about the object to be inspected, which is the state of the object, and a sample image of the object taken from the object, A first additional learning unit, based on the information of the object to be inspected and the sample image received by the reception unit, causes the image generation model to undergo additional training so that an image can be generated that visually represents the state of the object as indicated by the information of the object to be inspected. An image generation unit generates multiple variation images that visually represent the state of the object as indicated by the information of the object to be inspected, using an additionally trained image generation model further trained by the first additional learning unit, A second additional learning unit causes the image recognition model to undergo additional training based on the variation images generated by the image generation unit and the information of the object to be inspected, An image recognition unit that uses an additionally trained image recognition model further trained by the second additional learning unit to recognize an image of the object relating to the object being inspected, An inspection device characterized by being equipped with the following features.

2. The inspection apparatus according to claim 1, wherein the first additional learning unit generates an embedded vector based on user information, information on the object to be inspected, and the sample image, and causes the image generation model to undergo additional learning based on the embedded vector and the sample image.

3. The inspection apparatus according to claim 1, wherein the first additional learning unit discards the sample image after performing additional training on the image generation model.

4. The inspection apparatus according to claim 1, wherein the first additional learning unit performs additional learning on the image generation model and then copies the additionally learned image generation model to the image generation model.

5. The inspection apparatus according to claim 1, wherein the second additional learning unit performs additional learning on the image recognition model and then copies the additionally learned image recognition model to the image recognition model.

6. The inspection apparatus according to any one of claims 1 to 5, further comprising a training data selection unit that selects whether to use images generated by the image generation unit or sample images as images to be used for additional training of the image recognition model.

7. The inspection apparatus according to any one of claims 2 to 5, wherein the image generation unit generates a set of images for the image recognition model to perform additional training on the user with an inspection target different from the inspection target, which has been further trained by another user.

8. The inspection apparatus according to any one of claims 1 to 5, further comprising an inspection target selection unit that selects whether or not to allow the image recognition model to be further trained for the user on inspection targets that are different from the inspection target and have been further trained by other users.

9. An inspection method performed by an inspection device that inspects the condition of an object based on an image of the object, A receiving process that receives information about the object to be inspected, which is the state of the object, and a sample image of the object taken, A first additional learning step involves further training an image generation model so that, based on the information of the object to be inspected and the sample image received in the aforementioned reception step, an image generation model can be further trained so that an image can be generated that visually represents the state of the object as indicated by the information of the object to be inspected. An image generation step that generates multiple variation images in which the state of the object as indicated by the information of the object to be inspected is visually represented for the object, using the additionally trained image generation model that has been further trained in the first additional training step, A second additional learning step involves causing the image recognition model to undergo additional training based on the variation images generated in the image generation step and the information of the object to be inspected. An image recognition step in which an image relating to the object to be inspected is recognized using the additionally trained image recognition model that has been further trained in the second additional training step, A testing method characterized by including [a certain component].

10. In an inspection device that inspects the condition of an object based on an image of the object, Computers, A receiving means for receiving information about the object to be inspected, which is the state of the object, and a sample image of the object taken, A first additional learning means causes an image generation model to undergo additional training based on the information of the object to be inspected and the sample image received by the reception means, so that an image can be generated in which the state of the object as indicated by the information of the object to be inspected is visually represented. An image generation means that generates a plurality of variation images in which the state of the object as indicated by the information of the object to be inspected is visually represented for the object, using an additionally trained image generation model further trained by the first additional training means, A second additional learning means for causing an image recognition model to undergo additional training based on the variation images generated by the image generation means and the information of the object to be inspected, An image recognition means that performs image recognition of the object relating to the object to be inspected using the additionally trained image recognition model further trained by the second additional training means, A program characterized by being designed to function as such.

Citation Information

Patent Citations

  • Ai platform system

    JP2022191637A