Generation program, generation method, and generation device

By generating training images with controlled blur parameters, the method addresses the challenge of processing blurred images, enhancing the performance of machine learning models in defect detection tasks.

JP2024157684AInactive Publication Date: 2024-11-08BROTHER KOGYO KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023072179
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Machine learning models struggle to process images with blur effectively due to factors like relative movement between imaging devices and target objects, leading to suboptimal performance.

Method used

A method for generating training images that incorporate blur processing, using parameters such as transport speed, exposure time, and resolution to create training data suitable for machine learning models that handle images with motion blur.

Benefits of technology

The generated training images enable machine learning models to accurately process and correct for blur, improving defect inspection accuracy by simulating real-world imaging conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157684000001_ABST
    Figure 2024157684000001_ABST
Patent Text Reader

Abstract

To generate a training image suitable for training a machine learning model to process an image with blur.SOLUTION: A generation program for training data of a machine learning model acquires still image data of an object. The generation program accepts the input of the input parameter set including one or more parameters selected from a conveying speed during conveying the object, an exposure time of an imaging apparatus for photographing the object being conveyed, and the resolution of the imaging apparatus. Training image data is generated by using the still image data to execute generation processing. The generation processing includes a blur processing which is the additional processing of the blur using the input parameter set.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present specification relates to techniques for generating training image data. [Background technology]

[0002] Various machine learning models can be used to inspect the appearance of objects such as label sheets. In the technique disclosed in Patent Document 1, an autoencoder is trained to generate image data representing a label sheet without defects from image data representing a label sheet with defects. An image representing a target label sheet is input to the trained autoencoder. A difference image between the image input to the autoencoder and the image generated by the autoencoder is used to calculate the degree of defect of the target label sheet. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-174516 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, a captured image of an object may have blur due to various causes. For example, blur may occur in a captured image due to relative movement between an imaging device such as a digital camera and the object. However, there are cases where a machine learning model cannot properly process an image having blur.

[0005] This specification discloses techniques for generating training images suitable for training machine learning models to process images with blur. [Means for solving the problem]

[0006] The technology disclosed in this specification can be realized in the following application examples.

[0007] [Application Example 1] A generation program for generating training data for a machine learning model, the generation program causing a computer to realize the following generation functions: an acquisition function for acquiring still image data of an object; a reception function for accepting input of an input parameter set including one or more parameters selected from a transport speed of the object during transport, an exposure time of an imaging device for photographing the object while being transported, and a resolution of the imaging device; and a generation function for generating training image data by executing a generation process using the still image data, the generation process including a blur process that is an additional blur process using the input parameter set.

[0008] According to this configuration, the generation process includes a blur process, which is an additional blur process using an input parameter set including one or more parameters selected from the transport speed, exposure time, and resolution, so that training image data suitable for training a machine learning model that processes images having blur associated with one or more parameters selected from the transport speed, exposure time, and resolution can be generated.

[0009] The technology disclosed in this specification can be realized in various forms, for example, in the form of a method and device for generating training image data, a computer program for realizing the functions of the method or device, a recording medium (e.g., a non-temporary recording medium) on which the computer program is recorded, etc. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram illustrating a data processing device according to an embodiment; [Diagram 2] 1 is a perspective view of a digital camera 110, a multifunction device 900, and a belt 190. FIG. [Diagram 3] 1A is a schematic diagram showing an example of a label sheet, FIG. 1B is a schematic diagram showing an example of a reference image I10, and FIGS. 1C and 1D are diagrams showing example parameters. [Figure 4]FIG. 3 is a block diagram illustrating an example of an image generation model 300. [Diagram 5] 13 is a flowchart illustrating an example of a generation process of training image data. [Figure 6] 13A and 13B are diagrams illustrating examples of blur filters. [Figure 7] 13A to 13D are diagrams showing examples of intermediate image data. [Figure 8] 13A-13D are diagrams showing examples of training image data. [Figure 9] 13 is a flowchart illustrating an example of a training process. [Figure 10] 13A and 13B are a part of a flowchart of another embodiment of the training image data generation process. [Figure 11] FIG. 13 is a diagram showing an example of second type training image data. [Figure 12] 13 is a part of a flowchart of another embodiment of a training image data generation process. [Figure 13] 13A and 13B are diagrams showing examples of training image data. [Figure 14] 13A and 13B are diagrams illustrating another embodiment of the process for acquiring the transport direction Dt. [Figure 15] 13A and 13B are diagrams showing another example of the object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A. First Example: A1. Equipment configuration: 1 is an explanatory diagram showing a data processing device according to an embodiment. The data processing device 200 is, for example, a personal computer. The data processing device 200 performs various data processing for inspecting the appearance of an object (for example, a label sheet attached to a product such as a multifunction device). In the following, it is assumed that the appearance of a label sheet 800 attached to a multifunction device 900 is inspected.

[0012] The data processing device 200 includes a processor 210, a storage device 215, a display unit 240, an operation unit 250, and a communication interface 270. These elements are connected to each other via a bus. The storage device 215 includes a volatile storage device 220 and a non-volatile storage device 230.

[0013] The processor 210 is a device configured to perform data processing, and is, for example, a central processing unit (CPU) or a system on a chip (SoC). The volatile storage device 220 is, for example, a DRAM, and the non-volatile storage device 230 is, for example, a flash memory. The non-volatile storage device 230 stores data of a generation program 231, a training program 232, an inspection program 233, reference image data I10d, and an image generation model 300. In this embodiment, the image generation model 300 is a program module that forms a machine learning model. Details of the programs 231-233, the reference image data I10d, and the image generation model 300 will be described later.

[0014] The display unit 240 is a device configured to display images, such as a liquid crystal display or an organic EL display. The operation unit 250 is a device configured to receive operations by a user, such as a button, a lever, or a touch panel arranged overlaid on the display unit 240. The display unit 240 and the operation unit 250 may form a so-called touch screen. The user can input various requests and instructions to the data processing device 200 by operating the operation unit 250. The communication interface 270 is an interface for communicating with other devices (for example, a USB interface, a wired LAN interface, or an IEEE802.11 wireless interface). The digital camera 110 is connected to the communication interface 270. The digital camera 110 is used to photograph the label sheet 800.

[0015] FIG. 2 is a perspective view of the digital camera 110, the multifunction device 900, and the belt 190. In this embodiment, the multifunction device 900 is transported by the belt 190 of the belt conveyor. A part of the belt 190 is shown in the figure. The belt 190 forms a flat upper surface 191. In this embodiment, the bottom surface 909 of the multifunction device 900 is placed on the upper surface 191. The belt conveyor is configured so that the multifunction device 900 moves in front of the digital camera 110 in a transport direction D190. The transport direction D190 is a direction intersecting the shooting direction D110 of the digital camera 110 (for example, the transport direction D190 is approximately perpendicular to the shooting direction D110). A label sheet 800 is attached to a first side surface 901 of the multifunction device 900. The digital camera 110 is positioned so as to capture an image of the label sheet 800.

[0016] A2.Label sheet: 3A is a schematic diagram showing an example of a label sheet. In this embodiment, the label sheet 800 is a rectangular sheet. The label sheet 800 can represent various objects (e.g., logotypes, marks, descriptions, etc.).

[0017] 3B is a schematic diagram showing an example of a reference image I10 represented by reference image data I10d. The reference image is an image of an object (in this embodiment, a label sheet 800) without defects. Reference image I10 is a rectangular image with two sides parallel to a first direction Dx and two sides parallel to a second direction Dy perpendicular to the first direction Dx. In reference image I10, the label sheet 800 is positioned so as not to be tilted. That is, in reference image I10, of the contour of the label sheet 800, an upper side 800U and a lower side 800D are parallel to the first direction Dx, and a left side 800L and a right side 800R are parallel to the second direction Dy.

[0018] In this embodiment, the label sheet 800 is produced by printing an image of the label sheet 800 on a sheet. The reference image I10 represents an object image 800i, which is an image of the label sheet 800 to be printed. In producing the label sheet 800, the object image 800i is printed on a sheet using copy data representing the object image 800i. Copy data may be used as the reference image data I10d. In this embodiment, the reference image data I10d is color bitmap data representing the color components R (red), G (green), and B (blue) for each pixel. The data format of the reference image data I10d may be various other formats.

[0019] A3. Image generation model 300: FIG. 4 is a block diagram showing an example of the image generation model 300. The input image data I1d is image data input to the image generation model 300. The generated image data I2d is image data generated by the image generation model 300. The image data I1d and I2d are bitmap data. The images I1 and I2 of the image data I1d and I2d are rectangular images having two sides parallel to the first direction Dx and two sides parallel to the second direction Dy perpendicular to the first direction Dx. The number of pixels in the first direction Dx and the number of pixels in the second direction Dy are each determined in advance and are common to the image data I1d and I2d. The image data I1d and I2d represent the color values ​​of R, G, and B of each of a plurality of pixels. Note that the color of each pixel of the images I1 and I2 of the image data I1d and I2d is not limited to the three color components R, G, and B, and may be represented by any other color components (for example, luminance value). It is preferable that the reference image data I10d (FIG. 3(B)) represent the same color components as the image data I1d and I2d.

[0020] Image generation model 300 receives input image data representing a captured image of label sheet 800 captured by digital camera 110 (FIG. 2). Input image I1 of input image data I1d is a captured image of label sheet 800. In this embodiment, the label sheet 800 is captured while multifunction device 900 (and therefore label sheet 800) is being transported. That is, the label sheet 800 is captured while it is moving relative to digital camera 110. Input image I1 therefore represents label sheet 800 that is blurred in the transport direction (such blurring is also called motion blur).

[0021] Label sheet 800 may contain defects. Label sheet 800 in FIG. 4 contains defect 801 (here, a linear scratch). Image generation model 300 is trained to generate data I2d of generated image I2 from data I1d of input image I1, which represents the same label sheet 800 without defect 801. The position and size of label sheet 800 in generated image I2 are the same as those of label sheet 800 in input image I1. Note that generated image I2 has motion blur, just like input image I1.

[0022] The difference image I3 represents the difference between the input image I1 and the generated image I2. Each pixel in the difference image I3 represents the difference in color value of the corresponding pixel between the two images I1, I2. Among the pixels in the difference image I3, the pixels corresponding to the defect 801 may show a large difference. The remaining pixels may show a small difference (e.g., a value close to zero). In this embodiment, the difference image I3 is used to inspect the label sheet 800 for defects.

[0023] The image generation model 300 may be a variety of models that generate a blurred image of the same label sheet 800 without defects from a blurred image of the same label sheet 800 with defects. In this embodiment, the image generation model 300 is a so-called variational auto-encoder (VAE). The image generation model 300 has an encoder 302 and a decoder 307. The encoder 302 performs a dimensionality reduction process on the input image data I1d to generate latent data 305 that indicates the features of the input image I1. The decoder 307 performs a dimensionality restoration process on the latent data 305 to generate generated image data I2d. The encoder 302 and the decoder 307 may each have a variety of configurations. For example, the encoder 302 may have one or more fully connected layers that calculate the mean and standard deviation using the input image data I1d, and a layer that generates the latent data 305 using the mean and standard deviation. In generating the latent data 305, noise may be introduced by a so-called "reparameterization trick." The decoder 307 may then have one or more fully connected layers that use the latent data 305 to compute the generated image data I2d. The dimensionality of the latent data 305 may be any value less than the dimensionality of the image data I1d, I2d. The mean and standard deviation may be computed for each element of the latent data 305.

[0024] A4. Training image data generation process: 5 is a flowchart showing an example of a generation process of training image data used for training the image generation model 300. In this embodiment, the processor 210 (FIG. 1) uses the reference image data I10d (FIG. 3(B)) to generate a plurality of training image data representing an image similar to the captured image (here, an image having motion blur like the input image I1 in FIG. 4). In this embodiment, the processor 210 generates a plurality of training image data by rotating the reference image I10 at various angles, taking into account that the conveying direction with respect to the label sheet 800 (FIG. 2) may be changed. The processor 210 executes the process of FIG. 5 according to the generation program 231.

[0025] At S110, the processor 210 obtains reference image data I10d from the non-volatile storage device 230. As shown in Figure 3B, the reference image data I10d represents an image of the label sheet 800 in a stationary state. The reference image I10 is an example of a still image of the label sheet 800.

[0026] In S120, the processor 210 accepts input of a parameter set (the input parameter set is also referred to as an input parameter set). The parameter set is used to determine the amount of blur, which will be described later. In this embodiment, the operator inputs the parameter set by operating the operation unit 250 (FIG. 1). The processor 210 acquires the input parameter set by referring to information from the operation unit 250.

[0027] FIG. 3C and FIG. 3D are diagrams showing examples of parameters. In this embodiment, the parameter set includes a conveying speed V, an exposure time TM, a pixel resolution RS (also simply referred to as resolution RS), and a conveying direction Dt. FIG. 3C shows the parameters V, TM, and RS. The conveying speed V is the moving speed of the label sheet 800 relative to the digital camera 110 when the label sheet 800 (FIG. 2) is photographed (units are mm / sec, for example). The faster the moving speed of the belt 190, the faster the conveying speed V. In this embodiment, the moving speed of the belt 190, i.e., the conveying speed V, is constant. The exposure time TM is the exposure time of the digital camera 110 for photographing the label sheet 800 (units are sec, for example). The slower the shutter speed of the digital camera 110, the longer the exposure time TM. The pixel resolution RS is the length of the object per pixel of the photographed image (units are mm / pixel, for example). The pixel resolution RS can be increased by decreasing the distance between the digital camera 110 and the label sheet 800. The pixel resolution RS can also be increased by increasing the angle of view of the digital camera 110.

[0028] These parameters V, TM, and RS are determined according to the conditions for photographing the label sheet 800. In this embodiment, the conditions for photographing include the movement speed of the belt 190, the operational settings of the digital camera 110 (shutter speed, angle of view, etc.), and the distance between the label sheet 800 and the digital camera 110 at the time of photographing. In this embodiment, the parameters V, TM, and RS are determined in advance. The operator inputs the values ​​of the parameters V, TM, and RS via the operation unit 250.

[0029] FIG. 3D shows candidate directions D10a-D10d that are candidates for the transport direction Dt. In this embodiment, the multiple candidate directions D10a-D10d are determined based on a reference line 800m of the label sheet 800. In this embodiment, a preselected portion of the label sheet 800 is used as the reference line 800m. Here, the upper side 800U of the label sheet 800 is assumed to be the reference line 800m. The first candidate direction D10a is the direction from the center of the label sheet 800 toward the reference line 800m. In this embodiment, the first candidate direction D10a is the opposite direction to the second direction Dy. Hereinafter, the second direction Dy is also referred to as the +Dy direction, and the opposite direction to the +Dy direction is also referred to as the -Dy direction. The same applies to the other directions. A sign with "+" added to the beginning of the sign of the direction of interest indicates the same direction as the direction of interest, and a sign with "-" added to the beginning of the sign of the direction of interest indicates the opposite direction to the direction of interest. For example, the +Dx direction indicates the same direction as the first direction Dx, and the -Dx direction indicates the opposite direction to the first direction Dx.

[0030] The other candidate directions D10b-D10d are obtained by rotating the first candidate direction D10a by 90 degrees in a clockwise direction.

[0031] The worker selects one of the candidate directions D10a-D10d and inputs information indicating the selected candidate direction via the operation unit 250. The processor 210 uses the input candidate direction as the conveying direction Dt. Here, it is assumed that the candidate direction D10b is selected.

[0032] The conditions for photographing the label sheet 800 may be changed. The values ​​of the parameters V, TM, and RS may be changed by changing the conditions for photographing. The conveying direction Dt may also be changed. For example, the portion of the multifunction device 900 (FIG. 2) that is placed on the top surface 191 of the belt 190 may be changed from the bottom surface 909 to the side surface 908. In this case, the conveying direction Dt (FIG. 3(D)) is changed from the second candidate direction D10b to the third candidate direction D10c. In S120 (FIG. 5), the operator inputs the parameters V, TM, RS, and Dt that are associated with the latest conditions.

[0033] In S130, the processor 210 determines a blur reference direction Ds for the label sheet 800. The blur reference direction Ds is the reference direction of motion blur added in S200, which will be described later. The processor 210 determines the blur reference direction Ds to be the opposite direction to the transport direction Dt.

[0034] In S140, the processor 210 determines the blur amount. The blur amount is the amount of blur added in S200, which will be described later. FIGS. 6(A) and 6(B) are diagrams showing examples of a filter for adding blur (called a blur filter). The direction Db1 is the direction of the blur (also called the blur direction Db1). The blur direction Db1 can be set as the blur reference direction Ds (S130 (FIG. 5)). The direction Db2 is a direction perpendicular to the blur direction Db1 (also called the vertical direction Db2).

[0035] In this embodiment, the first blur filter BFa in FIG. 6A is used (also simply referred to as the first filter BFa). The first filter BFa indicates the weights of the pixels PX used to determine the color value of the pixel of interest PXi (such a filter is also called a kernel, a convolution matrix, or a mask). The first value B1 is the size of the first filter BFa in the blur direction Db1 (unit: number of pixels). The second value B2 is the size of the first filter BFa in the vertical direction Db2 (unit: number of pixels). The pixel of interest PXi is located at the end of the first filter BFa in the -Db1 direction and at the center of the vertical direction Db2. The first filter BF1 is configured to extend from the pixel of interest PXi in the blur direction Db1.

[0036] The color value of the pixel of interest PXi is set to the weighted average of the color values ​​of the pixels PX represented by the first filter BFa (the color value is calculated for each color component). In this embodiment, the weight of each pixel PX is uniform. Alternatively, the weight may vary depending on the position of the pixel PX (for example, the closer to the pixel of interest PXi, the larger the weight).

[0037] The larger the first value B1 of the first filter BFa, the larger the amount of blur in the blur direction Db1 added by the first filter BFa. The first value B1 indicates the amount of blur in the blur direction Db1 (the first value B1 is also called the first blur amount B1). Similarly, the larger the second value B2, the larger the amount of blur in the vertical direction Db2 added by the first filter BFa. The second value B2 indicates the amount of blur in the vertical direction Db2 (the second value B2 is also called the second blur amount B2).

[0038] FIG. 3C shows an example of a calculation formula for the blur amounts B1 and B2. The processor 210 calculates the first blur amount B1 by multiplying "V*TM / RS" by the first coefficient k1 ("*" is a multiplication symbol). "V*TM / RS" indicates the amount of movement of the label sheet 800 (FIG. 2) in the captured image within the exposure time TM when the label sheet 800 moves at a constant speed in the transport direction Dt (unit: number of pixels). "V*TM / RS" indicates the theoretical amount of motion blur. In reality, the label sheet 800 in the captured image may be blurred more than the theoretical amount of blur. Therefore, in this embodiment, the first blur amount B1 is calculated by multiplying "V*TM / RS" by the first coefficient k1, which is 1 or more. The first coefficient k1 is experimentally determined in advance so that the amount of blur added by the first filter BFa is approximately the same as the amount of motion blur in the actual captured image. Thus, the first filter BFa is configured to add motion blur to the image with a blur direction Db1.

[0039] The processor 210 calculates the second blur amount B2 by multiplying the first blur amount B1 by the second coefficient k2. When the label sheet 800 (FIG. 2) moves in the transport direction Dt, theoretically the blur amount in the vertical direction Db2 is zero. In reality, the label sheet 800 in the captured image may be blurred in the vertical direction Db2 due to vibrations of the multifunction device 900 (and therefore the label sheet 800) during transport. Therefore, in this embodiment, the second blur amount B2 is calculated by multiplying the first blur amount B1 by the second coefficient k2, which is greater than zero. Usually, the blur amount in the vertical direction Db2 is smaller than the blur amount in the blur direction Db1. Therefore, the second coefficient k2 is set to a value less than 1. The second coefficient k2 is experimentally determined in advance so that the blur caused by the first filter BFa is comparable to the blur of the actual captured image.

[0040] In this embodiment, the blur amounts B1 and B2 indicate the number of pixels. The processor 210 may use values ​​rounded to integers as the blur amounts B1 and B2.

[0041] In S150-S190 (FIG. 5), the processor 210 obtains intermediate image data using the reference image data I10d, and stores the intermediate image data in the storage device 215 (here, the non-volatile storage device 230). The intermediate image data is image data to which blur is added by the first blur filter BFa. FIG. 7(A)-FIG. 7(D) are diagrams showing examples of intermediate image data. In this embodiment, the processor 210 generates intermediate image data by rotating the reference image data I10d by a first angle R and a second angle Rf (angles R and Rf indicate clockwise rotation angles). Hereinafter, a code obtained by adding "d" to the end of a code representing an image indicates data representing that image. For example, the reference image I10 is represented by image data I10d.

[0042] The first angle R is selected from 0, 90, 180, and 270 (unit: degrees). Figures 7(A)-7(D) correspond to R=0, 90, 180, and 270, respectively. As described in Figure 2, the portion of the multi-function device 900 that is placed on the upper surface 191 of the belt 190 may be changed from the bottom surface 909 to another surface (e.g., the side surface 908). This change may cause the conveying direction Dt relative to the label sheet 800 (Figure 3(D)) to change. The conveying direction Dt may change in increments of 90 degrees, for example. The first angle R corresponds to such a change in the conveying direction Dt.

[0043] The second angle Rf corresponds to a small inclination of the label sheet 800 relative to the transport direction Dt. The second angle Rf is set to an angle smaller than 90 degrees. In this embodiment, the second angle Rf varies in 1 degree increments from -10 degrees to +10 degrees. Each of Figures 7(A) to 7(D) shows, from left to right, examples of three types of intermediate images: Rf=0, positive, and negative.

[0044] In S150-S190 of FIG. 5, the processor 210 generates intermediate image data for each of all combinations of the first angle R and the second angle Rf. In the first S150, the first angle R and the second angle Rf are set to zero. The processor 210 stores the reference image data I10d (FIG. 7(A)) corresponding to R=Rf=0 in the storage device 215 as intermediate image data. In FIG. 7(A), directions Ds and Ds0 are shown. The direction Ds is the blur reference direction Ds determined in S130, and is the opposite direction to the conveying direction Dt. The direction Ds0 indicates the blur reference direction Ds when R=Rf=0 (referred to as the target direction Ds0). In this embodiment, the target direction Ds0 is the -Dx direction.

[0045] In S160, the processor 210 determines whether rotation by the second angle Rf is necessary. If there is an unprocessed second angle Rf associated with the current first angle R, the determination result is Yes. In this case, in S170, the processor 210 sets the second angle Rf to an unprocessed angle and generates intermediate image data by rotating the reference image data I10d (the angle of rotation is R+Rf). As shown in images I11 and I12 in FIG. 7A, the processor 210 generates data I11d and I12d of intermediate images I11 and I12 by rotating the label sheet 800 by the second angle Rf in image I10 where Rf=0. As shown in the figure, the blur reference direction Ds also rotates together with the label sheet 800. The target direction Ds0 does not change.

[0046] After S170 (FIG. 5), in S150, the processor 210 stores the generated intermediate image data in the non-volatile storage device 230. The processor 210 repeats S170 and S150 until all second angles Rf corresponding to the current first angle R have been processed.

[0047] If all second angles Rf corresponding to the current first angle R have been processed (S160: No), in S180, the processor 210 determines whether rotation of the first angle R is necessary. If an unprocessed first angle R remains, the determination result is Yes. In this case, in S190, the processor 210 sets the first angle R to the unprocessed angle and generates intermediate image data by rotating the reference image data I10d. Here, the second angle Rf is set to zero. In this case, the angle of rotation is R.

[0048] Images I20, I30, and I40 in Figures 7(B)-7(D) show images generated in S190 (where Rf = 0). As shown, processor 210 generates data I20d, I30d, and I40d of intermediate images I20, I30, and I40 by rotating the entire reference image I10. Alternatively, processor 210 may generate data I20d, I30d, and I40d of intermediate images I20, I30, and I40 by rotating label sheet 800 within reference image I10. In this case, it is preferable that the length of reference image I10 in the first direction Dx and the length of reference image I10 in the second direction Dy are set to be sufficiently large so that label sheet 800 does not protrude from images I10, I20, I30, and I40.

[0049] After S190, in S150, processor 210 stores the generated intermediate image data in non-volatile storage device 230. Processor 210 then repeats S170 and S190 to generate intermediate image data for all combinations of first angle R and second angle Rf. Images I21 and I22 in FIG. 7(B) are generated by rotating label sheet 800 by the second angle Rf in image I20 where Rf=0. Images I31 and I32 in FIG. 7(C) are generated by rotating label sheet 800 by the second angle Rf in image I30 where Rf=0. Images I41 and I42 in FIG. 7(D) are generated by rotating label sheet 800 by the second angle Rf in image I40 where Rf=0.

[0050] If intermediate image data for all combinations of the first angle R and the second angle Rf have been generated (FIG. 5: S180: No), in S200, the processor 210 generates multiple training image data by adding blur to each of the multiple intermediate image data.

[0051] 8(A)-8(D) are diagrams showing examples of training image data. FIG. 8(A)-8(D) correspond to FIG. 7(A)-7(D), respectively. The code of a training image is a code obtained by replacing the first letter of the code of the corresponding intermediate image with "T" instead of "I". For example, training image T11 (FIG. 8(A)) is an image generated by adding blur to intermediate image I11 (FIG. 7(A)).

[0052] In this embodiment, the processor 210 adds blur to the intermediate image using a first blur filter BFa (FIG. 6(A)). The processor 210 sets the blur direction Db1 of the first blur filter BFa to the target direction Ds0 (FIGS. 7(A)-7(D)). As a result, blur in the target direction Ds0 (here, the -Dx direction) is added to each training image (FIGS. 8(A)-8(D)). As described in FIG. 6(A), the amount of blur added is approximately the same as the amount of blur in the actual captured image. In addition, blur in a vertical direction Db2 perpendicular to the blur direction Db1 is also added to each training image. As described in FIG. 6(A), the amount of blur in the vertical direction Db2 is smaller than the amount of blur in the blur direction Db1.

[0053] As explained in Figures 7(A)-7(D), the blur reference direction Ds of the label sheet 800 is the direction obtained by rotating the target direction Ds0 at an angle of "R+Rf." Therefore, in each of the training images T10-T12, T20-T22, T30-T32, and T40-T42 in Figures 8(A)-8(D), the blur direction Db1 is the direction obtained by rotating the blur reference direction Ds for the label sheet 800 at an angle of "-R-Rf."

[0054] In this manner, the processor 210 generates a plurality of training image data through the processes of S150-S200 (FIG. 5). In S210, the processor 210 stores the generated plurality of training image data in the storage device 215 (here, the non-volatile storage device 230). Then, the processor 210 ends the process of FIG. 5.

[0055] It should be noted that the processor 210 may increase the total amount of training image data by so-called data augmentation. The processing used in data augmentation may include various processing such as translation, magnification, and reduction.

[0056] A5. Training process of image generation model 300: Fig. 9 is a flowchart showing an example of a training process of the image generation model 300 (Fig. 4). In this embodiment, the image generation model 300 is trained so that, when training image data is input to the image generation model 300, the image generation model 300 generates generated image data showing the same image as the input training image. The processor 210 (Fig. 1) executes the process of Fig. 9 in accordance with the training program 232.

[0057] In S310, the processor 210 acquires a subset consisting of a plurality of training image data from the plurality of training image data generated in the process of Fig. 5. Unprocessed training image data is selected as the subset. The total number of training image data in the subset is predetermined.

[0058] In S320, the processor 210 generates generated image data by inputting the training image data of the subset to the image generation model 300. The processor 210 generates generated image data by performing calculations of each layer using calculation parameters of each layer of the image generation model 300. The generation of generated image data is performed for each training image data included in the subset.

[0059] Note that the width of the training image (the number of pixels in the first direction Dx) may be different from the width of an image that can be accepted by the image generation model 300. In this case, the processor 210 may convert the width of the training image to a width suitable for the image generation model 300 by performing a resolution conversion process in the first direction Dx of the training image. The same applies to the height of the training image (the number of pixels in the second direction Dy). The processor 210 may convert the height of the training image to a height suitable for the image generation model 300 by performing a resolution conversion process in the second direction Dy of the training image.

[0060] In S330, the processor 210 calculates an evaluation value using the training image data and the generated image data. The calculation of the evaluation value is performed for each training image data. Then, the processor 210 calculates a loss using a plurality of evaluation values ​​calculated from a plurality of training image data included in the subset. The loss may be, for example, the total value of the plurality of evaluation values.

[0061] The method for calculating the evaluation value may be any method suitable for training the image generation model 300. In this embodiment, the image generation model 300 is a variational autoencoder. In this case, various values ​​that enable maximization of the so-called variational lower bound can be adopted as the evaluation value. For example, the evaluation value may be the sum of a reconstruction error and a regularization error. The reconstruction error is a parameter indicating the difference between a training image and a generated image, for example, a cross-entropy error. The regularization error is a parameter indicating the difference between, for example, a combination of the mean and standard deviation used to calculate the latent data 305 and a standard normal distribution. The regularization error is calculated, for example, by the formula "-(1+log(s 2 )-m 2 -s 2 ) / 2" where m is the mean and s is the standard deviation. This formula represents the error of one element of the latent data 305. The regularization error may be the sum of the errors of each element of the latent data 305.

[0062] In S340, the processor 210 adjusts a plurality of calculation parameters of the image generation model 300 so as to reduce the loss. As an algorithm for adjusting the plurality of calculation parameters, for example, an algorithm using the backpropagation method and the gradient descent method may be adopted. Here, so-called Adam optimization may be performed.

[0063] In S350, the processor 210 determines whether or not a training end condition is satisfied. The training end condition may be any condition that indicates that the image generation model 300 has been properly trained. In this embodiment, the training end condition is that an end instruction is input from the operator. The processor 210 randomly acquires a predetermined number of training image data that have not been used for training from the multiple training image data. The processor 210 inputs each of the acquired multiple training image data to the image generation model 300 to acquire multiple generated image data. The processor 210 displays multiple pairs of training images and generated images on the display unit 240. The operator observes the display unit 240 and checks whether the generated image properly represents the training image. The operator operates the operation unit 250 according to the confirmation result to input an instruction to end training or an instruction to continue.

[0064] The training end condition may be other conditions. For example, the end condition may be that the loss calculated using a predetermined number of training image data that have not been used in the training is equal to or less than a predetermined loss threshold.

[0065] If it is determined that the training is not completed (S350: No), the processor 210 proceeds to S310 and executes processing of a new subset. If it is determined that the training is completed (S350: Yes), in S360, the processor 210 stores the data of the trained image generation model 300 in the storage device 215 (here, the non-volatile storage device 230). Then, the processor 210 ends the processing of FIG. 9.

[0066] In this manner, the image generation model 300 is trained to use training image data representing images of a defect-free label sheet 800 to generate generated image data representing the same label sheet 800, i.e., an image of the defect-free label sheet 800. If the label sheet 800 in the image input to the image generation model 300 has defects, the image generation model 300 reconstructs a defect-free partial image from the partial image showing the defects and generates generated image data representing an image of the label sheet 800 without the defects.

[0067] In this embodiment, as described in S200 of Fig. 5 and Fig. 8(A)-Fig. 8(D), each of the multiple training images has a blur similar to that of a real captured image. When a real captured image is input to the trained image generation model 300, the image generation model 300 can generate a generated image having a blur similar to that of the input captured image.

[0068] A6.Inspection process: The trained image generation model 300 (FIG. 1) is used to inspect the appearance of the label sheet 800. The processor 210 executes the inspection process according to the inspection program 233. The inspection process may be various processes for inspecting the appearance of the label sheet 800. For example, the processor 210 acquires data of a captured image of the label sheet 800 from the digital camera 110 (FIG. 2). Here, the processor 210 may convert the width of the captured image to a width suitable for the image generation model 300 by executing a resolution conversion process in the first direction Dx of the captured image. The processor 210 may also convert the height of the captured image to a height suitable for the image generation model 300 by executing a resolution conversion process in the second direction Dy of the captured image. The processor 210 generates generated image data by executing the calculation of the image generation model 300 (FIG. 4) using the data of the captured image. The processor 210 generates data of a difference image representing the difference between the captured image and the generated image. Processor 210 uses the difference image to determine whether label sheet 800 has a defect. The criteria for determining that label sheet 800 has a defect may be various criteria. For example, the criteria may be that the difference image contains pixels that exhibit a difference greater than a first predetermined threshold. The criteria may also be that the total number of pixels that exhibit a difference greater than a second predetermined threshold is greater than a third threshold.

[0069] In either case, as described above, the image generation model 300 generates a generated image having similar blur from a captured image having blur, so that the processor 210 can appropriately determine whether or not there is a defect.

[0070] As described above, in this embodiment, processor 210 (FIG. 1) executes the following processes in accordance with generation program 231. At S110 (FIG. 5), processor 210 acquires reference image data I10d of label sheet 800, which is an example of an object. The acquired reference image data I10d (FIG. 3(B)) represents label sheet 800 in a stationary state. Reference image data I10d is an example of still image data that represents a still image of label sheet 800.

[0071] At S120 (FIG. 5), processor 210 accepts input of a parameter set. In this embodiment, processor 210 obtains data representing the parameter set by referencing information from operation unit 250. As explained at S120, in this embodiment, the parameter set includes the transport speed V of label sheet 800 when it is transported, the exposure time TM of digital camera 110 for photographing label sheet 800 while it is being transported, and the resolution RS of digital camera 110.

[0072] After S120 (FIG. 5), the processor 210 generates training image data by executing a generation process SG using the reference image data I10d. The processes of S130-S200 corresponding to R=Rf=0 are an example of the generation process SG. In this embodiment, the generation process SG includes a blur process SB. The blur process SB includes S130, S140, and S200.

[0073] As described in FIG. 4, the image generation model 300 processes the captured image of the label sheet 800. As described in FIG. 3(C) and FIG. 4, the captured image has blur associated with the conveying speed V, the exposure time TM, and the resolution RS. In S200 (FIG. 5), the processor 210 executes additional blur processing using a parameter set including the conveying speed V, the exposure time TM, and the resolution RS. In this way, in the blur processing SB for generating training image data, a parameter set including the parameters V, TM, and RS associated with the blur included in the captured image processed by the image generation model 300 is used. Thus, the processor 210 can generate training image data suitable for training the image generation model 300.

[0074] In this embodiment, the parameter set includes the conveying direction Dt for the label sheet 800, as described in S120 of FIG. 5. S200 for the generation process SG of FIG. 5 includes a process of adding blur to the image I10 (FIG. 7A) that corresponds to R=Rf=0. As shown in the training image T10 of FIG. 8A, this blur adding process is a process of adding blur in the blur reference direction Ds parallel to the conveying direction Dt for the label sheet 800. The blur added by this adding process can appropriately represent the motion blur included in the captured image showing the label sheet 800 moving in the conveying direction Dt. In this way, the processor 210 can generate training image data suitable for training the image generation model 300 that processes captured images having such motion blur.

[0075] In this embodiment, the process of S200 (FIG. 5) includes a process of adding blur to images I20, I30, and I40 (FIGS. 7(B)-7(D)) corresponding to R=90, 180, and 270. As shown in the corresponding training images T20, T30, and T40 (FIGS. 8(B)-8(D)), in these blur adding processes, the blur direction Db1 is a direction obtained by rotating the blur reference direction Ds counterclockwise by any of the angles of 90, 180, and 270 degrees.

[0076] In this way, after S120 (FIG. 5), the processor 210 generates additional training image data T20d, T30d, and T40d by executing the first additional generation process SG1a using the reference image data I10d. The processes of S130-S200 corresponding to R=90, 180, and 270 and Rf=0 are examples of the first additional generation process SG1a. The first additional generation process SG1a includes S190 and S200. S200 for the first additional generation process SG1a includes three blur addition processes of three blur directions Db1 obtained by rotating the blur reference direction Ds at an angle of 90 degrees, 180 degrees, or 270 degrees (FIGS. 8(B)-8(D)). Then, in S200, the processor 210 generates three types of additional training image data T20d, T30d, and T40d that are respectively associated with the three blur directions Db1 relative to the blur reference direction Ds. 2, the orientation of the multifunction device 900 having the label sheet 800 may change when the label sheet 800 is photographed. The additional training image data T20d, T30d, and T40d can be used to train a machine learning model (e.g., image generation model 300) to accommodate such changes.

[0077] In this embodiment, as shown in FIG. 6A, the first blur filter BFa adds blur in the vertical direction Db2 to the image in addition to blur in the blur direction Db1. That is, the blur adding process using the first blur filter BFa includes the blur adding process in the blur direction Db1 and the blur adding process in the vertical direction Db2. The blur amount B1 in the blur direction Db1 is greater than the blur amount B2 in the vertical direction Db2. For example, the process of adding blur to the image I10 (FIG. 7A) associated with R=Rf=0 includes the blur adding process in the direction perpendicular to the blur reference direction Ds (FIG. 8A). The blur amount B1 in the blur reference direction Ds is greater than the blur amount B2 in the perpendicular direction. In reality, the label sheet 800 in the captured image may be blurred in a direction perpendicular to the conveying direction Dt (however, the amount of blur in the perpendicular direction is smaller than the amount of blur in a direction parallel to the conveying direction Dt). In this embodiment, the processor 210 can generate data for a training image (e.g., training image T10) having blur similar to that of the captured image.

[0078] In this embodiment, the process of S200 (FIG. 5) includes a process of adding blur to a plurality of images (e.g., images I11, I12 (FIG. 7(A))) that correspond to a plurality of second angles Rf in the range from -10 degrees to +10 degrees. As shown in the corresponding training images (e.g., training images T11, T12 (FIG. 8(A))), in these blur adding processes, M directions (M=20 in this embodiment) that have a difference from the blur reference direction Ds by the second angle Rf are used as the blur direction Db1.

[0079] In this way, after S120 (FIG. 5), the processor 210 generates additional training image data (for example, training image data T11d, T12d (FIG. 8(A))) by executing the second additional generation process SG2a using the reference image data I10d. The processes of S130-S200 corresponding to the second angle Rf different from zero and R=0 are examples of the second additional generation process SG2a. The second additional generation process SG2a includes S170 and S200. S200 for the second additional generation process SG2a includes M blur addition processes of M blur directions Db1 having a difference (here, the second angle Rf) of less than 90 degrees from the blur reference direction Ds. In this embodiment, the M blur directions Db1 are 20 directions corresponding to 20 differences (excluding zero degrees) from -10 degrees to +10 degrees. Then, in S200, the processor 210 generates M types of additional training image data (e.g., training image data T11d, T12d) corresponding to the M blur directions Db1, respectively. The orientation of the label sheet 800 relative to the multifunction device 900 (FIG. 2) may include an error. Also, the orientation of the multifunction device 900 relative to the moving direction of the belt 190 (i.e., the orientation of the label sheet 800) may include an error. That is, the direction of the blur in the captured image may deviate from a direction parallel to the conveying direction Dt relative to the label sheet 800. The additional training image data (e.g., training image data T11d, T12d) is training of a machine learning model (e.g., image generation model 300) and can be used for training to deal with such deviations in the blur direction.

[0080] The magnitude of such deviation in the blur direction is usually less than 90 degrees. The magnitude of the second angle Rf may be various values ​​less than 90 degrees. For example, the multiple second angles Rf may be set in increments of 4 degrees from -20 degrees to +20 degrees. In either case, it is preferable that the multiple second angles Rf are set symmetrically in the clockwise direction and the counterclockwise direction.

[0081] In addition, in this embodiment, as described in FIG. 4, FIG. 8(A)-FIG. 8(D), etc., the training image represented by the training image data represents the label sheet 800 and has motion blur similar to the motion blur of the captured image. That is, the training image data represents an image of the label sheet 800 being conveyed, which is an image that imitates a captured image having motion blur. When a machine learning model (e.g., the image generation model 300) is trained using such training image data, the trained machine learning model can appropriately process a captured image having motion blur. Therefore, the process of reducing the blur of the image input to the machine learning model (also called blur correction) can be omitted.

[0082] In this embodiment, the blurring process SB (FIG. 5) includes S140. In S140, the processor 210 determines the blurring amount B1 using the transport speed V, exposure time TM, and resolution RS included in the parameter set. In this manner, the blurring process SB includes a process for determining the blurring amount using one or more parameters included in the parameter set among the transport speed V, exposure time TM, and resolution RS. With this configuration, the processor 210 can determine an appropriate blurring amount B1 using the parameter set.

[0083] B. Second Example: 10(A) and 10(B) are part of a flow chart of another embodiment of the training image data generation process. FIG. 10(A) shows S152 and S154 inserted between S150 and S160 in FIG. 5. FIG. 10(B) shows S210b executed in place of S210 in FIG. 5. The training image data generation process of this embodiment is the same as the process obtained by adding S152 and S154 and changing S210 to S210b in the training image data generation process of FIG. 5. In this embodiment, the processor 210 generates a plurality of first type training image data representing a label sheet 800 without defects and a plurality of second type training image data representing a label sheet 800 with defects. The first type training image data is the same as the training image data (FIGS. 8(A)-8(D)) generated in the embodiment of FIG. 5.

[0084] S150 (FIG. 10(A)) is the same as S150 in FIG. 5. At S150, the processor 210 stores the intermediate image data in the non-volatile storage device 230. This intermediate image data represents a defect-free label sheet 800, as shown in FIGS. 7(A)-7(D) (called the first type of intermediate image data).

[0085] In S152, the processor 210 generates the second type of intermediate image data by adding an image of a defect to a portion of the image of the label sheet 800 of the first type of intermediate image data. Examples of defect images are shown in the box indicating S152 in Fig. 10(A). The first defect image I91 represents a linear scratch. The second defect image I92 represents a stain. In this embodiment, data of a plurality of defect images representing different defects is prepared in advance.

[0086] In S152, the processor 210 executes the following process. The processor 210 randomly selects a defect image from the multiple available defect images. The processor 210 randomly determines a position within the image of the label sheet 800 of the first type of intermediate image data stored in the non-volatile storage device 230 in S150. The processor 210 then generates the second type of intermediate image data by adding the selected defect image to the determined position.

[0087] In S154, the processor 210 stores the second type intermediate image data generated in S152 in the non-volatile storage device 230 in association with the first type intermediate image data referred to for generating the second type intermediate image data. The only difference between the first type intermediate image and the second type intermediate image that are associated with each other is that a defect image is added to the second type intermediate image. After S154, the processor 210 proceeds to S160. The processor 210 generates a plurality of pairs of the first type intermediate image and the second type intermediate image by repeating the processes of S150-S154.

[0088] 10(B), if the judgment result in S180 is No (i.e., if the processing of all combinations of angles R and Rf is completed), in S200, the processor 210 generates a plurality of training image data by adding blur to each of the plurality of intermediate image data. The first type of training image data generated from the first type of intermediate image data is the same as the training image data described in FIG. 8(A)-FIG. 8(D).

[0089] Fig. 11 is a diagram showing an example of second type training image data generated from second type intermediate image data. Fig. 11 shows second type training images U10, U11, and U12 corresponding to the first type training images T10, T11, and T12 in Fig. 8(A), respectively. The code of the second type training image is a code obtained by replacing the first character of the code of the corresponding first type training image from "T" to "U". For example, the second type training image U11 (Fig. 11) corresponds to the first type training image T11 (Fig. 8(A)).

[0090] The difference between the first type training image and the second type training image that are associated with each other is that the second type training image has a defect image (the position and size of the label sheet 800 in the image are common to the first type training image and the second type training image that are associated with each other). For example, the difference between the first type training image T10 (FIG. 8(A)) and the second type training image U10 (FIG. 11) is that the second type training image U10 has a defect image I910. The same is true for other pairs of the first type training image and the second type training image (the second type training images U11 and U12 in FIG. 11 have defect images I911 and I912, respectively). In S200 (FIG. 10(B)), the processor 210 generates a plurality of pairs of the first type training image data and the second type training image data from a plurality of pairs of the first type intermediate image data and the second type intermediate image data.

[0091] In S210b (FIG. 10(B)), the processor 210 associates the first type training image data with the second type training image data and stores them in the non-volatile storage device 230. For example, the processor 210 stores data representing each of a plurality of pairs of the first type training image data and the second type training image data in the non-volatile storage device 230. Then, the processor 210 ends the training image data generation process.

[0092] A pair of a first type training image and a second type training image (referred to as a training image pair) can be used to train various machine learning models. A plurality of training image pairs may be used to train, for example, an image generation model 300 (FIG. 4). Here, the training process of FIG. 9 may be modified as follows. In S320-S330, the following two types of losses may be calculated: (1) First-type loss: A loss calculated using generated image data obtained by inputting a first-type training image data without defects to the image generation model 300 and the first-type training image data input to the image generation model 300. (2) Second type loss: Loss calculated using generated image data obtained by inputting second type training image data having defects to the image generation model 300 and first type training image data corresponding to the second type training image data input to the image generation model 300.

[0093] The first type of loss is the same as the loss in the embodiment of Fig. 9. With the first type of loss, the image generation model 300 is trained to generate an image of a defect-free label sheet 800 from an image of a defect-free label sheet 800.

[0094] The second type of loss is a loss for cases where the label sheet 800 in the captured image has defects. The image generation model 300 is trained to generate an image of the label sheet 800 without defects from an image of the label sheet 800 with defects using the second type of loss.

[0095] When the image generation model 300 (FIG. 4) is trained using the second type loss in addition to the first type loss, the trained image generation model 300 can properly generate an image of a defect-free label sheet 800 (e.g., generated image I2) from a captured image of a defective label sheet 800 (e.g., input image I1). Therefore, the possibility of misjudgment can be reduced in the inspection process using the image generation model 300.

[0096] As described above, in this embodiment, the image generation model 300 is used to inspect the label sheet 800 for defects. The processor 210 generates training image data by executing the generation process SGb (FIGS. 10(A) and 10(B)) using the reference image data I10d. The processes S130-S200 (FIGS. 5, 10(A) and 10(B)) for generating the second type training image data corresponding to R=Rf=0 are an example of the generation process SGb. The generation process SGb includes S152 (FIG. 10(A)) and a blur process SBb. The blur process SBb includes S130, S140 (FIG. 5) and S200 (FIG. 10(B)).

[0097] In S152, processor 210 adds an image of the defect to a portion of the image of label sheet 800. S200 (FIG. 10(B)) for blurring SBb includes a process of adding blur to the second type intermediate image to which the image of the defect has been added by S152. In this way, processor 210 generates second type training image data representing a blurred image of label sheet 800 having a defect. The second type training image data represents an image similar to the blurred captured image of label sheet 800 having a defect. The second type training image data can be used to train various machine learning models (e.g., image generation model 300) that process such captured images.

[0098] In this embodiment, the image generation model 300 is a model that generates a blurred image of a label sheet 800 without defects from a blurred image of the same label sheet 800 with defects, as described in FIG. 4. The process of S200 (FIG. 10(B)) includes a process of adding blur to the first type of intermediate image data without defects. In this manner, the processor 210 generates the first type of training image data (also called the corresponding training image data) that corresponds to the second type of training image data representing the defect by executing the correspondence generation process SG3b (FIG. 10(B)) using the reference image data I10d. The process of S130-S200 (FIGS. 5, 10(A), 10(B)) for generating the first type of training image data corresponding to R=Rf=0 is an example of the correspondence generation process SG3b. The correspondence generation process SG3b includes S200 without including S152 (FIG. 10(A)) of adding the defect. The S200 for the corresponding generation process SG3b includes the same blurring process as the blurring process SBb included in the generation process SGb (i.e., the blurring process for generating the second type of training image data). The corresponding training image data (i.e., the first type of training image data) represents a blurred image of a defect-free label sheet 800 as shown in Figures 8(A)-8(D). The label sheet 800 represented by the corresponding training image data (e.g., the label sheet 800 of the training images T10-T12 in Figure 8(A)) is the same label sheet 800 as the label sheet 800 represented by the corresponding second type of training image data (e.g., the label sheet 800 of the second type of training images U10-U12 in Figure 11). Pairs of the second type of training image data representing the defective label sheet 800 and the corresponding training image data representing the defective label sheet 800 can be used to train a machine learning model (e.g., image generation model 300) to generate, from a blurred image of the defective label sheet 800, a blurred image of the same defective label sheet 800. For example, as described with reference to FIG. 9, the machine learning model may be trained to generate, from a second type of training image representing the defects, a corresponding training image without the defects.

[0099] The training image data generation process of this embodiment includes the training image data generation process of Fig. 5. Therefore, this embodiment has the same various advantages as the embodiment of Fig. 5.

[0100] C. Third Example: Fig. 12 is a part of a flow chart of another embodiment of the training image data generation process. The only difference from the embodiment of Fig. 5 is that the blur direction Db1 is rotated by angles R and Rf instead of the image of the label sheet 800. Fig. 12 shows steps S150c-S210 after S140. Although not shown, the process before S140 is the same as S110-S140 in Fig. 5.

[0101] FIG. 13(A) and FIG. 13(B) are diagrams showing examples of training image data. In this embodiment, the blur direction Db1 rotates according to the first angle R and the second angle Rf (angles R and Rf indicate clockwise rotation angles). The first angle R indicates the angle of rotation from the blur reference direction Ds. As in the embodiment of FIG. 5, the first angle R is selected from 0, 90, 180, and 270 (unit: degree). FIG. 13(A) and FIG. 13(B) correspond to R=0 and 90, respectively (illustration of the cases of R=180 and 270 is omitted). The second angle Rf changes in 1 degree increments from -10 degrees to +10 degrees. In each of FIG. 13(A) and FIG. 13(B), examples of three types of training images are shown, in order from the left, with Rf=0, positive, and negative.

[0102] In S150c (FIG. 12), the processor 210 stores data indicating the current blur direction Db1 in the storage device 215 (for example, the non-volatile storage device 230). The data indicating the blur direction Db1 may be, for example, data representing "R+Rf". In the initial S150c, it is assumed that R=Rf=zero.

[0103] S160 is the same as S160 in Fig. 5. If an unprocessed second angle Rf remains (S160: Yes), in S170c, the processor 210 sets the second angle Rf to the unprocessed angle and sets the angle of the new blur direction Db1 to "R+Rf". After S170c, in S150, the processor 210 stores data indicating the new blur direction Db1 in the non-volatile storage device 230. The processor 210 repeats S170c and S150c until all second angles Rf corresponding to the current first angle R are processed.

[0104] If all the second angles Rf corresponding to the current first angle R have been processed (S160: No), the processor 210 proceeds to S180. S180 is the same as S180 in FIG. 5. If an unprocessed first angle R remains (S180: Yes), in S190c, the processor 210 sets the first angle R to the unprocessed angle and sets the angle of the new blur direction Db1 to "R+Rf". Here, the second angle Rf may be set to zero. In this case, the angle of the blur direction Db1 is R. After S190c, in S150c, the processor 210 stores data indicating the new blur direction Db1 in the non-volatile storage device 230.

[0105] The processor 210 repeats S170c and S190c to generate multiple angles of the blur direction Db1 using all combinations of the first angle R and the second angle Rf.

[0106] If all combinations of the first angle R and the second angle Rf have been processed (S180: No), in S200c, the processor 210 generates multiple training image data by performing a process of adding blur to the reference image data I10d for each blur direction Db1.

[0107] FIG. 13(A) shows three types of training images T50, T51, and T52 when "R=0". The angle between the blur direction Db1 of each training image T50, T51, and T52 and the direction determined by the first angle R (here, the blur reference direction Ds) is less than 90 degrees. FIG. 13(B) shows three types of training images T60, T61, and T62 when "R=90". The angle between the blur direction Db1 of each training image T60, T61, and T62 and the direction determined by the first angle R (here, the -Dy direction) is less than 90 degrees. Although not shown, in the cases of R=180 and 270, the angle between the blur direction Db1 and the direction determined by the first angle R is also less than 90 degrees.

[0108] S210 (FIG. 12) is the same as S210 in FIG 5. The processor 210 stores the generated training image data in the storage device 215 (here, the non-volatile storage device 230). Then, the processor 210 ends the process in FIG.

[0109] As described above, in this embodiment, the processor 210 generates training image data by executing the generation process SGc (FIG. 12) using the reference image data I10d. The processes of S130-S200c (FIGS. 5 and 12) corresponding to R=Rf=0 are examples of the generation process SGc. The generation process SGc includes the blur process SBc. The blur process SBc includes S130, S140 (FIG. 5), and S200c (FIG. 12). The process of S200c includes a process of adding blur to the blur direction Db1 corresponding to R=0, 90, 180, and 270. The blur direction Db1 corresponding to R=90, 180, and 270 is a direction obtained by rotating the blur reference direction Ds clockwise at angles of 90, 180, and 270 degrees, respectively.

[0110] In this way, after S120 (FIG. 5), the processor 210 generates additional training image data by executing the first additional generation process SG1c (FIG. 12) using the reference image data I10d. The processes of S130-S200c (FIGS. 5 and 12) corresponding to R=90, 180, 270 and Rf=0 are examples of the first additional generation process SG1c. The first additional generation process SG1c includes S190c and S200c. S200c for the first additional generation process SG1c includes three blur addition processes of three blur directions Db1 obtained by rotating the blur reference direction Ds at any angle of 90 degrees, 180 degrees, or 270 degrees. Then, in S200c, the processor 210 generates three types of additional training image data respectively corresponding to the three blur directions Db1. For example, training image T60 in FIG. 13(B) is an example of an additional training image corresponding to R=90 (additional training images corresponding to R=180, 270 are not shown). As described in FIG. 2, when the label sheet 800 is photographed, the orientation of the multifunction device 900 having the label sheet 800 may be changed. The additional training image data (e.g., data of training image T60) is training of a machine learning model (e.g., image generation model 300) and can be used for training to deal with such changes. Note that when the orientation of the multifunction device 900 having the label sheet 800 is changed, the orientation of the digital camera 110 may also be changed to the same orientation as the multifunction device 900.

[0111] In this embodiment, the process of S200 (FIG. 12) includes a process of adding blur of a plurality of blur directions Db1 (for example, a plurality of blur directions Db1 in FIG. 13(A)) corresponding to a plurality of second angles Rf in the range of −10 degrees to +10 degrees. As shown in the corresponding training images (for example, training images T51, T52), in the process of adding blur, M directions (M=20 in this embodiment) having a difference from the blur reference direction Ds by the second angle Rf are used as the blur directions Db1.

[0112] In this way, after S120 (FIG. 5), the processor 210 generates additional training image data (for example, data of training images T51, T52 (FIG. 13(A))) by executing the second additional generation process SG2c using the reference image data I10d. The processes of S130-S200c (FIGS. 5, 12) corresponding to the second angle Rf different from zero and R=0 are examples of the second additional generation process SG2c. The second additional generation process SG2c includes S170c and S200c. S200c for the second additional generation process SG2c includes M blur addition processes of M blur directions Db1 having a difference (here, the second angle Rf) of less than 90 degrees from the blur reference direction Ds. In this embodiment, the M blur directions Db1 are 20 directions corresponding to 20 differences (excluding zero degrees) from -10 degrees to +10 degrees. Then, in S200c, the processor 210 generates M types of additional training image data (e.g., data for training images T51, T52) corresponding to the M blur directions Db1, respectively. As described above, the direction of blur in the captured image may deviate from a direction parallel to the conveying direction Dt for the label sheet 800. The additional training image data (e.g., data for training images T51, T52) is training of a machine learning model (e.g., image generation model 300) and can be used to train it to handle such deviations in the blur direction.

[0113] The training image data generation process of this embodiment is similar to that of the embodiment of Fig. 5, except that the blur direction Db1 is rotated instead of the image of the label sheet 800. Therefore, this embodiment has the same various advantages as the embodiment of Fig. 5.

[0114] The embodiment of Fig. 12 may also be applied to the embodiments of Fig. 10(A) and Fig. 10(B). That is, in the embodiments of Fig. 10(A) and Fig. 10(B), processor 210 may rotate blur direction Db1 instead of rotating the image of label sheet 800.

[0115] D. Fourth Example: 14(A) and 14(B) are diagrams showing another embodiment of the process for acquiring the conveying direction Dt for the label sheet 800. Unlike the embodiment of FIG. 3(D), the conveying direction Dt is determined using the product conveying direction Dt9 and the object direction Dm. The product conveying direction Dt9 is the conveying direction relative to the multifunction device 900. The object direction Dm is the direction that indicates the orientation of the label sheet 800 relative to the multifunction device 900. In this embodiment, the parameter set input in S120 (FIG. 5) includes the product conveying direction Dt9 and the object direction Dm. The determination of the conveying direction Dt in this embodiment can be applied to the training image data generation process of any of the above embodiments.

[0116] 14(A) shows candidates for the product conveying direction Dt9 and the target direction Dm. In the figure, the surface of the multifunction device 900 on which the label sheet 800 is provided and candidate directions D9a-D9d are shown. In this embodiment, the candidate directions D9a-D9d are determined in advance based on the bottom surface 909 of the multifunction device 900. The downward candidate direction D9c is the direction from the center of the multifunction device 900 toward the bottom surface 909. The other candidate directions D9d, D9a, and D9b are obtained by rotating the candidate direction D9c clockwise by 90 degrees each time.

[0117] In S120 (FIG. 5), the worker inputs information indicating the selected product transport direction Dt9 and object direction Dm via the operation unit 250 (FIG. 1). The worker selects one candidate direction from the candidate directions D9a-D9d as the product transport direction Dt9. Here, it is assumed that the candidate direction D9b is selected as the product transport direction Dt9. The worker also selects one candidate direction from the candidate directions D9a-D9d as the object direction Dm. The object direction Dm is the direction from the center of the label sheet 800 toward the reference line 800m. Here, it is assumed that the candidate direction D9d is selected as the object direction Dm. The processor 210 uses the input product transport direction Dt9 and object direction Dm to determine the transport direction Dt for the label sheet 800.

[0118] Figure 14(B) shows candidate directions D10a-D10d for the conveying direction Dt, the same as Figure 3(D). The first candidate direction D10a is the direction from the center of the label sheet 800 toward the reference line 800m. The processor 210 determines the conveying direction Dt for the label sheet 800 by applying the relationship between the object direction Dm and the product conveying direction Dt9 to the first candidate direction D10a. For example, if Dm=D9d and Dt9=D9b, then Dt=D10c.

[0119] Thus, in this embodiment, the label sheet 800 is transported together with the multifunction device 900 in a state in which it is placed relative to the multifunction device 900, which is an example of a product (see FIG. 2). The parameter set received in S120 (FIG. 5) includes a product transport direction Dt9, which is the transport direction relative to the multifunction device 900, and an object direction Dm, which represents the orientation of the label sheet 800 relative to the multifunction device 900. S200 (FIGS. 5 and 10(B)) and S200c (FIG. 12) include a process of adding blur in a direction associated with R=Rf=0. As shown in the training images T10 and T50 in FIGS. 8(A) and 13(A), this blur addition process is a process of adding blur in a blur reference direction Ds parallel to the transport direction Dt relative to the label sheet 800. Here, the transport direction Dt is the transport direction relative to the label sheet 800 associated with the combination of the product transport direction Dt9 and the object direction Dm. The blur added by this additional processing can properly represent the motion blur contained in the captured image showing the label sheet 800 moving in the conveying direction Dt. In this manner, the processor 210 can generate training image data suitable for training the image generation model 300 to process captured images having such motion blur.

[0120] E. Fifth Example: 15(A) and 15(B) are diagrams showing another example of an object. In this embodiment, the object is a woven fabric 700. As shown in FIG. 15(A), the woven fabric 700 is transported in a transport direction D700 by a transport device (not shown). The digital camera 110 captures an image of the woven fabric 700 during transport, thereby generating captured image data of the woven fabric 700.

[0121] In this embodiment, the processor 210 (FIG. 1) inspects the appearance of the textile 700 using a captured image of the textile 700 and the image generation model 300 (FIG. 4) in the same manner as in the above embodiments. The processor 210 generates a plurality of training image data for training the image generation model 300. As the training image data generation process, the training image data generation process of any of the above embodiments can be adopted (FIGS. 5, 10(A)-10(B), and 12). In any case, the training image data is generated using reference image data representing the textile 700 in a stationary state without moving.

[0122] FIG. 15(B) shows an example of a reference image I90. The reference image I90 shows a detailed pattern of the woven fabric 700. An example of the pattern of the woven fabric 700 is shown in the lower part of FIG. 15(B). The woven fabric 700 is woven using warp threads Wp and weft threads Wf. The woven fabric 700 can be woven in various ways. The patterns W1, W2, and W3 in the figure show the patterns of "plain weave", "twill weave", and "satin weave", respectively. The reference image I90 shows a pattern formed by the warp threads Wp and the weft threads Wf.

[0123] The data I90d of the reference image I90 may be generated by various methods. For example, the data I90d of the reference image I90 may be acquired by taking a picture of the textile 700 in a stationary, motionless state with the digital camera 110.

[0124] Although not shown, the photographed image generated by the digital camera 110 (FIG. 15(A)) in the inspection process also shows the pattern formed by the warp threads Wp and the weft threads Wf. However, unlike the reference image I90, the photographed image for inspection has motion blur.

[0125] In S120 (FIG. 5) of the training image data generation process, the operator inputs information indicating the conveying direction Dt for the woven fabric 700 via the operation unit 250. In FIG. 15(B), candidate directions D90a and D90b of the conveying direction Dt are shown. The woven fabric 700 is usually conveyed in a direction parallel to the direction in which the warp threads Wp extend. In the reference image I90, the warp threads Wp extend in the second direction Dy. In this embodiment, the first candidate direction D90a is the -Dy direction, and the second candidate direction D90b is the +Dy direction. The operator selects one candidate direction from these candidate directions D90a and D90b, and inputs information indicating the selected candidate direction via the operation unit 250. The processor 210 uses the input candidate direction as the conveying direction Dt. Here, the candidate direction D90b may be selected. In S130, the processor 210 determines the blur reference direction Ds to be the opposite direction to the transport direction Dt.

[0126] The process following S130 (FIG. 5) may be a corresponding process of the training image data generation process of any of the above embodiments (FIG. 5, FIG. 10(A)-FIG. 10(B), FIG. 12). In any case, the processor 210 generates a plurality of training image data corresponding to a plurality of combinations of the first angle R and the second angle Rf. Here, the textile 700 may be attached to a conveying device (not shown) so that the conveying direction is reversed. In this case, too, the textile 700 is conveyed in a direction parallel to the extension direction of the warp threads Wp. Usually, the possibility that the textile 700 is conveyed in a direction parallel to the extension direction of the weft threads Wf is small. Therefore, in this embodiment, two angles, 0 degrees and 180 degrees, are used as the first angle R.

[0127] Thus, in this embodiment, the object is a woven fabric 700. As shown in FIG. 15(B), one angle (here, 180 degrees) is used as the first angle R other than 0 degrees. That is, the process of S200 (FIGS. 5 and 10(B)) or the process of S200c (FIG. 12) includes a process of adding blur to the blur direction Db1 associated with R=180 in addition to a process of adding blur to the blur direction Db1 associated with R=0 (i.e., the blur reference direction Ds). As shown in FIG. 15(B), the blur direction Db1 associated with R=180 is a direction obtained by rotating the blur reference direction Ds associated with R=0 by an angle of 180 degrees.

[0128] After S120 (FIG. 5), the processor 210 generates additional training image data by executing one of the first additional generation processes SG1a and SG1c (FIGS. 5 and 12) described above. In S200 and S200c for the first additional generation processes SG1a and SG1c, the processor 210 generates one type of additional training image data associated with one blur direction Db1 obtained by rotating the blur reference direction Ds at an angle of 180 degrees. When the first additional generation process SG1a (FIG. 5) is executed, the additional training image data is an image obtained by adding blur to an image obtained by rotating the reference image I90 (FIG. 15(B)) of the textile 700 at an angle of 180 degrees. When the first additional generation process SG1c (FIG. 12) is executed, the additional training image data is an image obtained by adding blur in the opposite direction to the blur reference direction Ds to the reference image I90 (FIG. 15(B)) of the textile 700.

[0129] As described above, the processor 210 generates additional training image data by one of the first additional generation processes SG1a and SG1c. As described above, the conveying direction of the textile 700 may be changed to the opposite direction. The additional training image data can be used to train a machine learning model (e.g., the image generation model 300) to respond to such a change.

[0130] After S120 (FIG. 5), the processor 210 generates additional training image data by executing one of the above-mentioned second additional generation processes SG2a and SG2c (FIGS. 5 and 12). In S200 and S200c for the second additional generation processes SG2a and SG2c, the processor 210 generates M types of additional training image data corresponding to M blur directions Db1, respectively (the M blur directions Db1 have a difference of less than 90 degrees (here, the second angle Rf) from the blur reference direction Ds). When the second additional generation process SG2a (FIG. 5) is executed, the additional training image data is an image obtained by adding blur to an image obtained by rotating the image of the textile 700 at the second angle Rf in the reference image I90 (FIG. 15(B)). When the second additional generation process SG2c (Figure 12) is executed, the additional training image data is an image obtained by adding blur in a blur direction Db1 indicated by the second angle Rf to the reference image I90 (Figure 15 (B)) of the fabric 700.

[0131] As described above, the processor 210 generates additional training image data by one of the second additional generation processes SG2a and SG2c. The orientation of the pattern (e.g., patterns W1-W3) of the textile 700 relative to the conveying device (not shown) may include an error. That is, the direction of the blur in the captured image may deviate from the direction parallel to the conveying direction Dt relative to the textile 700. The additional training image data can be used to train a machine learning model (e.g., image generation model 300) to handle such deviations in the blur direction.

[0132] F. Variations: (1) The parameter set (FIG. 5: S120) input for the training image data generation process may be composed of one or more various parameters. The input parameter set preferably includes one or more parameters selected from the transport speed V, the exposure time TM, and the resolution RS. One or two parameters of the parameters V, TM, and RS may be omitted from the input parameter set. Predetermined values ​​may be used as the values ​​of the omitted parameters.

[0133] Furthermore, the conveying direction Dt may be omitted from the input parameter set. In this case, a predetermined direction may be used as the conveying direction Dt. Furthermore, as in the embodiment of FIG. 14(A) and FIG. 14(B), the product conveying direction Dt9 and the object direction Dm may be used to determine the conveying direction Dt. Here, any one of the directions Dt9 and Dm may be omitted from the input parameter set. A predetermined direction may be used as the omitted direction.

[0134] (2) The input method of the input parameter set may be any other method instead of using the operation unit 250. The input method of the input parameter set may be various methods of inputting the input parameter set to the data processing device 200 from outside the data processing device 200. For example, the operator may prepare input data representing the input parameter set and store the input data in the non-volatile storage device 230 of the data processing device 200. In S120 (FIG. 5), the processor 210 may obtain the input parameter set by reading the input data from the non-volatile storage device 230. In this manner, the input parameter set may be input by storing the input data in the storage device.

[0135] (3) In each of the above embodiments, the second blur filter BFb (FIG. 6(B)) may be used as the blur filter instead of the first blur filter BFa (FIG. 6(A)). The second blur filter BFb is a filter obtained by setting the second blur amount B2 of the first blur filter BFa to "1". In this way, the addition of blur in the vertical direction Db2 may be omitted. When the second blur filter BFb is used, the processor 210 can add an appropriate blur in the blur direction Db1 to the image, as in the case where the first blur filter BFa is used. Note that the weight of each pixel PX of the blur filters BFa and BFb may be various values. For example, the closer the pixel PX is to the pixel of interest PXi, the greater the weight of the pixel PX.

[0136] The process of adding blur may be various other processes instead of the process using a blur filter. For example, the processor 210 may calculate the color value of the pixel of interest PXi by integrating the color value of the part of the object corresponding to the pixel of interest PXi according to the trajectory of the object's movement on the captured image within the exposure time TM.

[0137] In either case, various values ​​that indicate the amount of movement on the image during the exposure time TM can be used as the amount of blur. When a blur filter is used to calculate the color value of the pixel of interest PXi, the size of the blur filter (in number of pixels) can be used as the amount of blur.

[0138] (4) The first angle R used in S180 and S190 in Fig. 5 and S180 and S190c in Fig. 12 may include one or more angles arbitrarily selected from 90, 180, and 270 (degrees). In each embodiment and each modified example, the process for the first angle R other than zero (for example, S180 and S190 in Fig. 5 and S180 and S190c in Fig. 12) may be omitted.

[0139] (5) The total number M of the second angles Rf used in S160 and S170 in Fig. 5 and S160 and S170c in Fig. 12 may be various numbers equal to or greater than 2. In any case, the second angles Rf may include various angles less than 90 degrees. Note that in each embodiment and each modified example, the process for the second angles Rf (for example, S160 and S170 in Fig. 5 and S160 and S170c in Fig. 12) may be omitted.

[0140] (6) The training image data generation process may be various other processes instead of the processes of each embodiment and each modification. For example, in S170 and S190 of Fig. 5, the processor 210 may rotate the image and add blur to the rotated image. In S170c and S190c of Fig. 12, the processor 210 may rotate the blur direction Db1 and add blur in the rotated blur direction Db1.

[0141] (7) The machine learning model may be various other models instead of an autoencoder such as a variational autoencoder (FIG. 4). For example, the machine learning model may be a classification model that classifies the type of defect. The classification model may be a model that selects the type of defect from two classes, “with defect” and “without defect”. The classification model may be a model that selects the type of defect from multiple types (e.g., “without defect”, “scratch”, and “dirt”). The machine learning model may also be a feature extraction model used in anomaly detection technology called PaDiM (a Patch Distribution Modeling Framework for Anomaly Detection and Localization) or FastFlow (Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows). As the feature extraction model, various classification models or various object detection models can be adopted (e.g., convolutional neural network).

[0142] In either case, the machine learning model may be trained using training image data representing objects without defects. Alternatively, the machine learning model may be trained using training image data representing objects with defects. Alternatively, the machine learning model may be trained using both training image data representing objects without defects and training image data representing objects with defects. The training method may be a variety of methods appropriate for the machine learning model.

[0143] (8) The object may be various objects such as a pattern painted on the product, a nameplate (e.g., a three-dimensional name), knitted fabric, etc., instead of a label sheet and a woven fabric attached to the product. The product may be any product such as a scanner, a printer, a sewing machine, a cutting machine, a machine tool, a smartphone, etc., instead of the multifunction device 900.

[0144] (9) Still image data of the object used to generate the training image data may be data representing a design image of the object, such as block copy data, or alternatively, data of a photographed image of the object that is stationary and not moving may be used as the still image data.

[0145] (10) The method of transporting the object in the inspection process may be any method. For example, the multifunction device 900 (FIG. 2) may be placed on a table instead of the belt 190. The table may move to transport the multifunction device 900 (and thus the label sheet 800). In addition, in the inspection process, an area representing the object may be extracted from the captured image to generate image data to be input to a machine learning model.

[0146] Furthermore, multiple digital cameras may be used in the inspection process. The multiple digital cameras may capture different parts of the same product. For example, a first digital camera may capture a part of the multifunction device 900 that includes the label sheet 800, similar to digital camera 110 (FIG. 2), and a second digital camera may capture an image of the top surface of the multifunction device 900. Each of the captured images generated by the multiple digital cameras may be used to inspect the multifunction device 900.

[0147] (11) The generating device that generates the training image data may be a type of device (e.g., a digital camera, a scanner, a smartphone) different from a personal computer. In addition, a plurality of devices (e.g., computers) that can communicate with each other via a network may share a portion of the function of the training image data generation process, and as a whole, provide the function of the training image data generation process (a system including these devices corresponds to a generating device). In addition, training of the machine learning model may be performed by a device (e.g., a computer) different from the generating device for the training image data. In addition, inspection processing using the machine learning model may be performed by a device (e.g., a computer) different from the generating device for the training image data.

[0148] In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part or all of the configuration realized by software may be replaced by hardware. For example, the function of adding blur (e.g., S200 and S200c in Figs. 5 and 12) may be realized by a dedicated hardware circuit.

[0149] Furthermore, when some or all of the functions of the present disclosure are realized by a computer program, the program can be provided in a form stored in a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used in a state stored in the same or a different recording medium (computer-readable recording medium) from when it was provided. The "computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but may also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.

[0150] The above-mentioned examples and modifications can be combined as appropriate. The above-mentioned examples and modifications are provided to facilitate understanding of the present disclosure, and do not limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and the present invention includes equivalents thereof. [Explanation of symbols]

[0151] 110...digital camera, 190...belt, 191...top surface, 200...data processing device, 210...processor, 215...storage device, 220...volatile storage device, 230...non-volatile storage device, 231...generation program, 232...training program, 233...inspection program, 240...display unit, 250...operation unit, 270...communication interface, 300...image generation model, 302...encoder, 305...latent data, 307...decoder, 700...textile, 800...label sheet, 800D...bottom edge, 800L...left edge, 800R...right edge, 800U...top edge, 800i...object image, 800m...reference line, 801...defect, 900...multifunction device, 901...first side, 908...side, 909...bottom surface, I10d...reference image data

Claims

1. A program for generating training data for a machine learning model, comprising: An acquisition function for acquiring still image data of an object; a receiving function for receiving an input of an input parameter set including one or more parameters selected from a conveying speed of the object when conveyed, an exposure time of an imaging device for capturing an image of the object being conveyed, and a resolution of the imaging device; A generation function that generates training image data by executing a generation process using the still image data, the generation process including a blur process that is an additional process of blur using the input parameter set; A generating program that realizes the above on a computer.

2. The generating program according to claim 1 , The input parameter set further includes a conveying direction for the object; The blurring process includes an additional process of blurring the object in a reference direction that is a direction parallel to the transport direction. Generator.

3. The generating program according to claim 1 , the object is transported together with the product in a state in which the object is disposed relative to the product; The set of input parameters includes a conveying direction relative to the product and an orientation of the object relative to the product; The blurring process includes an additional process of blurring in a reference direction, which is a direction parallel to a conveying direction of the object, and which corresponds to a combination of the conveying direction of the product and the orientation of the object with respect to the product. Generator.

4. The generating program according to claim 2 or 3, further comprising: A first additional generation function is implemented in the computer to generate additional training image data by executing a first additional generation process using the still image data. The first additional generation process includes additional processing of N blurs in N directions (N is an integer of 1 to 3) obtained by rotating the reference direction at any one of an angle of 90 degrees, 180 degrees, and 270 degrees, The first additional generation function generates N types of additional training image data corresponding to the N directions, respectively. Generator.

5. The generating program according to claim 4, the object is a textile; N is 1, The N directions (N=1) are directions obtained by rotating the reference direction by an angle of 180 degrees. Generator.

6. The generating program according to claim 2 or 3, the blurring process includes an additional process of blurring in a direction perpendicular to the reference direction; The amount of blur in the reference direction is greater than the amount of blur in the perpendicular direction. Generator.

7. The generating program according to claim 2 or 3, further comprising: A second additional generation function is implemented in the computer to generate additional training image data by executing a second additional generation process using the still image data; The second additional generation process includes additional processes of M blurs in M ​​directions (M is an integer equal to or greater than 2) that differ from the reference direction by less than 90 degrees, The second additional generation function generates M types of additional training image data corresponding to the M directions, respectively. Generator.

8. A generating program according to any one of claims 1 to 3, The machine learning model is used to inspect the object for defects; The generating process includes a defect adding process for adding an image of a defect to a portion of the image of the object; the blurring process is performed on an image to which the image of the defect has been added by the defect addition process; The generating function generates the training image data representing a blurred image of an object having defects. Generator.

9. The generating program according to claim 8, The machine learning model is a model that generates a blurred image of an object without defects from a blurred image of the same object having a defect, The generation program further comprises: A correspondence generation function is realized in a computer, which generates corresponding training image data that is associated with the training image data by executing a correspondence generation process using the still image data, the correspondence generation process does not include the defect addition process, and includes a blur process that is the same as the blur process included in the generation process, the corresponding training image data represents a blurred image of a corresponding object without a defect, and the corresponding object is the same object as the object represented by the training image data. Generator.

10. A generating program according to any one of claims 1 to 3, The training image data represents a captured image of the object being transported, the captured image being simulated to have motion blur. Generator.

11. A generating program according to any one of claims 1 to 3, the blurring process includes a process of determining an amount of blurring by using one or more parameters among the conveying speed, the exposure time, and the resolution, which are included in the input parameter set; Generator.

12. 1. A method for generating training data for a machine learning model, comprising: acquiring still image data of the object; a receiving step of receiving an input of an input parameter set including one or more parameters selected from a conveying speed of the object during conveyance, an exposure time of an imaging device for capturing an image of the object during conveyance, and a resolution of the imaging device; A generating step of generating training image data by executing a generating process using the still image data, the generating process including a blurring process that is an additional process of blurring using the input parameter set; A generating method comprising:

13. A device for generating training data for a machine learning model, comprising: An acquisition unit that acquires still image data of an object; a receiving unit that receives an input of an input parameter set including one or more parameters selected from a conveying speed of the object when it is conveyed, an exposure time of an imaging device for capturing an image of the object being conveyed, and a resolution of the imaging device; A generation unit that generates training image data by executing a generation process using the still image data, the generation process including a blur process that is an additional process of blur using the input parameter set; A generating device comprising:

Citation Information

Patent Citations

  • Image processing method, computer program, image processing apparatus, and training method

    JP2022174516A