Program, information processing device, information processing method, learning model generation method, and photography system

Abstract

To provide a program or the like capable of accurately numeralizing the quality of an object.SOLUTION: A program acquires image data obtained by reading an object by a first scanner, inputs the acquired image data to a first learning model that has learned by first training data including the image data obtained by reading a plurality of kinds of reference objects by the first scanner, and image data obtained by averaging the color of each pixel in the image data and outputs corrected image data. Preferentially, the first learning model generates a red learning model, a green learning model, and a blue learning model in accordance with each color of red, green and blue, inputs red color data of the acquired image data to the red learning model, inputs green color data of the acquired image data to the green learning model, inputs blue color data of the acquired image data to a blue learning model, and outputs image data composed of colors outputted from the respective models.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing device, an information processing method, a learning model generation method, and an imaging system. [Background technology]

[0002] In quality inspections such as appearance inspections or functional inspections, a reference product and a finished product are sometimes compared visually, but human evaluations are unreliable. Therefore, a proposal has been made to quantify the color tone of an object using a spectrophotometer (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-292259 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the proposal disclosed in Patent Document 1 has a smaller area that can be measured at one time compared to an imaging device such as a scanner, etc. Therefore, when measuring an object to be inspected for quality, the measured value varies depending on the measurement location, making it difficult to accurately quantify the value.

[0005] In one aspect, an object is to provide a program or the like that can accurately digitize an object. [Means for solving the problem]

[0006] A program according to one aspect acquires image data obtained by scanning an object using a first scanner, inputs the acquired image data into a first learning model trained using first training data including image data obtained by scanning multiple types of reference objects using the first scanner and image data obtained by averaging the colors of each pixel in the image data, and outputs corrected image data. [Effects of the Invention]

[0007] In one aspect, it is possible to provide a program or the like that can accurately quantify the quality of an object. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram illustrating an example of the configuration of a color difference correction system according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of the configuration of an information processing device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a block diagram showing a configuration example of a terminal according to the present embodiment. [Figure 4] FIG. 1 is a schematic diagram illustrating an example of the configuration of a learning model. [Figure 5] FIG. 10 is a schematic diagram for explaining an outline of calculation of a correction value. [Figure 6] 10 is a flowchart showing a procedure for generating a red learning model. [Figure 7] 10 is a flowchart showing a processing procedure of a program executed by the information processing device. [Figure 8] FIG. 2 is an explanatory diagram showing the data layout of a building material DB. [Figure 9] FIG. 10 is an explanatory diagram showing the data layout of a reference value DB. [Figure 10] FIG. 10 is an explanatory diagram showing a screen for accepting input of type information. [Figure 11] FIG. 10 is an explanatory diagram showing a screen displaying the results of quality evaluation. [Figure 12] 10 is a flowchart showing a processing procedure of a program executed by the information processing device. [Figure 13] FIG. 10 is a schematic diagram illustrating an example of the configuration of a color difference correction system according to a second embodiment. [Figure 14] 10 is a flowchart showing a processing procedure for fine-tuning a red learning model. [Figure 15] FIG. 2 is an external perspective view of the scanner device. [Figure 16] FIG. 1 is a schematic diagram illustrating acquisition of image data of a plurality of objects. [Figure 17] 10 is a flowchart showing a processing procedure of a program executed by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] In this embodiment, we will explain a color difference correction system that corrects color differences (color variations) in image data acquired by capturing an image of an object. Color differences are the discrepancies in values ​​that occur when capturing an image of an object using, for example, a scanner. Even when capturing an image of a single-color piece of drawing paper, the colors of all pixels in the captured area do not necessarily match. Therefore, the color difference correction system eliminates the discrepancies in values ​​and accurately quantifies the color of the object.

[0010] An example of an object to which this system can be applied is an object with a pattern, such as a building material. Building materials are materials used in constructing a building, and include tiles, glass, flooring, wallpaper, etc. In this embodiment, the building material is described as tiles.

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] [Embodiment 1] 1 is a schematic diagram showing an example of the configuration of a color difference correction system according to the first embodiment. The color difference correction system includes an information processing device 10, a terminal 20, and a scanner 200. The information processing device 10 and the terminal 20 are connected to a communication network NW. The terminal 20 is connected to the scanner 200 so as to be able to communicate with each other.

[0013] The information processing device 10 is a server computer, a personal computer, or the like, and provides a color difference correction system. The information processing device 10 performs machine learning to learn predetermined training data and generates a machine learning model. Specifically, the information processing device 10 generates a first learning model described in the first embodiment and a second learning model described in the second embodiment. Note that the information processing device 10 may generate a first learning model for each scanner model.

[0014] The terminal 20 is an information processing terminal operated by a user who uses the color difference correction system (for example, an employee who performs quality inspections), and is a personal computer, a smartphone, a tablet terminal, etc. The terminal 20 can use the color difference correction system provided by the information processing device 10 via the communication network NW.

[0015] The scanner 200 is a two-dimensional or three-dimensional scanner used by a provider of a color difference correction system when generating a first learning model. The scanner 200 transmits captured image data to the information processing device 10 via the terminal 20.

[0016] 2 is a block diagram showing an example of the configuration of the information processing device 10 according to this embodiment. The information processing device 10 includes a control unit 11, a main storage unit 12, a communication unit 13, and an auxiliary storage unit .

[0017] The control unit 11 is a processor such as one or more central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), or quantum processors, and executes various types of information processing.

[0018] The main memory unit 12 is a temporary storage area such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), and temporarily stores data required for the control unit 11 to execute processing.

[0019] The communication unit 13 is a communication interface for connecting to the communication network NW.

[0020] The auxiliary storage unit 14 is a memory such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive). The auxiliary storage unit 14 stores a program 140 (program product) that causes the information processing device 10 to execute processing, a building material DB (database) 150, a reference value DB 151, and other data. The information processing device 10 may be provided with a reading unit that reads the portable storage medium 10a, and may read the program 140 from the portable storage medium 10a.

[0021] The auxiliary storage unit 14 includes a training data storage unit 141 and a learning model storage unit 142 .

[0022] The training data storage unit 141 stores first training data used in the generation process of the learning model. The first training data is, for example, data in which input information and output information for the learning model are associated with each other. In this embodiment, the input information is image data of a reference material, and the output information is image data obtained by averaging the colors of each pixel in the image data.

[0023] The learning model storage unit 142 stores data for the first learning model, including configuration information of the neural network and coefficients and thresholds of each neuron, etc. As the first learning model, a red learning model, a green learning model, and a blue learning model are generated corresponding to the colors red, green, and blue, respectively.

[0024] The reference materials are single-colored drawing paper, single-colored tiles, etc. The scanner 200 captures images of the 77 colored reference materials, and then transmits the image data of the reference materials to the information processing device 10.

[0025] The image data of the reference material includes red, green, and blue color data. When input to the learning model, the red color data is input to the red learning model, the green color data is input to the green learning model, and the blue color data is input to the blue learning model, corresponding to each color. The information processing device 10 combines the image data output from the learning models for each color and outputs image data with color differences corrected.

[0026] The color of image data can be expressed as absolute RGB (Red, Green, Blue) values ​​or as a ratio of RGB. For example, (0,0,0) is black, (255,255,255) is white, (255,0,0) is red, (0,255,0) is green, and (0,0,255) is blue.

[0027] 3 is a block diagram showing an example of the configuration of terminal 20 in this embodiment. Terminal 20 includes a control unit 21, a storage unit 22, a first communication unit 23, a second communication unit 24, an input unit 25, and a display unit 26.

[0028] The control unit 21 is one or more processors such as a CPU, an MPU, a GPU, or a quantum processor, and executes various types of information processing.

[0029] The storage unit 22 is a temporary storage area such as an SRAM or a DRAM, and temporarily stores data required for the control unit 21 to execute processing.

[0030] The first communication unit 23 is a communication interface for connecting to the communication network NW.

[0031] The second communication unit 24 is a communication interface for transmitting and receiving information to and from the scanner 200 .

[0032] The input unit 25 is an input interface such as a touch panel or mechanical operation buttons, and receives operation inputs from the user. Note that the input unit 25 may also be a microphone that collects voice commands from the user.

[0033] The display unit 26 is a display screen such as a liquid crystal display or an organic EL (Electro Luminescence) display, and displays images.

[0034] Although the present embodiment will be described assuming that one computer performs the processing, the processing may be distributed among a plurality of computers.

[0035] Fig. 4 is a schematic diagram showing an example of the configuration of a learning model. The neural network learning model includes an input layer that receives one or more pieces of data as input, an intermediate layer that performs arithmetic processing on the data received by the input layer, and an output layer that aggregates the arithmetic results of the intermediate layer and outputs one or more values.

[0036] As shown in Fig. 4, in this embodiment, red color data of each pixel is input as an actual measurement value, and red color data of each pixel is output as a correction value. Note that, although Fig. 4 illustrates the data input to the learning model as red, the same applies to green and blue.

[0037] In this embodiment, an example is shown in which information in units of one pixel is input to the learning model and information in units of one pixel is output, but this is not limited to this. Information in units of a region consisting of multiple pixels may be input and information in units of the region may be output. For example, information in units of 100 x 100 pixels may be input and information in units of 100 pixels may be output. In this case, the average, median, mode, etc. of the 100 pixel units is input.

[0038] A learning model has a neural network structure in which multiple neurons are interconnected. A neuron is an element that performs calculations on multiple inputs and outputs a single value as the calculation result. Neurons contain information such as weighting coefficients and thresholds used in the calculation.

[0039] In addition to neural networks, the learning model may be constructed using algorithms such as CNN (Convolutional Neural Network), Transformer, U-Net, autoencoder, decision tree, random forest, gradient boosting, or SVM (Support Vector Machine), or may be constructed by combining multiple algorithms.

[0040] The information processing device 10 learns a set of image data of a reference material and image data obtained by averaging the color of each pixel in the image data as first training data, and generates a learning model.

[0041] Specifically, the information processing device 10 inputs image data of a reference material into a neural network and obtains image data obtained by averaging the color of each pixel in the image data as output. The information processing device 10 compares the color of each pixel output from the neural network with the color averaged for each actual input pixel, and updates the weights between neurons, for example, using backpropagation, so that the error becomes zero. The information processing device 10 also inputs multiple types of reference materials into the neural network and updates the weights between neurons, thereby generating a final learning model for use in quality inspection of building materials.

[0042] Next, the process of generating a red, green, and blue learning model will be described with reference to Figures 5 and 6. Hereinafter, the red, green, and blue color data input to the learning model will be referred to as R values, G values, and B values.

[0043] 5 is a schematic diagram for explaining an overview of calculating a correction value. For ease of explanation, the image data is divided into blocks of 3×3 images (pixels), and a correction value is calculated in a 3×3 image area.

[0044] 5A shows the measured R-values ​​for each region. The information processing device 10 acquires the R-value of the reference material for each region from the scanner 200. For these measurement results, the information processing device 10 calculates the average R-value to be 75 by calculating (average R-value) = (sum of nine R-values) ÷ 9.

[0045] 5B shows the correction values ​​calculated based on the actual measurement values ​​and average values. The information processing device 10 calculates the correction value for each region by the following formula: (correction value) = (average R value) - (actually measured R value).

[0046] 5C shows the corrected R value. The information processing device 10 performs processing on the measurement results (actual measured values) based on the correction value to calculate the corrected R value.

[0047] The information processing device 10 performs similar processing on the G value and the B value. The information processing device 10 stores the calculated correction value for each color in the auxiliary storage unit 14.

[0048] 6 is a flowchart showing the procedure for generating a red learning model. The control unit 11 of the information processing device 10 executes the following process based on the program 140. The procedures for generating a green learning model and a blue learning model are similar, so their explanations will be omitted.

[0049] The control unit 11 acquires image data of a reference material captured by a scanner (step S101). The control unit 11 acquires the R value of the image data (step S102). The control unit 11 calculates the average R value for each pixel (step S103). The control unit 11 stores first training data including the measured R value and the calculated average R value (step S104).

[0050] The control unit 11 repeats the processes of steps S101 to S104 for different reference materials and stores multiple pieces of first training data in the main memory unit 12. The control unit 11 generates a red learning model based on the multiple pieces of first training data stored in the main memory unit 12 (step S105). The control unit 11 ends the series of processes.

[0051] 7 is a flowchart showing the processing procedure of the program 140 executed by the information processing device 10. The control unit 11 of the information processing device 10 executes the following processing based on the program 140.

[0052] The control unit 11 reads out the first learning model from the learning model storage unit 142 (step S201). The control unit 11 acquires image data of building materials from the scanner 200 (step S202).

[0053] The control unit 11 extracts red, green, and blue colors from the image data (step S203). The control unit 11 inputs each extracted color into the corresponding red, green, and blue learning model (step S204). The control unit 11 acquires the corrected R value, G value, and B value (step S205). The control unit 11 acquires the corrected image data (step S206). The control unit 11 ends the series of processes.

[0054] As described above, by generating red, green, and blue learning models, the information processing device 10 can correct color differences that occur when the scanner 200 captures an image of an object.

[0055] Next, details of specifying the number of spots on a tile and the area of ​​the pattern will be described.

[0056] 8 is an explanatory diagram showing the data layout of the building material DB 150. The building material DB 150 is a DB that stores fields for lot, tile type, shape and size, color number, manufacturing location, threshold value (black), and threshold value (white).

[0057] The lot field stores a predetermined number assigned to each tile. The tile type field stores the type of tile, such as interior tile. The shape and dimensions field stores the shape and dimensions of tiles that are equal in size to 200 mm square. The color number field stores a predetermined color number assigned to each tile color. The manufacturing location field stores the location where the tile was manufactured, such as the location of the factory. The threshold (black) field stores a threshold for identifying the area of ​​black spots or patterns contained in white or nearly white tiles. The threshold (white) field stores a threshold for identifying the area of ​​white spots or patterns contained in black or nearly black tiles.

[0058] 9 is an explanatory diagram showing the data layout of the reference value DB 151. The reference value DB 151 is a DB that stores fields for the serial number, R value, G value, B value, number of black spots, number of white spots, and pattern area. The fields for the R value, G value, B value, number of black spots, number of white spots, and pattern area store predetermined reference values ​​that serve as standards when shipping an industrial product.

[0059] The serial number field stores a predetermined serial number assigned to each tile. The R value field stores a reference R value. The G value field stores a reference G value. The B value field stores a reference B value. The number of black spots field stores a reference number of black spots. The number of white spots field stores a reference number of white spots. The pattern area field stores a reference pattern area.

[0060] The worker places a tile on the scanner 200 and captures an image of the tile. The worker inputs information (type information) about the type of the imaged tile via the terminal 20. The information processing device 10 accepts the input of the type information.

[0061] The information processing device 10 requests the scanner 200 to acquire image data of the tile. The scanner 200 transmits the image data to the information processing device 10. The information processing device 10 inputs the received image data into the first learning model and stores the output corrected image data in the auxiliary storage unit 14.

[0062] The information processing device 10 identifies pixels containing spots from the image data stored in the auxiliary storage unit 14 based on a threshold value corresponding to the color number. Specifically, the information processing device 10 identifies spots according to the color of the building material. For example, spots are identified for tiles that are white or near-white based on a threshold value (black), for tiles that are black or near-black based on a threshold value (white), and for tiles that are blue based on a threshold value (black) and a threshold value (white).

[0063] After counting the number of pixels containing the speckles, the information processing device 10 determines whether or not a predetermined number of pixels or more are connected. When determining whether or not they are connected, if there are a predetermined number of pixels or more consecutively in the X-axis direction or the Y-axis direction in the image area, the speckles of the tile are considered to be a pattern.

[0064] If a predetermined number of pixels or more are connected, the information processing device 10 determines that the identified spot is a pattern. The information processing device 10 measures the area of ​​the pattern based on the number of connected pixels. The information processing device 10 stores the measured area of ​​the pattern in the auxiliary storage unit 14.

[0065] On the other hand, if the predetermined number of pixels or more are not combined, the information processing device 10 identifies specks on the tile based on a threshold value read from the building material DB 150. The information processing device 10 counts the number of specks based on the identified specks on the tile. The information processing device 10 stores the counted number of specks in the auxiliary storage unit 14.

[0066] The information processing device 10 displays on the terminal 20 the coordinate information (for example, (X, Y) = (100, 150)) of the location of the identified spots or patterns and the measured number of spots or area of ​​the patterns in correspondence with each other.

[0067] As described above, the information processing device 10 can identify the number of spots on a tile and the area of ​​the pattern in association with the coordinate information.

[0068] FIG. 10 is an explanatory diagram showing a screen for accepting input of type information.

[0069] The information processing device 10 accepts input of type information through the terminal 20. The type information includes the type of tile (e.g., interior floor tile, etc.), shape and dimensions (e.g., 200 mm square, flat), color number, and manufacturing location. The information processing device 10 identifies the tile based on the accepted type information. In the example screen of FIG. 10, the terminal 20 displays type information such as tile type: interior floor tile, shape and dimensions: 200 mm square, color number: 2, and manufacturing location: Gifu factory. The terminal 20 also displays image data acquired by the scanner 200.

[0070] In addition, when the worker inputs the lot through the terminal 20 and then selects the designation button, the information processing device 10 may read out the type information corresponding to the lot from the building material DB 150 and designate the input item corresponding to the type information.

[0071] Alternatively, the worker may use the terminal 20 to read the one-dimensional code or two-dimensional code attached to the tile, thereby identifying the tile.

[0072] In FIG. 10, after the terminal 20 receives the selection of the execute button at the bottom right, it displays the screen of FIG.

[0073] FIG. 11 is an explanatory diagram showing a screen that displays the results of the quality evaluation.

[0074] The terminal 20 displays the results of the quality evaluation of the building materials. In the example screen of Fig. 11, the terminal 20 displays the current lot, standard value, type information, image data of the scanned tile, RGB information, and the number of spots or the area of ​​the pattern.

[0075] The operator can select any area of ​​the tile to be measured via terminal 20. After receiving the selection of the area, terminal 20 displays the selected area (e.g., selected area: 6) surrounded by a bold frame. Terminal 20 displays the selected area as coordinate information, e.g., (X, Y) = (100, 150). Terminal 20 displays the measurement values ​​and average values ​​of the selected area.

[0076] The information processing device 10 may read the results of the previous measurement together with the results of the tile measured this time from the building material DB 150 and display them on the terminal 20. The worker can evaluate the quality of the building material by comparing the reference value, the measurement results of the previous lot, and the measurement results of the current lot.

[0077] 12 is a flowchart showing the processing procedure of the program 140 executed by the information processing device 10. The control unit 11 of the information processing device 10 executes the following processing based on the program 140.

[0078] The control unit 11 acquires image data of the tile through the scanner 200 (step S301). The control unit 11 receives input of tile type information from the worker (step S302). The control unit 11 reads out threshold values ​​corresponding to the color number from the building material DB 150 (step S303).

[0079] The control unit 11 inputs the image data of the acquired tile to the first learning model (step S304). The control unit 11 acquires the corrected image data output from the first learning model (step S305). The control unit 11 identifies pixels containing spots from the image data based on the read threshold (step S306).

[0080] After counting the number of pixels including the speck (step S307), the control unit 11 determines whether or not a predetermined number of pixels or more are connected (step S308). If the predetermined number of pixels or more are connected (step S308: YES), the control unit 11 determines that the identified speck is a pattern. The control unit 11 measures the area of ​​the pattern based on the number of connected pixels (step S309). On the other hand, if the predetermined number of pixels or more are not connected (step S308: NO), the control unit 11 counts the number of speckles based on the speckles of the identified tile (step S310).

[0081] The control unit 11 acquires coordinate information of locations where spots or patterns are present (step S311). The control unit 11 displays the number of spots or the area of ​​the patterns and the coordinate information as measurement results on the terminal 20 (step S312). The control unit 11 ends the series of processes.

[0082] As described above, according to the first embodiment, the quality of building materials can be accurately quantified.

[0083] [Embodiment 2] In the second embodiment, a color difference correction system is used by a person who captures images of building materials using a scanner 300 different from the scanner 200. When capturing images of an object using different scanners, color differences may occur even for the same object due to differences between the scanners. Therefore, by generating a second learning model that is fine-tuned from the first learning model, a more accurate color difference correction system can be provided. Note that the same reference numerals are used to designate parts that overlap with the first embodiment, and descriptions thereof will be omitted.

[0084] 13 is a schematic diagram showing an example of the configuration of a color difference correction system according to Embodiment 2. The color difference correction system includes an information processing device 10, a terminal 20, a scanner 200, and a scanner 300. The scanner 300 is connected to the terminal 20 so as to be able to communicate with it.

[0085] The scanner 300 is a two-dimensional or three-dimensional scanner used when fine-tuning the first learning model. The scanner 300 transmits the acquired image data to the information processing device 10 via the terminal 20.

[0086] The information processing device 10 acquires image data of a reference material captured by the scanner 300 via the terminal 20. The information processing device 10 generates a second learning model using second training data including the acquired image data of the reference material and image data obtained by averaging the colors of each pixel in the image data. The amount of second training data may be less than the amount of first training data.

[0087] The generated second learning model is a learning model obtained by fine-tuning the first learning model. The information processing device 10 re-learns the weights of the entire model using the weights between neurons in the first learning model as initial values.

[0088] The information processing device 10 transmits the generated second learning model to the terminal 20. The terminal 20 uses the received second learning model for the digitization process of building materials.

[0089] 14 is a flowchart showing the processing procedure for fine-tuning the red learning model. The control unit 11 of the information processing device 10 executes the following processing based on the program 140. Note that the fine-tuning of the green learning model and the blue learning model is similar, so a description thereof will be omitted.

[0090] The control unit 11 acquires image data of a reference material captured by the scanner 300 (step S401). The control unit 11 acquires the R value of the image data (step S402). The control unit 11 calculates the average R value for each pixel (step S403). The control unit 11 stores second training data including the actually measured R value and the calculated average R value (step S404).

[0091] The control unit 11 repeats the processes of steps S401 to S404 for different reference materials and stores multiple pieces of second training data in the main memory unit 12. The control unit 11 fine-tunes the red learning model based on the multiple pieces of second training data stored in the main memory unit 12 (step S405). The control unit 11 then ends the series of processes.

[0092] As described above, according to the second embodiment, fine tuning can be performed to more appropriately quantify the quality of building materials.

[0093] As described above, according to the first and second embodiments, it is possible to provide a program or the like that can accurately quantify the quality of building materials.

[0094] In this embodiment, the color difference correction system is described as being applied to building materials, but the application is not limited to this. The system may also be applied to various other products, such as clothing, resin films, or printed materials.

[0095] In addition to building materials, this system may also be applied to various products such as food (eggs, bread, cookies, ham, tea leaves, and coffee beans), cloth, wood, plastic, leather, paper, posters, Japanese paper, powder, liquid (wine), stone, tatami mats, sponges, etc.

[0096] [Embodiment 3] In the third embodiment, an auxiliary device 40 (hereinafter referred to as scanner device 40) used when acquiring image data of an object will be described.

[0097] [Structure of scanner device 40] FIG. 15 is a perspective view of the appearance of the scanner device 40. As shown in FIG. 15, the scanner device 40 includes a scanner 200, a holder 41, a support 42, hinges 43, and a light-shielding case 44. The two-dot chain lines indicate rectangular plate-shaped tiles. The tiles are examples of objects. The following explanation will use the arrows in the figure to indicate the front, back, left, right, top, and bottom directions.

[0098] The scanner 200 is placed on a desk (not shown) with the scanning surface facing upward. The scanner 200 is connected to the terminal 20 via a cable (not shown). The scanner 200 may also be connected to the terminal 20 via wireless communication.

[0099] The holding part 41 has a left base 41a, a right base 41b, and a connecting plate 41c. The left base 41a and the right base 41b are rectangular pillars with their longitudinal direction aligned in the front-to-rear direction. The left base 41a and the right base 41b have the same shape and dimensions. The connecting plate 41c is a rectangular plate with its thickness aligned in the front-to-rear direction and its longitudinal direction aligned in the left-to-right direction. The connecting plate 41c connects the front end of the left base 41a to the front end of the right base 41b.

[0100] The left base 41a, the right base 41b, and the connection plate 41c together form a U-shape with an opening facing the rear side in Fig. 15. The width of the opening is wider than the width of the scanner 200.

[0101] The connecting plate 41c includes a generally U-shaped handle 41d. The handle 41d is located in the center of the connecting plate 41c. The handle 41d is attached to the front surface of the connecting plate 41c, with its opening facing the connecting plate 41c.

[0102] The holding part 41 has two guides 410. The guides 410 are rectangular pillars with their longitudinal direction oriented in the left-right direction. The guides 410 have a length equal to the left-right width of the left base 41a. The guides 410 are located at the rear ends of the upper surfaces of the left base 41a and the right base 41b, respectively. The front surfaces of the guides 410 are arranged on the same plane. The guides 410 are made of a material such as rubber.

[0103] The support parts 42 are two round rods extending in the front-to-rear direction. The support parts 42 are arranged parallel to each other and parallel to the ground. The two support parts 42 pass through the centers of the left pedestal 41a and the right pedestal 41b, respectively. The support parts 42 support the holding part 41 so that it can move in the longitudinal direction. The length of the support parts 42 is more than twice the length of the left pedestal 41a.

[0104] Legs 420 are fixed to the front and rear end surfaces of the support part 42. The legs 420 are rectangular pillars with their longitudinal directions facing up and down. The legs 420 are installed on the top surface of a desk (not shown). The vertical length of the legs 420 is such that the holding part 41 does not come into contact with the desk when the scanner device 40 is placed on the desk. The support part 42 also supports the holding part 41 so that the top surface of the holding part 41 is higher than the top surface of the scanner 200.

[0105] The light-shielding case 44 is a rectangular box. Figure 15 shows the light-shielding case 44 in an open state. The opening of the light-shielding case 44 faces forward. The lower edge of the light-shielding case 44 is connected to the upper surfaces of two legs 420 on the rear side via two hinges 43.

[0106] The light-shielding case 44 is closed by rotating approximately 90 degrees counterclockwise around the hinge 43 from the state shown in FIG. 15. When closed, the light-shielding case 44 is deep enough to form a space between it and the scanner 200 or the holder 41. When closed, the light-shielding case 44 is sized to cover the holder 41 in the second position, which will be described later.

[0107] The light-shielding case 44 has a generally U-shaped handle 44d. The handle 44d is attached with its opening facing the top surface of the light-shielding case 44. The handle 44d is located toward the front from the center of the top surface when the light-shielding case 44 is closed.

[0108] [How to use the scanner device 40] The initial state when using the scanner device 40 is when the light-shielding case 44 is open and the holder 41 is in the first position. In the first position, the front end of the holder 41 abuts against the front legs 420. In the first position, the scanner 200 is located behind the rear end of the holder 41. The worker places the tile so that it straddles both the left base 41a and the right base 41b. The worker slides the tile on the holder 41 and positions it by abutting the edge of the tile against the rubber guide 410. Because the guide 410 is made of rubber, the tile can be placed without being damaged at the corners, allowing image data to be acquired with good reproducibility.

[0109] After placing the tile at the first position, the worker presses the handle 41d to slide the holder 41 to the second position. The second position is a position where the rear end of the holder 41 abuts against the rear legs 420. In the second position, the scanner 200 is located between the left base 41a and the right base 41b. The vertical length of the legs 420 is such that the holder 41 does not come into contact with the desk when the scanner device 40 is placed on the desk. Therefore, the holder 41 can slide without coming into contact with the desk.

[0110] The support portion 42 supports the holder 41 so that it can move from a first position to a second position. In other words, the scanner device 40 is configured so that the tile placed on the holder 41 can slide together with the holder 41 from the first position to the second position. By distinguishing between the first position where the tile is placed and the second position where the tile is scanned in this way, even if an operator accidentally drops a tile, damage to the scanner 200 body can be avoided as long as the operation is performed at the first position. Furthermore, because the tile is not placed directly on the scanner 200, there is no risk of damaging the scan surface of the scanner 200.

[0111] The worker slides the holder 41 with the tile placed on it to the second position, and then uses the handle 44d to close the light-shielding case 44. When the light-shielding case 44 is closed, a space is formed inside the light-shielding case 44. This makes it possible to apply this system to three-dimensional objects such as tiles.

[0112] After closing the light-shielding case 44, the worker operates the terminal 20 to start scanning. By providing the light-shielding case 44 to the scanner device 40, it is possible to scan while blocking external light, enabling more accurate digitization of objects such as tiles. In addition, if the worker wants to check the status of the scan, they can do so by opening the opening on the front of the light-shielding case 44.

[0113] After scanning is complete, the worker uses handle 44d to open light-shielding case 44. Then, the worker pulls handle 41d to return holder 41 from the second position to the first position. When scanning another tile, the worker removes the scanned tile, places the new tile on holder 41, and repeats the above-described procedure.

[0114] [Modification of Scanner Device 40] When scanning, a transparent holding plate may be placed on the holding unit 41 to scan the tile. By placing this transparent holding plate, even if the tile size is smaller than the width that can be placed on the holding unit 41, the tile can be scanned without being placed directly on the scanner 200.

[0115] The guide 410 may be formed as an L-shaped rectangular parallelepiped, in addition to the rectangular pillar shape shown in FIG. 15. This shape allows tiles to be placed on the holder 41 with good reproducibility not only in the front-to-back direction but also in the left-to-right direction. Furthermore, the number of guides 410 installed on the holder 41 is not limited to two, and four may be installed to match the four corners of the tile. In this case, the guide 410 is configured so that its installation location can be moved to suit the size of the tile.

[0116] 15, the support portion 42 is described as being made up of two pieces, but this is not limited to this. As long as the holding portion 41 is supported so as to be parallel to the ground, the support portion 42 may be made up of one piece on either the left or right side. Furthermore, the support portion 42 may be a square bar instead of a round bar.

[0117] The handles 41d and 44d may be provided to improve the operability of the scanner device 40, but are not essential components.

[0118] As described above, according to the third embodiment, the present system can be applied more preferably using the scanner device 40.

[0119] [Variations] Next, an example of application of this system to scanning multiple objects will be described. In this modification, the case of scanning six eggs will be described as an example.

[0120] FIG. 16 is a schematic diagram showing the acquisition of image data of a plurality of objects.

[0121] As shown in FIG. 16, an operator places six eggs on scanner 200. When scanning food such as eggs, a transparent film or petri dish is first placed on scanner 200 to prevent scanner 200 from getting dirty, and then the food is placed on top of the film or dish. Before scanning begins, the operator places a light-shielding case 44 over the six eggs. Scanner 200 reads the six eggs and acquires image data. This configuration allows scanning of three-dimensional objects while blocking external light, thereby enabling more efficient acquisition of image data. Furthermore, scanning multiple objects with a single operation allows for efficient acquisition of image data.

[0122] Thereafter, similarly to the first and second embodiments, the scanner 200 transmits the image data to the information processing device 10 via the terminal 20. The information processing device 10 inputs the received image data into the first learning model and outputs corrected image data. This makes it possible to digitize the colors of the egg yolk and egg white and inspect the quality of the egg.

[0123] In addition to the eggs mentioned above, if the object is bread, it can determine the degree of doneness that will give it the best taste, if the object is a cookie, it can quantify the degree of doneness depending on its position in the oven, or if the object is raisin bread, it can measure the number and distribution of raisins. Furthermore, if the object is a liquid such as wine, it can acquire image data of the liquid by pouring wine into a petri dish placed on the scanner 200.

[0124] 17 is a flowchart showing the processing procedure of the program 140 executed by the information processing device 10. The control unit 11 of the information processing device 10 executes the following processing based on the program 140.

[0125] An operator places multiple objects on the scanner 200. The control unit 11 reads out the first learning model from the learning model storage unit 142 (step S501). The control unit 11 acquires image data including multiple objects from the scanner 200 (step S502). The control unit 11 inputs the acquired image data into the first learning model and outputs corrected image data (step S503). The control unit 11 ends the series of processes.

[0126] As mentioned above, this system can acquire image data not only from flat objects such as printed matter, but also from three-dimensional objects such as eggs and liquid objects such as wine, making it possible to accurately quantify multiple types of objects.

[0127] According to the above-described modified examples, the quality of not only building materials but also other objects such as food (eggs, bread, cookies, ham, tea leaves, and coffee beans), cloth, wood, plastic, leather, paper, printed matter (posters), Japanese paper, powder, liquid (wine), stone, tatami mats, sponges, and resin films can be appropriately controlled by quantifying the color.

[0128] The embodiments disclosed herein are illustrative in all respects and are not limiting. The technical scope of the present invention is determined not by the meaning disclosed above but by the description of the claims, and includes all modifications within the meaning and scope equivalent to the claims.

[0129] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, although the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limited to this format. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0130] 10. Information processing equipment 10a Portable storage media 11 Control section 12 Main memory 13 Communications Department 14 Auxiliary storage 140 Programs (Program Products) 141 Training data storage unit 142 Learning model memory unit 150 Building materials DB 151 Reference Value DB 20 terminals 21 Control Unit 22 Memory section 23 First Communications Department 24 Second Communications Department 25 Input section 26 Display section 200 scanners 300 scanner 40 Scanner device (auxiliary device) 41 Holding part 410 Guide 41a Left pedestal 41b Right pedestal 41c Connection board 41d Toride 42 Support part 420 legs 43 Hinge 44 Light-blocking case 44d Toride NW communication network

Claims

1. Acquiring image data by scanning the object with a first scanner; inputting the acquired image data into a first learning model that has been trained using first training data including image data obtained by reading a plurality of types of reference objects using the first scanner and image data obtained by averaging the colors of each pixel in the image data, and outputting corrected image data obtained by correcting the image data; The first learning model is fine-tuned to generate a second learning model using second training data including image data obtained by scanning a plurality of types of reference objects using a second scanner different from the first scanner and image data obtained by averaging the colors of each pixel in the image data. A program that causes a computer to perform a process.

2. the first learning model includes a red learning model, a green learning model, and a blue learning model generated corresponding to the colors red, green, and blue, respectively; inputting red color data of the acquired image data into the red learning model; inputting green color data of the acquired image data into the green learning model; inputting blue color data of the acquired image data into the blue learning model; Output image data composed of the colors output from each model The program according to claim 1.

3. the object is a tile, Obtain the image data of the corrected tile, obtaining type information relating to the type of the tile; Identifying the spots or patterns of the tiles based on the corrected image data and type information The program according to claim 1 or 2.

4. The type information includes the type, shape, and color number of the tile, and a threshold value for identifying spots or patterns corresponding to the type information is read out; The number of spots or the area of ​​the pattern is determined based on the read threshold value. The program according to claim 3.

5. Display the image data, type information, and spot count or pattern area of ​​the scanned tile The program according to claim 3.

6. acquiring image data obtained by scanning a plurality of objects with a first scanner; The acquired image data is input to the first learning model, and the corrected image data is output. The program according to claim 1 or 2.

7. In an information processing device including a control unit, The control unit Acquiring image data by scanning the object with a first scanner; inputting the acquired image data into a first learning model that has been trained using first training data including image data obtained by reading a plurality of types of reference objects using the first scanner and image data obtained by averaging the colors of each pixel in the image data, and outputting corrected image data obtained by correcting the image data; The first learning model is fine-tuned to generate a second learning model using second training data including image data obtained by scanning a plurality of types of reference objects using a second scanner different from the first scanner and image data obtained by averaging the colors of each pixel in the image data. Information processing device.

8. Acquiring image data by scanning the object with a first scanner; inputting the acquired image data into a first learning model that has been trained using first training data including image data obtained by reading a plurality of types of reference objects using the first scanner and image data obtained by averaging the colors of each pixel in the image data, and outputting corrected image data obtained by correcting the image data; The first learning model is fine-tuned to generate a second learning model using second training data including image data obtained by scanning a plurality of types of reference objects using a second scanner different from the first scanner and image data obtained by averaging the colors of each pixel in the image data. An information processing method in which processing is performed by a computer.

9. generating a first learning model that outputs corrected image data when image data is input, based on first training data including image data obtained by reading a plurality of types of reference objects with a first scanner and image data obtained by averaging the colors of each pixel in the image data; The first learning model is fine-tuned to generate a second learning model using second training data including image data obtained by scanning a plurality of types of reference objects using a second scanner different from the first scanner and image data obtained by averaging the colors of each pixel in the image data. A method for generating a learning model in which processing is performed by a computer.

10. In an imaging system including an information processing device, an auxiliary device, and a scanner, the information processing device includes a control unit, The control unit Acquiring image data by reading an object with the scanner; inputting the acquired image data into a learning model that has been trained using training data including image data obtained by reading a plurality of types of reference objects using the scanner and image data obtained by averaging the colors of each pixel in the image data, and outputting corrected image data obtained by correcting the image data; The auxiliary device is a holder for holding an object; a support portion that supports the holding portion so as to be movable from a first position to a second position where the object can be scanned by the scanner; a light-shielding case that shields the object from light at the second position; Equipped with The light-shielding case is provided on the support part so as to be openable and closable. Shooting system.

Images