Image data learning device, program, and image data learning method

Abstract

An image data learning device (100) is provided with a pre-learning unit (130) that pre-learns a model by using image data representing a pre-learning image generated from a geometric shape drawn according to a mathematical formula, and a main learning unit (140) that, on the model that has already pre-learned, executes main learning, which is learning necessary for performing inference, by using image data representing an image unrelated to the geometric shape drawn according to the mathematical formula.

Description

Image data learning device, program, and image data learning method

[0001] The present disclosure relates to an image data learning device, a program, and an image data learning method.

[0002] Artificial intelligence (AI) is known to be capable of making very complex and sophisticated judgments. For example, in the case of anomaly detection, it is possible to automatically generate a trained model that incorporates anomaly detection logic by learning data from normal and abnormal states.

[0003] However, in order for AI to make highly accurate judgments, a large amount of training data needs to be prepared in advance, and the human cost for acquiring the data is significant. The learning device described in Patent Document 1 reduces the cost of acquiring training data by generating training data that includes a synthetic image including a CG (Computer Graphics) model of the object and a teacher signal of the object.

[0004] Japanese Patent Application Laid-Open No. 2020-87310

[0005] However, since Patent Document 1 requires a composite image including a CG model of an object, the cost of creating the CG model is still required, and it cannot be said that the cost reduction is sufficient. In particular, there is a problem that this cost increases further in scenes where complex images are required as training data.

[0006] Therefore, one or more aspects of the present disclosure aim to sufficiently reduce the cost of acquiring training data.

[0007] An image data learning device according to one aspect of the present disclosure is characterized by comprising a pre-learning unit that performs pre-learning of a model using image data representing a pre-learning image, which is an image generated from geometric figures drawn according to mathematical formulas, and a main learning unit that performs main learning, which is learning necessary for making inference, on the model on which the pre-learning has been performed using image data representing an image unrelated to the geometric figures drawn according to mathematical formulas.

[0008] A program according to one aspect of the present disclosure is characterized in that it causes a computer to function as a pre-learning unit that performs pre-learning of a model using image data representing a pre-learning image, which is an image generated from geometric figures drawn according to mathematical formulas, and a main learning unit that performs main learning, which is learning necessary for making inference, on a model on which the pre-learning has been performed using image data representing an image unrelated to the geometric figures drawn according to mathematical formulas.

[0009] An image data learning method according to one aspect of the present disclosure is characterized in that it performs pre-learning of a model using image data representing a pre-learning image, which is an image generated from geometric figures drawn according to a mathematical formula, and then performs main learning, which is the learning necessary for making inference, on the model for which the pre-learning has been performed using image data representing an image unrelated to the geometric figures drawn according to a mathematical formula.

[0010] According to one or more aspects of the present disclosure, the cost of acquiring training data can be significantly reduced.

[0011] FIG. 1 is a block diagram showing an outline of the configuration of an image data learning device according to embodiment 1. FIG. 2 is a block diagram showing an outline of the configuration of a pre-learning data generation unit. (A) and (B) are schematic diagrams for explaining a mechanism for generating contours from mathematical expressions. FIG. 3 is a schematic diagram for explaining processing in a normal information addition unit. FIG. 4 is a block diagram showing an outline of the configuration of a PC. FIG. 5 is a flowchart showing the overall operation of the image data learning device according to embodiment 1. FIG. 6 is a flowchart showing the operation of the pre-learning unit. FIG. 7 is a flowchart showing the operation of the pre-learning data generation unit. FIG. 8 is a flowchart showing the operation of the inference unit in embodiment 1. FIG. 9 is a block diagram showing an outline of the configuration of an image data learning device according to embodiment 2. FIG. 10 is a flowchart showing the overall operation of the image data learning device according to embodiment 2. FIG. 11 is a flowchart showing the operation of the inference unit in embodiment 2.

[0012] 1 is a block diagram showing a schematic configuration of an image data learning device 100 according to embodiment 1. The image data learning device 100 includes a storage unit 110, an input unit 120, a pre-learning unit 130, a main learning unit 140, and an inference unit 150.

[0013] The storage unit 110 stores programs and data necessary for processing in the image data learning device 100. For example, the storage unit 110 includes a formula storage unit 111 and a model storage unit 112.

[0014] The mathematical formula storage unit 111 stores mathematical formulas used by the pre-learning unit 130 (described later). The stored mathematical formulas will be described later. The model storage unit 112 stores models for performing inference using image data as input data.

[0015] The input unit 120 accepts input of instructions from the user of the image data learning device 100 .

[0016] The pre-learning unit 130 performs pre-learning of the model using image data representing a pre-learning image, which is an image generated from a geometric figure drawn according to a mathematical formula. Here, the pre-learning unit 130 generates image data representing an image unrelated to the image to be inferred used by the inference unit 150, and performs pre-learning of the model using the image data so that learning by the main learning unit 140 is performed efficiently. For example, the pre-learning unit 130 performs pre-learning of the model using image data generated using a mathematical formula stored in the mathematical formula storage unit 111, and stores the model on which pre-learning has been performed in the model storage unit 112. The pre-learning unit 130 includes a pre-learning data generation unit 131 and a pre-learning execution unit 138.

[0017] The pre-learning data generation unit 131 generates image data representing an image unrelated to the image to be inferred used by the inference unit 150. Fig. 2 is a block diagram showing a schematic configuration of the pre-learning data generation unit 131. The pre-learning data generation unit 131 includes a contour generation unit 132, a normal information addition unit 133, an appearance setting unit 134, a lighting environment setting unit 135, a viewpoint setting unit 136, and a surface reflection synthesis unit 137.

[0018] The contour generation unit 132 draws a geometric figure according to a mathematical formula and generates a contour image showing a contour surrounding the center of the geometric figure. Here, the contour generation unit 132 reads a mathematical formula from the mathematical formula storage unit 111 shown in FIG. 1 and outputs contour information showing a contour image. FIGS. 3A and 3B are schematic diagrams for explaining a mechanism for generating a contour from a mathematical formula. For example, when the parameters h=1 and n=5 are given to the mathematical formula shown in the following formula (1), the contour generation unit 132 can obtain a figure Im1 as shown in FIG. 3A. Furthermore, when the parameters h=2 and n=5 are given to the mathematical formula shown in the following formula (1), the contour generation unit 132 can obtain a figure Im2 as shown in FIG. 3B.

[0019]

[0020] Then, the outline generating unit 132 sets the outlines of the above-described figures Im1 and Im2 as lines surrounding the centers CP1 and CP2 within the predetermined ranges Ar1 and Ar2.

[0021] In this way, the contour generation unit 132 identifies contours from geometric figures defined by mathematical expressions and generates contour information that indicates the contours as images. The generated contour information is provided to the normal information addition unit 133. Note that, since the contour generation unit 132 performs learning using a large number of image data, it is desirable to generate contour information that indicates a plurality of contours using a plurality of mathematical expressions and a plurality of parameters. Furthermore, the mathematical expressions used in this embodiment may be any expressions that can draw a geometric figure and have a line surrounding the center of the geometric figure.

[0022] 2 , the normal information addition unit 133 is a normal information generation unit that generates normal information indicating a normal vector included in the plane of the contour image and perpendicular to the contour shown in the contour image, and a normal vector perpendicular to the plane of the contour image at the center of the geometric figure, by interpolating the normal vector with a vector. For example, the normal information addition unit 133 receives contour information, identifies a normal to the contour shown in the contour information, and generates normal information indicating the normal. The normal information addition unit 133 then provides the generated normal information and contour information to the surface reflection synthesis unit 137.

[0023] The normals here are not two-dimensional normals in the image plane, but three-dimensional normals. Therefore, the normal information is represented by a three-dimensional vector having arbitrary components for each pixel in the image represented by the contour information.

[0024] For example, as shown in Fig. 4, the normal information adding unit 133 assigns a three-dimensional vector in the direction perpendicular to the plane of the image at the center pixel P1 of the image indicated by the contour information. Here, the three-dimensional vector is assigned in the depth direction or front direction in Fig. 4.

[0025] Then, for a pixel included in the contour, for example, pixel P2, the normal information adding unit 133 assigns a three-dimensional vector V in a direction perpendicular to the contour at pixel P2 within the plane of the image. Note that, although the three-dimensional vector V assigned to pixel P2 in Fig. 4 points outward from the central pixel P1, it may also point in the direction of the central pixel P1.

[0026] Furthermore, for pixels other than the center pixel and the pixels of the contour in the image indicated by the contour information, which are on the straight line between the center pixel and the contour, the normal information adding unit 133 may assign a three-dimensional vector by interpolating between the three-dimensional vector assigned to the center pixel and the three-dimensional vector assigned to the pixel at the intersection of the straight line and the contour. Note that the interpolation may be linear interpolation or curved interpolation such as quadratic curve interpolation.

[0027] Then, the three-dimensional shape can be specified by the normal vectors and the vertices of the interpolated vectors.

[0028] 2 , the appearance setting unit 134 sets the appearance of the three-dimensional shape specified by the vertices of the normal vectors and the interpolated vectors, and generates appearance information indicating the set appearance. The generated appearance information is provided to the surface reflection synthesis unit 137.

[0029] For example, the appearance setting unit 134 may set the appearance by generating a texture based on a predetermined pattern or parameters. The appearance setting unit 134 may generate a texture by combining one or more predetermined patterns, or may generate a texture using a random pattern. Alternatively, the appearance setting unit 134 may set the appearance using a single color without generating a texture. The above-described appearance setting may be input by the user via the input unit 120 shown in FIG. 1, or may be input by the user and stored in the storage unit 110 in advance.

[0030] The lighting environment setting unit 135 sets the lighting environment and generates lighting environment information indicating the set lighting environment. The generated lighting environment information is provided to the surface reflection synthesis unit 137.

[0031] For example, the lighting environment information consists of a parameter set including a light source position, type, direction, intensity, and color. One or more parameter sets may be set. Here, the light source position is expressed as a three-dimensional relative position with respect to the plane shown in the image represented by the contour information. The type is a point light source, a surface light source, or an ambient light. The lighting environment settings described above may be input by a user via the input unit 120 shown in FIG. 1, or may be input by a user in advance and stored in the storage unit 110.

[0032] The viewpoint setting unit 136 sets a viewpoint for viewing the three-dimensional shape specified by the normal vector and the vertices of the interpolated vector, and generates viewpoint information indicating the set viewpoint. The generated viewpoint information is provided to the surface reflection synthesis unit 137.

[0033] For example, the viewpoint information includes the position of the viewpoint, the direction of the line of sight, the angle of view, and the focal length. The viewpoint information is specified by a three-dimensional relative position with respect to the plane indicated by the image represented by the contour information. The viewpoint setting described above may be input by the user via the input unit 120 shown in FIG. 1, or may be input by the user in advance and stored in the storage unit 110.

[0034] The surface reflection synthesis unit 137 is an image generation unit that generates, as a pre-learning image, an image of a three-dimensional shape specified by the vertices of the normal vectors and the interpolated vectors, viewed from a set viewpoint under a set appearance and lighting environment. Here, the surface reflection synthesis unit 137 receives contour information, normal information, appearance information, lighting information, and viewpoint information, and generates image data representing the image for pre-learning. The image is generated by calculating the value of each pixel. The value of each pixel is determined as follows.

[0035] First, the surface reflection synthesis unit 137 determines, based on the appearance information, the basic values ​​of pixels included in an image representing a three-dimensional shape specified by the vertices of the normal vectors and interpolated vectors from the contour information and normal information. Next, the surface reflection synthesis unit 137 corrects the pixel values ​​based on the illumination information and viewpoint information.

[0036] The pixel value correction is performed taking into account diffuse reflection and specular reflection. For example, for diffuse reflection, the surface reflection synthesis unit 137 refers to the intensity and color of the illumination and calculates it by multiplying these by the basic value of the pixel. Specifically, if the illumination is white, the surface reflection synthesis unit 137 multiplies by "1" to leave the basic color as is, but if the illumination is black (e.g., no illumination), the surface reflection synthesis unit 137 multiplies by "0" to make the pixel value "0". This multiplication is performed, for example, for each of the three primary colors of RGB.

[0037] The surface reflection synthesis unit 137 calculates specular reflection based on the lighting direction, intensity, color, normal direction, and viewing direction. Specifically, the surface reflection synthesis unit 137 calculates the light source direction and specular reflection direction based on the lighting color, with the normal direction as the axis, and calculates specular reflection in a manner that reduces light intensity (darkens brightness) as the viewing direction deviates from the specular reflection direction. In other words, when the specular reflection direction and viewing direction are the same, the surface reflection synthesis unit 137 leaves the lighting color unchanged, and when they deviate by 90 degrees, it sets the color to "0." Furthermore, when the angles are between 0 and 90 degrees, the surface reflection synthesis unit 137 determines the degree of light reduction using a trigonometric function.

[0038] As described above, the surface reflection synthesis unit 137 generates an image for pre-learning by determining the pixel value of each image data based on the sum of diffuse reflection and specular reflection. This generates pre-learning data, which is image data for pre-learning.

[0039] 1 , the pre-learning execution unit 138 performs learning using the pre-learning data generated by the pre-learning data generation unit 131. For example, an appropriate correct answer (e.g., a numerical value) is determined for each formula stored in the formula storage unit 111, and the pre-learning execution unit 138 performs pre-learning of a model using the pre-learning data generated using the formula and the correct answer corresponding to the formula. The model pre-learned by the pre-learning execution unit 138 is stored in the model storage unit 112.

[0040] The main learning unit 140 performs main learning, which is learning necessary for inference, on a model that has undergone pre-learning, using image data representing images unrelated to geometric figures drawn according to mathematical formulas. The image data used in main learning is prepared in advance according to the purpose of the inference. Note that, as described above, images that are not generated from geometric figures are assumed to be images unrelated to geometric figures drawn according to mathematical formulas.

[0041] For example, the main learning unit 140 updates, through main learning, a model that has undergone pre-learning and is stored in the model storage unit 112. Here, the main learning unit 140 performs main learning using learning data that is generated by adding correct answer data according to the purpose of inference in the inference unit 150 to image data that indicates an image related to the image to be inferred used by the inference unit 150. The model trained by the main learning unit 140 is stored in the model storage unit 112.

[0042] The inference unit 150 performs inference using the model for which main learning has been performed by the main learning unit 140. The inference unit 150 includes an inference execution unit 151 and a relearning unit 152.

[0043] The inference execution unit 151 executes inference by inputting image data representing an image to be inferred into a model stored in the model storage unit 112 .

[0044] The relearning unit 152 receives the inference results from the inference execution unit 151 and performs relearning of the model stored in the model storage unit 112. Relearning does not need to be performed every time an inference is performed by the inference execution unit 151; for example, it may be performed after a predetermined number of inferences have been performed, or it may be performed when the need for relearning arises depending on the inference results. The need for relearning can be determined, for example, when the confidence level included in the inference result is equal to or lower than a predetermined value, or when a predetermined number of consecutive inferences have a confidence level equal to or lower than a predetermined value. In relearning, correct answer data is associated with image data representing the image to be inferred and used. The relearning performed here is the same as the main learning performed by the main learning unit 140, except for the input data (in other words, learning data).

[0045] The image data learning device 100 described above can be realized by a computer such as the PC 10 shown in Fig. 5. The PC 10 includes a storage 11 such as a hard disk drive (HDD) and a solid state drive (SSD), a memory 12, a processor 13 such as a central processing unit (CPU), a communication interface (I / F) 14 such as a network interface card (NIC), an input I / F 15 such as a keyboard and a mouse, and a display 16.

[0046] For example, the memory unit 110 can be realized by the storage 11 or the memory 12. The pre-learning unit 130, the main learning unit 140, and the inference unit 150 can be realized by loading a program stored in the storage 11 into the memory 12 and executing the program with the processor 13. The input unit 120 can be realized by the input I / F 15.

[0047] The program may be downloaded to the storage 11 from a recording medium (not shown) via a reader / writer (not shown) or from a network via the communication I / F 14, and then loaded onto the memory 12 and executed by the processor 13. Alternatively, the program may be directly loaded onto the memory 12 from a recording medium via the reader / writer or from a network via the communication I / F 14, and then executed by the processor 13. In other words, the program may be provided by a computer program product such as a recording medium.

[0048] 6 is a flowchart showing the overall operation of the image data learning device 100 according to embodiment 1. First, the pre-learning unit 130 generates image data for pre-learning using a formula, and then pre-learns a model using the image data for pre-learning (S10). The pre-learned model is stored in the model storage unit 112 of the storage unit 110.

[0049] Next, the main learning unit 140 performs main learning on the pre-trained model stored in the model storage unit 112, using image data representing an image related to the image that is the target of inference by the inference unit 150 (S11). Then, the trained model that is the result of the main learning is stored in the model storage unit 112. Learning consisting of the pre-learning and main learning is completed with the above processing.

[0050] Next, the inference unit 150 performs inference using the trained model stored in the model storage unit 112 (S12). The inference result may be output from the image data learning device 100 and presented to the user, or may be stored in the storage unit 110.

[0051] Furthermore, the inference unit 150 performs re-learning on the trained model stored in the model storage unit 112 based on the inference result (S13).

[0052] 7 is a flowchart showing the operation of the pre-learning unit 130. First, the pre-learning data generating unit 131 generates image data for pre-learning (S20).

[0053] Next, the pre-learning execution unit 138 reads out a model stored in the model storage unit 112 (S21). The model read out here is an unlearned model, and the parameters remain at their initial values.

[0054] Next, the pre-learning execution unit 138 executes pre-learning on the model read out in step S21 using the image data for pre-learning generated in step S20 (S22).The pre-learning execution unit 138 then stores the trained model, which is the training result, in the model storage unit 112 (S23).

[0055] 8 is a flowchart showing the operation of the pre-training data generation unit 131. First, the contour generation unit 132 reads out the mathematical formula stored in the mathematical formula storage unit 111 (S30). Then, the contour generation unit 132 generates contour information indicating the contour of a geometric figure based on the read out mathematical formula (S31).

[0056] Next, the normal information adding unit 133 identifies normals to the contour based on the contour information generated in step S31, thereby generating normal information and adding it to the contour information (S32).

[0057] Furthermore, the appearance setting unit 134 sets the appearance of the three-dimensional shape specified by the contour information and normal information, and generates appearance information indicating the set appearance (S33).

[0058] Furthermore, the lighting environment setting unit 135 sets one or more lighting environments including the position, type, direction, intensity, and color of the light source, and generates lighting environment information indicating the set one or more lighting environments (S34).

[0059] Furthermore, the viewpoint setting unit 136 sets a viewpoint for viewing the three-dimensional shape specified by the contour information and normal information, and generates viewpoint information indicating the set viewpoint (S35).

[0060] The surface reflection synthesis unit 137 then receives the contour information, normal information, appearance information, lighting environment information, and viewpoint information and generates image data for pre-learning (S36). Note that the order of the processes in steps S33 to S35 may be arbitrary.

[0061] FIG. 9 is a flowchart showing the operation of the inference unit 150 in the first embodiment. First, the inference execution unit 151 reads a trained model from the model storage unit 112 (S40). Then, the inference execution unit 151 acquires image data representing an image to be inferred (S41). The inference execution unit 151 may acquire the image data of the inference target from outside the image data learning device 100, for example, from a network such as the Internet, via a communication unit (not shown). Alternatively, the image data of the inference target may be stored in the storage unit 110 and the inference target image data may be acquired from the storage unit 110. The external device may be an imaging device that outputs image data or may be network storage. The communication unit is, for example, a functional unit realized by the communication I / F 14 shown in FIG. 5.

[0062] Next, the inference execution unit 151 executes inference by inputting the image data acquired in step S41 into the model read out in step S40 (S42).

[0063] Then, the re-learning unit 152 analyzes the inference result in step S42 and re-learns the model read out in step S40 (S43). The re-learned model is stored in the model storage unit 112.

[0064] As described above, according to the first embodiment, the pre-learning unit 130 performs pre-learning before performing main learning using image data unrelated to the images input to the main learning unit 140 and the inference unit 150. This makes it possible to generate a trained model with high inference accuracy even when the amount of training image data input to the main learning unit 140 is small. In other words, by using a pre-learned model and performing fine tuning through additional learning in the main learning unit 140 using training data according to the purpose of inference, high inference accuracy can be obtained even when the amount of training data for main learning is small. Because the image data used by the pre-learning unit 130 is unrelated to the images input to the main learning unit 140 and the inference unit 150, it can be automatically generated by the pre-learning unit 130 based on a mathematical formula. This reduces the cost of collecting image data required for learning.

[0065] 10 is a block diagram showing a schematic configuration of an image data learning device 200 according to embodiment 2. The image data learning device 200 includes a storage unit 210, an input unit 120, a pre-learning unit 130, a main learning unit 140, and an inference unit 250.

[0066] The input unit 120, pre-learning unit 130 and main learning unit 140 of the image data learning device 200 of embodiment 2 are similar to the input unit 120, pre-learning unit 130 and main learning unit 140 of the image data learning device 100 of embodiment 1.

[0067] The storage unit 210 stores programs and data necessary for processing in the image data learning device 200. For example, the storage unit 210 includes a formula storage unit 211 and a model storage unit 112. The model storage unit 112 of the storage unit 210 in the second embodiment is similar to the model storage unit 112 of the storage unit 110 in the first embodiment.

[0068] The mathematical formula storage unit 211 stores mathematical formulas used by the pre-learning unit 130. In the second embodiment, the mathematical formula storage unit 211 also stores mathematical formulas generated by the inference unit 250, as will be described later.

[0069] The inference unit 250 performs inference using the model for which main learning has been performed by the main learning unit 140. In the second embodiment, the inference unit 250 identifies features that need to be improved based on the confidence level of the inference, and identifies a new mathematical formula for drawing a geometric figure including the feature. Note that, since the feature amount typically differs for each feature, identifying the feature amount is synonymous with identifying the feature. Then, in the second embodiment, the pre-learning unit 130 may, at an appropriate timing, perform learning of the model for which main learning has been performed, using image data representing an image generated from a geometric figure drawn according to the identified new mathematical formula.

[0070] The inference unit 250 includes an inference execution unit 151, a relearning unit 152, a verification unit 253, and a mathematical formula identification unit 254. The inference execution unit 151 and the relearning unit 152 of the inference unit 250 in the second embodiment are similar to the inference execution unit 151 and the relearning unit 152 of the inference unit 150 in the first embodiment.

[0071] The verification unit 253 receives an input of the inference result from the inference execution unit 151, analyzes the inference result, and verifies the trained model to determine whether the trained model needs to be improved. If the verification unit 253 determines that the trained model needs to be improved, it provides judgment / inference result improvement information to the mathematical formula identification unit 254.

[0072] For example, the verification unit 253 verifies the trained model based on the inference result from the inference execution unit 151 and determines a policy for improving the inference result. For example, when the certainty of the inference for image data including a specific feature among the inference results is lower than a predetermined threshold, the verification unit 253 determines that learning of the image data including the feature is necessary and generates judgment / inference result improvement information indicating the feature.

[0073] The mathematical formula identification unit 254 identifies a new mathematical formula based on the inference result improvement information. The mathematical formula is identified based on the improvement policy included in the inference result improvement information. Here, since the inference result improvement information indicates a feature, a mathematical formula that can generate a geometric figure including the feature is identified.

[0074] Specifically, it is assumed that information for identifying a mathematical formula capable of generating a geometric figure including each feature is stored in the storage unit 210. For example, if the feature corresponds to a "trigonometric function," the geometric figures will be a Lissajous curve, a rose curve, a cardioid, etc., and the mathematical formula identification unit 254 identifies a mathematical formula for generating these. Also, if the feature corresponds to a "fractal," the geometric figures will be a Mandelbrot set, a Julia set, etc., and the mathematical formula identification unit 254 identifies a mathematical formula for generating these.

[0075] Fig. 11 is a flowchart showing the overall operation of the image data learning device 200 according to embodiment 2. Note that, in the flowchart shown in Fig. 11, steps in which the same processing as the steps included in the flowchart shown in Fig. 6 are performed are assigned the same reference numerals as in the flowchart shown in Fig. 6.

[0076] The processing of steps S10 to S13 in Fig. 11 is the same as the processing of steps S10 to S13 in Fig. 6. However, in Fig. 11, after the processing of step S13, the processing proceeds to step S54.

[0077] In step S54, the inference unit 250 performs a verification process for the inference result. If the verification process determines that the trained model needs to be improved, a new formula is added to the formula storage unit 211.

[0078] Fig. 12 is a flowchart showing the operation of the inference unit 250 in embodiment 2. Note that, in the flowchart shown in Fig. 12, steps in which the same processing as steps included in the flowchart shown in Fig. 9 are performed are assigned the same reference numerals as in the flowchart shown in Fig. 9.

[0079] The processing of steps S40 to S43 in Fig. 12 is the same as the processing of steps S40 to S43 in Fig. 9. However, in Fig. 12, after the processing of step S43, the processing proceeds to step S64.

[0080] In step S64, the verification unit 253 verifies the trained model based on the inference result after the re-learning process in step S43 (S64). Then, the verification unit 253 determines whether or not the trained model needs to be improved in the verification of step S64 (S65). If improvement is needed (Yes in S65), the verification unit 253 generates inference result improvement information indicating the feature quantities that need improvement and provides the inference result improvement information to the mathematical formula identification unit 254, and the process proceeds to step S66. If improvement is not needed (No in S65), the process ends.

[0081] In step S66, the mathematical formula identification unit 254 identifies a mathematical formula capable of generating a geometric figure including the feature quantity indicated in the inference result improvement information. Then, the mathematical formula identification unit 254 stores the identified mathematical formula in the mathematical formula storage unit 211.

[0082] The pre-learning unit 130 may perform pre-learning again at a predetermined interval or when a predetermined number of new formulas are added. At this time, the pre-learning unit 130 may perform pre-learning using all formulas included in the formula storage unit 211, or may perform pre-learning using only formulas that have been added since the previous pre-learning was performed.

[0083] As described above, according to the second embodiment, new formulas are added based on the inference results, and thus the accuracy of inference can be improved at any time by re-executing pre-learning using such formulas. Note that by re-executing learning based on formulas, it is possible to improve the accuracy of inference while avoiding overlearning.

[0084] 100, 200 Image data learning device, 110, 210 Memory unit, 111, 211 Formula memory unit, 112 Model memory unit, 120 Input unit, 130 Pre-learning unit, 131 Pre-learning data generation unit, 132 Contour generation unit, 133 Normal information addition unit, 134 Appearance setting unit, 135 Lighting environment setting unit, 136 Viewpoint setting unit, 137 Surface reflection synthesis unit, 138 Pre-learning execution unit, 140 Main learning unit, 150, 250 Inference unit, 151 Inference execution unit, 152 Re-learning unit, 253 Verification unit, 254 Formula identification unit.

Claims

1. An image data learning device comprising: a pre-learning unit that performs pre-learning of a model using image data representing pre-learning images, which are images generated from geometric figures drawn according to mathematical formulas; and a main learning unit that performs main learning, which is the learning necessary for making inference, on the model on which the pre-learning has been performed, using image data representing images unrelated to geometric figures drawn according to mathematical formulas.

2. The image data learning device according to claim 1, characterized in that the image data used in the main learning is prepared in advance according to the purpose of the inference.

3. The image data learning device of claim 1 or 2, characterized in that the pre-learning unit comprises: a contour generation unit that draws the geometric figure according to the formula and generates a contour image showing the contour surrounding the center of the geometric figure; a normal information generation unit that generates normal vectors and normal information showing the vectors by interpolating between normal vectors that are included in the plane of the contour image and are perpendicular to the contour and normal vectors that are perpendicular to the plane of the contour image at the center; an appearance setting unit that sets the appearance of the three-dimensional shape shown by the normal vectors and the vertices of the vectors; a lighting environment setting unit that sets a lighting environment; a viewpoint setting unit that sets a viewpoint; and an image generation unit that generates an image of the three-dimensional shape of the appearance viewed from the viewpoint in the lighting environment as the pre-learning image.

4. An image data learning device as described in any one of claims 1 to 3, further comprising an inference unit that performs the inference using the model for which the main learning has been performed, wherein the inference unit identifies features that need to be improved based on the confidence level of the inference and identifies a new formula for drawing a geometric figure including the features, and the pre-learning unit performs learning of the model for which the main learning has been performed using image data that represents an image generated from a geometric figure drawn according to the new formula.

5. A program that causes a computer to function as a pre-learning unit that performs pre-learning of a model using image data representing pre-learning images, which are images generated from geometric figures drawn according to mathematical formulas, and a main learning unit that performs main learning, which is learning necessary for making inference, on a model that has undergone pre-learning using image data representing images unrelated to geometric figures drawn according to mathematical formulas.

6. An image data learning method characterized by performing pre-learning of a model using image data representing pre-learning images, which are images generated from geometric figures drawn according to mathematical formulas, and then performing main learning, which is the learning necessary for making inference, on the model for which pre-learning has been performed using image data representing images unrelated to geometric figures drawn according to mathematical formulas.

Images