Detection support device, learning device, detection support method, learning method, and program

Abstract

To execute a processing process based on the visual perception of a human for a content image as processing on a processor.SOLUTION: A detection support device which supports detection of presence / absence of an unexpected pattern based on continuity of a pattern in a content image, generated in the content image generated by repeatedly arranging a predetermined pattern comprises: a content image acquisition unit which acquires the content image; a line-of-sight characteristic image generation unit which estimates a line-of-sight characteristic which is a feature of a line-of-sight of when detecting the pattern from the content image by using a line-of-sight characteristic learning model, and generates a line-of-sight characteristic image indicating the estimated line-of-sight characteristic for each pixel in the content image; and an analysis result output unit which displays the line-of-sight characteristic image. The line-of-sight characteristic learning model is a learned model obtained by learning a correspondence relation between the content image for learning, and the line-of-sight characteristic of a human obtained by detecting the pattern from the content image for learning.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a detection support device, a detection support method, and a program that support a person to easily detect defects in a content image. [Background technology]

[0002] In the field of building materials, design has long been considered an important added value, and decorative sheets with designs such as wood grain and abstract patterns are used by adhering them to the interior and exterior of buildings, furniture, furnishings, etc. Some designs of such decorative sheets are designed to harmonize a certain pattern by repeating the pattern as a unit.

[0003] In designs (hereinafter referred to as content images or visual content) in which predetermined patterns are synchronized, the continuity of the patterns can create unexpected patterns or shadows, resulting in defects that impair the design. Such defects cannot be detected at the stage of designing a single pattern, and are often only detected when an image is created in which a single pattern is repeatedly arranged, and the image is observed from a certain distance away. This is thought to be because a person observing an image visually senses some kind of spatial regularity (pattern) in the content image in which the pattern is repeated.

[0004] In general, there is a difference in the visual features that trained people (experts) and untrained people (non-experts) can detect in content images of the same design. This is thought to be because the visual features that people can detect are greatly influenced not only by the physical characteristics of the content image but also by the visual characteristics of the person observing it.

[0005] That is, while an expert can detect defects in the appearance of such content images, an unskilled person often cannot detect such defects. This is considered to be because an expert has acquired a method of detecting defects in content images through training. In other words, an expert is considered to have established a unique way of looking at content images and a unique processing method in the visual information processing process. If such a unique way of looking at content images can be quantified, even an unskilled person would be able to detect defects in such content images without undergoing burdensome training. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] L.Itti, C.Koch, E.Niebr: “A model of saliency-based visual attention for rapid scene analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 20, Issue:11:P.1254-1259, Nov 1998. [Non-Patent Document 2] D. Gao, V. Mahadevan, N. Vasconcelos: “On the plausibility of the discriminant center-surround hypothesis for visual saliency”, Journal of Vision, Vol. 8, 13, June 2008. [Non-Patent Document 3] J. Harel, C. Koch, P. Perona: “Graph-Based Visual Saliency”, Advances in neural information processing systems, 19:545-552, January, 2006. Summary of the Invention [Problem to be solved by the invention]

[0007] By reproducing on a processor a process that mimics the basic processing method of human vision, it is possible to reproduce to some extent the way humans see content images as processing performed by a device. For example, human vision receives light information from the eye and extracts two-dimensional information indicating brightness and color on the retina. Human vision then transmits the extracted information to the visual cortex of the brain. In the visual cortex of the brain, the intensity of brightness, spatial discontinuity (edges), continuity (gradients), chromaticity representation of color information (three primary color representations of red, green, and blue, and opponent color representations such as red-green / yellow-blue), etc. are individually processed based on the information such as brightness obtained from vision.

[0008] Furthermore, in the visual cortex of the brain, the spatial contrast (between the center and periphery) and directional continuity / discontinuity of the processed information are processed, and further combinations of these are processed, and so on. By repeating further processing using the results of the processing, higher-level and more complex patterns are processed in stages. If each of these processes can be realized on a sequential processor, it will be possible to process the same type of information as human vision, and it will be possible to control certain circuits (processing) to act strongly and others to act weakly.

[0009] On the other hand, for example, if we focus on the process by which an experienced person with a good eye (expert) looks at an object and notices a defect, we can imagine imitating that observation process and reproducing it on a processor. In other words, if the knowledge and sense that people acquire through a certain amount of training, known as experiential knowledge, can be reproduced on a processor, even an unskilled person will be able to achieve the same processing as an expert by using the processing results of the processor.

[0010] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a detection support device, a learning device, a detection support method, a learning method, and a program that can execute the human visual processing process for content images as processing on a processor. [Means for solving the problem]

[0011] In order to solve the above-mentioned problems, a detection support device that is one aspect of the present invention is a detection support device that supports detection of the presence or absence of unexpected patterns that arise in a content image generated by repeatedly arranging a specific pattern, due to the continuity of the pattern in the content image, and includes a content image acquisition unit that acquires the content image, a gaze feature image generation unit that estimates gaze features that are characteristics of the gaze when detecting the pattern from the content image using a gaze feature learning model and generates a gaze feature image that shows the estimated gaze features for each pixel in the content image, and an analysis result output unit that displays the gaze feature image, and the gaze feature learning model is a learned model that has learned the correspondence between the content image for learning and the gaze features of a person who detected the pattern from the content image for learning.

[0012] In order to solve the above-mentioned problems, a learning device that is one aspect of the present invention is a learning device that generates a gaze feature learning model used by a detection assistance device that assists in detecting the presence or absence of unexpected patterns due to the continuity of patterns in a content image generated by repeatedly arranging a specific pattern, the gaze feature learning model is a trained model that has learned the correspondence between the content image for learning and gaze features that are characteristics of the gaze of a person who detected the pattern from the content image for learning, and is configured to be able to select either deep learning or transfer learning using an existing learning model as a method for generating the gaze feature learning model, and is equipped with a deep learning unit that generates the gaze feature learning model using the selected method.

[0013] In order to solve the above-mentioned problems, a detection assistance method that is one aspect of the present invention is a detection assistance method performed by a computer, which is a detection assistance device that assists in detecting the presence or absence of unexpected patterns that arise in a content image generated by repeatedly arranging a specific pattern, due to the continuity of the pattern in the content image, wherein a content image acquisition unit acquires the content image, a gaze feature image generation unit estimates gaze features that are characteristics of the gaze when detecting the pattern from the content image using a gaze feature learning model, and generates a gaze feature image that shows the estimated gaze features for each pixel in the content image, an analysis result output unit displays the gaze feature image, and the gaze feature learning model is a learned model that has learned the correspondence between the content image for learning and the gaze features of a person who detected the pattern from the content image for learning.

[0014] In order to solve the above-mentioned problems, a learning method that is one aspect of the present invention is a learning method performed by a computer, which is a learning device that generates a gaze feature learning model used by a detection assistance device that assists in detecting the presence or absence of unexpected patterns due to the continuity of patterns in a content image generated by repeatedly arranging a specified pattern, wherein the gaze feature learning model is a trained model that has learned the correspondence between the content image for learning and gaze features that are characteristics of the gaze of a human who detects the pattern from the content image for learning, and a deep learning unit is configured to be able to select either deep learning or transfer learning using an existing learning model as a method for generating the gaze feature learning model, and generates the gaze feature learning model using the selected method.

[0015] A program according to one aspect of the present invention is a program for causing a computer to function as the detection assistance device.

[0016] A program according to one aspect of the present invention is a program for causing a computer to function as the learning device. Effect of the Invention

[0017] As described above, according to the present invention, the processing of a content image based on human vision can be executed as processing on a processor. [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram showing an example of the configuration of a detection support device 100 according to a first embodiment of the present invention. [Diagram 2] 3 is a flowchart showing a flow of processing performed by the detection support device 100 of the first embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing an example of an image to be processed in the first embodiment of the present invention. [Figure 4] 4 is an example of a visual feature image according to the first embodiment of the present invention. [Figure 5A] FIG. 4 is a diagram showing an analysis result of the first exemplary embodiment of the present invention. [Figure 5B] FIG. 4 is a diagram showing an analysis result of the first exemplary embodiment of the present invention. [Figure 5C] FIG. 4 is a diagram showing an analysis result of the first exemplary embodiment of the present invention. [Figure 6] 1 is a block diagram showing an example of the configuration of a detection support device 100A according to a second embodiment of the present invention. [Figure 7] 10 is a flowchart showing a flow of processing performed by a detection support device 100A according to a second embodiment of the present invention. [Figure 8] FIG. 2 is a block diagram showing an example of the configuration of a learning device 200 according to an embodiment. [Figure 9] 1 is a flowchart showing the flow of processing performed by the learning device 200 of the embodiment. [Figure 10] 4 is a flowchart showing an example of the operation of a process performed by the learning device 200 according to the present embodiment. [Figure 11] FIG. 13 is a block diagram showing an example of the configuration of a detection support device 100B according to a third embodiment of the present invention. [Figure 12] 13 is a flowchart showing a flow of processing performed by a detection support device 100B according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, a detection support device according to an embodiment will be described with reference to the drawings.

[0020] <First embodiment> First, the first embodiment will be described. The ability of humans to make certain judgments or feel something is wrong based on information obtained visually is dependent on the processing of the visual nerve mechanism in the human brain. Currently, the relatively early stages of brain processing are becoming clear. Therefore, the detection support device 100 is considered as a device that performs processing that models this processing. By having the detection support device 100 perform processing of the visual nerve mechanism, it becomes possible to more accurately reproduce visual information processing in the human brain.

[0021] The detection support device 100 of this embodiment uses, for example, a reference image and an inspection image as processing targets. The reference image is an image in which a defect has been detected by an expert. The inspection image is an image in which the defect has been eliminated by processing by the expert.

[0022] By targeting a reference image and a test image, the detection support device 100 can capture the difference in how an expert and an unskilled person see both images as a difference in content features that model the processing process of the neural mechanisms of human visual perception. In other words, the detection support device 100 presents the difference in how an expert and an unskilled person see both images in an easy-to-understand manner for each of several indicators, thereby showing what properties of the test image differ from the reference image and to what extent, thereby helping even an unskilled person to easily detect defects.

[0023] In the following, the image to be processed (content image) will be described as a still image, but the present invention is not limited to this. The content image may be a moving image, a video, or the like.

[0024] 1 is a block diagram showing an example of a configuration of a detection support device 100 according to a first embodiment of the present invention. The detection support device 100 includes, for example, a content image selection unit 101, a visual feature selection unit 102, a visual feature image generation unit 103, an image feature selection unit 104, a content feature amount calculation unit 105, an analysis method selection unit 106, an analysis unit 107, a content image DB (database) 108, a visual feature DB 109, a visual feature image storage unit 110, an image feature DB 111, a content feature amount storage unit 112, an analysis method DB 113, an analysis result storage unit 114, and an analysis result output unit 115. The content image selection unit 101 is an example of a "content acquisition unit."

[0025] The content image selection unit 101 acquires a content image. The content image is an image expressing a design generated by repeatedly arranging a predetermined pattern. For example, the content image is an image showing the design of a decorative sheet such as wallpaper used as a building material.

[0026] The content image selection unit 101 acquires an image selected by a user or the like from among a plurality of content images stored in the content image DB 108 as a content image. The method of selection by the user or the like may be any method. For example, the content image selection unit 101 refers to the content image DB 108 and causes the content image to be displayed on a display unit (not shown). The content image selection unit 101 acquires an image selected by the user or the like operating an external input device such as a mouse or keyboard as a content image.

[0027] In addition, the content images are not limited to those stored in the content image DB 108, but may be, for example, those acquired by the detection assistance device 100 via any input means such as a portable memory, a scanner, or a communication network.

[0028] The visual feature selection unit 102 selects visual features. Visual features are features that can be recognized by the human eye at a relatively early stage of processing in the human brain, such as luminance, chromaticity, contrast, gradient, edges, optical flow, and the like. The visual features include luminance, chromaticity, red-green chromaticity, yellow-blue chromaticity, orientation, luminance gradient, chromaticity gradient, red-green gradient, yellow-blue gradient, orientation gradient, luminance contrast, chromaticity contrast, red-green contrast, yellow-blue contrast, orientation contrast, and the like. The visual feature may be an index representing the ease with which a person's eye is attracted. Examples of the index representing the ease with which a person's eye is attracted include a visual attention model, a gaze prediction model, a saliency model, and a saliency model. For example, the methods described in Non-Patent Document 1, Non-Patent Document 2, Non-Patent Document 3, and the like can be used. The visual feature selection unit 102 selects a visual feature selected by a user or the like from a plurality of visual features stored in the visual feature DB 109, for example.

[0029] The visual feature image generating unit 103 generates a visual feature image by applying the visual feature to the content image. The visual feature image is an image showing the visual feature in the content image, for example, an image in which the degree of the visual feature (visual feature amount) calculated for each pixel in the content image corresponds to the position coordinate of the pixel. The content image used here is an image selected by the content image selecting unit 101. The visual feature used here is a visual feature selected by the visual feature selecting unit 102. The visual feature image generating unit 103 stores the generated visual feature image in the visual feature image storage unit 110.

[0030] The image feature selection unit 104 selects an image feature. The image feature is an image feature extracted using a known image processing technique, for example, a texture feature that can extract a spatial regularity (pattern) in a pattern formed by arranging a plurality of the same patterns. Examples of the texture feature include contrast, correlation, angular second moment, uniformity, etc.

[0031] The image feature selection unit 104 selects, for example, an image feature selected by a selection operation by a user or the like from a plurality of image features stored in the image feature DB 111. The image feature selection unit 104 outputs the acquired image feature to the content feature amount calculation unit 105.

[0032] The content feature amount calculation unit 105 calculates a content feature amount. The content feature amount is an index indicating the degree of features related to the appearance of the content image, and is, for example, a statistical amount statistically indicating image features in the visual feature image, which is calculated by applying image features to the visual feature image.

[0033] For example, when luminance is selected as the visual feature and contrast is selected as the image feature, the content feature amount is a value indicating what kind of contrast is formed by the luminance state that can be recognized by human vision in the content image. The visual feature image used here is an image generated by the visual feature image generation unit 103. The image feature used here is an image feature selected by the image feature selection unit 104.

[0034] The content feature amount calculation unit 105 outputs the calculated content feature amount to the analysis unit 107. In addition, the content feature amount calculation unit 105 causes the content feature amount storage unit 112 to store the calculated content feature amount.

[0035] The content feature amount may be a degree of visual feature (visual feature amount). In this case, the content feature amount calculation unit 105 calculates the content feature amount by using, for example, a visual feature image (without using image features).

[0036] The analysis method selection unit 106 selects an analysis method. The analysis method is a method for presenting content features, and is, for example, information indicating the type of graph showing the content features. Examples of graph types include line graphs, bar graphs, pie charts, and radar charts.

[0037] The analysis method selection unit 106 selects an image feature selected by a selection operation by a user or the like from a plurality of image features stored in the analysis method DB 113. The image feature selection unit 104 outputs the acquired image feature to the content feature amount calculation unit 105.

[0038] The analysis unit 107 applies an analysis method to the content feature amount to generate information for presenting the content feature amount in the content image. The analysis unit 107 stores the generated information in the analysis result storage unit 114. The analysis unit 107 also outputs the generated information to the analysis result output unit 115.

[0039] The content image DB 108 stores content images. For example, the content image DB 108 stores a content image associated with identification information that uniquely indicates the content image. The content image is acquired by the detection support device 100 via any input means, such as an external input device such as a mouse or a keyboard, or a portable memory, a scanner, or a communication network, and is stored in the content image DB 108.

[0040] The content image DB 108 may store a reference image and an inspection image obtained by processing the reference image in association with each other, or may store the content images in a state of being classified according to the type of the content image. The type of the content image may be classified according to, for example, the method of combining patterns, whether or not the sheet surface has irregularities when it is made into a decorative sheet, and the like.

[0041] The visual feature DB 109 stores visual features. For example, the visual feature DB 109 stores perceptual features associated with identification information that uniquely indicates the visual feature. The perceptual features are acquired by the detection support device 100 via, for example, an external input device or an input means, and are stored in the visual feature DB 109.

[0042] The visual feature image storage unit 110 stores the visual feature image generated by the visual feature image generation unit 103. The visual feature image storage unit 110 stores, for example, a visual feature image associated with identification information that uniquely indicates the visual feature image, identification information of a content image used to generate the visual feature image, and identification information of a visual feature used to generate the visual feature image.

[0043] The image feature DB 111 stores image features. For example, the image feature is stored in the image feature DB 111 in association with identification information that uniquely indicates the image feature. The image feature is acquired by the detection support device 100 via an external input device or an input means, and is stored in the image feature DB 111.

[0044] The content feature amount storage unit 112 stores the content feature amount calculated by the content feature amount calculation unit 105. The content feature amount storage unit 112 stores, for example, a content feature amount associated with identification information that uniquely identifies the content feature amount, identification information of a visual feature image used in calculating the content feature amount, and identification information of an image feature used in calculating the content feature amount.

[0045] The analysis method DB 113 stores the analysis method. For example, the analysis method associated with identification information that uniquely indicates the analysis method is stored in the analysis method DB 113. For example, the analysis method is acquired by the detection support device 100 via an external input device or an input means, and is stored in the analysis method DB 113.

[0046] The analysis result storage unit 114 stores the analysis result (information for presenting content features) by the analysis unit 107. The analysis result storage unit 114 stores, for example, the analysis result associated with identification information that uniquely indicates the analysis result, identification information of the content features used in the analysis, and the analysis method used in the analysis.

[0047] The analysis result output unit 115 outputs the analysis result (information for presenting the content feature amount) by the analysis unit 107. The analysis result output unit 115 outputs the analysis result to, for example, a display unit (not shown), and causes the display unit to display the analysis result.

[0048] FIG. 2 is a flowchart showing a flow of processing performed by the detection support device 100 according to the first embodiment of the present invention. Step S10: The detection support device 100 selects a content image by the content image selection unit 101. The content image selection unit 101 selects a content image by referring to the content image DB 108, and outputs the selected content image to the visual feature image generation unit 103. Step S11: The detection support device 100 selects a visual feature by the visual feature selection unit 102. The visual feature selection unit 102 selects a visual feature by referring to a visual feature DB 109, and outputs the selected visual feature to the visual feature image generation unit 103. Step S12: The detection support device 100 generates a visual feature image by the visual feature image generation unit 103. The visual feature image generation unit 103 generates a visual feature image by calculating the perceptual features selected in step S11 in the content image selected in step S10. The visual feature image generation unit 103 outputs the generated visual feature image to the content feature amount calculation unit 105.

[0049] Step S13: The detection support device 100 selects an image feature by the image feature selection unit 104. The image feature selection unit 104 selects an image feature by referring to the image feature DB 111, and outputs the selected image feature to the content feature amount calculation unit 105. Step S14: The detection support device 100 calculates a content feature in the content image by the content feature calculation unit 105. The content feature calculation unit 105 calculates the content feature by calculating the image feature selected in step S13 in the visual feature image generated in step S12. Step S15: Detection support device 100 selects an analysis method by analysis method selection unit 106. Analysis method selection unit 106 selects an analysis method by referring to analysis method DB 113, and outputs the selected analysis method to analysis unit 107. Step S16: The detection support device 100 performs analysis (generation of information for presenting feature amounts of a content image) by the analysis unit 107. The analysis unit 107 generates information indicating the feature amounts of the content image calculated in step S14 by the analysis method selected in step S15. The analysis unit 107 outputs the generated information to the analysis result storage unit 114 and the analysis result output unit 115.

[0050] Step S17: The detection support device 100 causes the analysis result storage unit 114 to store the analysis result (information for presenting the feature amount of the content image) obtained in step S16. Step S18: Detection support device 100 causes analysis result output unit 115 to output the analysis result (information for presenting the feature amount of the content image) analyzed in step S16 to a display unit (not shown) or the like.

[0051] 3 is a diagram showing an example of a reference image and an inspection image according to the first embodiment of the present invention, in which the reference image is shown on the left and the inspection image is shown on the right. As shown in Figure 3, for example, the reference image and the test image look so similar that they seem to be the same design. There seems to be almost no difference between the reference image and the test image. However, an expert can detect defects in the reference image and process the reference image to look like the test image based on the detected defects. In this embodiment, the detection support device 100 can visualize and present the difference in appearance between the reference image and the test image due to human visual perception as a difference in content features. In other words, the detection support device 100 can numerically indicate the degree to which the characteristics of the test image differ from the reference image. This helps an unskilled person to recognize the difference between the two images, which at first glance seem to have almost no difference.

[0052] Fig. 4 is a diagram showing an example of a visual feature image according to the first embodiment of the present invention, in which contrast is selected as the visual feature.

[0053] In Fig. 4, an example of a visual feature image in a reference image is shown on the left, and an example of a visual feature image in a test image is shown on the right, with three visual feature images shown for each scale for the reference image and the test image. The scale is an index showing the size of the space (image size) for calculating the visual feature (contrast in this example), with finer scale values ​​(smaller image size for calculating contrast) shown upwards and coarser scale values ​​(larger image size) shown downwards. The three visual feature images show the visual feature images for the reference image and the test image, with 1x1 showing the case where one reference image is connected vertically and horizontally (one image), 2x2 showing the case where two images are connected vertically and horizontally (four images), and 3x3 showing the case where three images are connected vertically and horizontally (nine images).

[0054] That is, the upper part of Fig. 4 shows the visual feature images of the reference image and the test image, from left to right, with image sizes of (1x1), (2x2), and (3x3) when the scale value is calculated as 8 (denoted as "scl8"). Similarly, the upper part of Fig. 4 shows the visual feature images of the reference image and the test image, from left to right, with image sizes of (1x1), (2x2), and (3x3) when the scale value is calculated as 16 (denoted as "scl16"). The upper part of Fig. 4 shows the visual feature images of the reference image and the test image, from left to right, with image sizes of (1x1), (2x2), and (3x3) when the scale value is calculated as 8 (denoted as "scl8").

[0055] As shown in Figure 4, when comparing the visual feature image generated from the reference image and the visual feature image generated from the test image, for example, when comparing the two images corresponding to (1 x 1) on scale 8 and the visual feature image corresponding to (1 x 1) on scale 16, there appear to be differences compared to when comparing the reference image and test image in Figure 3.

[0056] When a difference is found in the visual feature image, it indicates that there is a difference in the visual features (contrast in this example) between the reference image and the test image. In other words, by converting both images into visual feature images, it is possible to emphasize the difference in appearance caused by the visual features in both images.

[0057] Fig. 5A is a diagram showing the analysis result in the first embodiment of the present invention, Fig. 5A shows an example in which a line graph is selected as the analysis method.

[0058] In Fig. 5A, three line graphs are shown for each scale value. Each line graph shows the correlation difference of the visual feature images in the reference image and the test image in four directions. The four directions are 0[deg], 45[deg], 90[deg], and 135[deg] from a predetermined reference axis set in the image. The three line graphs show the correlation difference in the image sizes of (1x1), (2x2), and (3x3) from the left.

[0059] FIG. 5B shows a (2×2) correlation difference at scale 8 out of the multiple correlation differences shown in FIG. 5A. As shown in FIG. 5B, for example, at scale 8 (2×2), the correlation differences in the directions of 0 [deg] and 90 [deg] tend to exhibit larger values ​​than the correlation differences in other directions.

[0060] In places where a relatively large difference in visual features is shown, it is shown that there is a relatively large difference between the two images. In other words, by converting both images into visual feature images and showing the visual features, it is possible to quantitatively show the difference caused by the visual features in both images.

[0061] Fig. 5C is a diagram showing the analysis result in the first embodiment of the present invention, in which a radar chart is selected as the analysis method.

[0062] FIG. 5C shows three radar charts for each scale value. Each radar chart shows the content feature (in this example, the contrast of the visual feature image) in the reference image and the test image in eight directions. The eight directions are up, down, left, right, top right, top left, bottom left, and bottom right from the center of the image. The three radar charts show the correlation difference in image sizes of (1×1), (2×2), and (3×3) from the left.

[0063] As shown in FIG. 5C, for example, the contrast in all directions of the reference image at (1×1) on scale 32 tends to exhibit larger values ​​than the contrast of the test image.

[0064] In places where a relatively large difference is shown in the content feature amount, it is shown that there is a relatively large difference between the two images. In other words, by showing the content feature amount in both images, it is possible to quantitatively show the difference in appearance between the two images.

[0065] As described above, the detection support device 100 of the first embodiment includes a content image selection unit 101 (an example of a "content image acquisition unit") and a content feature amount calculation unit 105. The content image selection unit 101 acquires information about the content image. The content feature amount calculation unit 105 calculates the content feature amount by applying visual features to the content image. As a result, according to the detection support device 100 of the first embodiment, visual features that are features that can be recognized by human vision can be applied to the content image, and therefore a processing process based on human vision can be executed as processing on a processor.

[0066] Moreover, the detection support device 100 of the first embodiment may further include a visual feature image generating unit 103. The visual feature image generating unit 103 generates a visual feature image. The visual feature image is an image in which a visual feature amount, which is the degree of visual feature for each pixel in a content image, is associated with the pixel. As a result, according to the detection support device 100 of the first embodiment, it is possible to display visual features in association with pixels of the content image, and it is possible to clearly show which part of the content image has what kind of visual feature amount.

[0067] In addition, in the detection support device 100 of the first embodiment, the visual feature image generating unit 103 generates a visual feature image using at least one selected from luminance, chromaticity, contrast, edge, optical flow, and skewness as a visual feature. As a result, the detection support device 100 of the first embodiment can generate a visual feature image with higher accuracy by using a technique similar to visual appearance among existing image processing techniques.

[0068] Moreover, in the detection support device 100 of the first embodiment, the visual feature image generating unit 103 generates a visual feature image by using a recognition index that indicates the ease of recognition of a content image by the human eye as a visual feature. As a result, the detection support device 100 of the first embodiment can generate a visual feature image that is closer to recognition by the human eye.

[0069] In addition, in the detection support device 100 of the first embodiment, the recognition index includes at least one of a visual attention model, a gaze prediction model, a saliency model, and a saliency model. As a result, the detection support device 100 of the first embodiment can generate a visual feature image with higher accuracy by using an existing model.

[0070] Furthermore, in the detection support device 100 of the first embodiment, the content feature amount calculation unit 105 calculates the content feature amount by applying the image feature to the visual feature image. As a result, according to the detection support device 100 of the first embodiment, the visual feature image can be subjected to an image processing technique to statistically process the degree of the feature indicated by the visual feature, and the content feature amount can be indicated more quantitatively.

[0071] As the understanding of human brain functions progresses, many features that can be recognized visually are being discovered, and these features may be included in the visual features.

[0072] <Second embodiment> The second embodiment will be described below with reference to the drawings. The detection support device 100A of this embodiment differs from the above-mentioned embodiment in that it presents a pseudo line of sight when an expert detects a defect in a content image. The detection support device 100A presents the line of sight of an expert, thereby supporting the detection of defects, so that even an unskilled person can easily detect defects. In this embodiment, only the configuration different from the first embodiment will be described, and the same reference numerals will be used for the configuration similar to that of FIG. 1 according to the first embodiment, and the description thereof will be omitted unless particularly necessary.

[0073] 6 is a block diagram showing an example of the configuration of a detection support device 100A according to the second embodiment. The detection support device 100A includes, for example, a gaze feature learning model selection unit 116, a gaze feature image generation unit 117, a gaze feature learning model DB 118, and a content feature amount calculation unit 105A. The detection support device 100A does not include a visual feature selection unit 102 and a visual feature image generation unit 103.

[0074] In the following, in this embodiment, the content image will be described as a still image, but similar to the first embodiment, the present invention may be applied to other content images such as moving images and video.

[0075] The gaze feature learning model selection unit 116 selects a gaze feature learning model. The gaze feature learning model is a model that estimates gaze features in a content image, generated by a machine learning technique. The gaze feature is information indicating features related to the gaze when an expert detects a defect in a content image, and is, for example, a gaze feature image described later, or information statistically indicating the degree of visibility for each area of ​​the content image.

[0076] The gaze characteristic learning model is generated, for example, by performing machine learning using learning data in which each of a plurality of different learning content images (learning content images) is associated with the gaze characteristic performance of an expert who viewed each learning content image.

[0077] The means for acquiring the gaze feature may be a commercially available dedicated gaze measurement device (for example, Tobii Pro Glasse2, a body-worn measurement device, or Tobii Pro X2, X3, a stationary measurement device, etc.) or a calculation method using a combination of a consumer camera and a gaze estimation method. The gaze feature is generally stored in chronological order as the coordinates of the viewpoint measured at a certain sampling time, and these are converted into coordinates on the image for use.

[0078] The machine learning method used here may be any method, but is performed using an estimation model such as a deep neural network. The deep neural network has, for example, an input layer, an output layer, and a configuration in which the input layer, output layer, and the intermediate layers are connected by a multi-layered convolution layer and a pooling layer. Then, when a learning content image is input to the input layer of the multi-layered neural network, learning is repeated so that information output from the output layer of the multi-layered neural network becomes a gaze feature associated with the learning content image, and a coupling coefficient and a bias value for coupling each layer are determined. A gaze feature learning model is generated by determining the coupling coefficient and bias value of the estimation model.

[0079] The gaze feature learning model selection unit 116 selects a gaze feature learning model selected by a selection operation by a user or the like from a plurality of gaze feature learning models stored in the gaze feature learning model DB 118. The gaze feature learning model selection unit 116 outputs the acquired gaze feature learning model to the gaze feature image generation unit 117.

[0080] The gaze feature image generating unit 117 generates a gaze feature image by applying the gaze feature estimated by the gaze feature learning model to the content image. The gaze feature image is an image that indicates the degree of the gaze feature in the content image. As a method for obtaining the gaze feature image from the gaze feature, for example, a method is used in which the accumulation of viewpoints on an image within a measurement time is approximated as a probability distribution to output a heat map.

[0081] The gaze feature image is an image in which gaze features are condensed into information in one image. Here, as a method of condensing information, for example, there is a method of counting gaze distribution for each pixel in a content image to construct a two-dimensional histogram, and approximately expressing the peak of the histogram using a two-dimensional normal distribution and expressing it as a real value in the range of intensity [0,1] (generally called a heat map).

[0082] The content image used by the gaze feature image generation unit 117 is an image selected by the content image selection unit 101. The gaze feature used by the gaze feature image generation unit 117 is estimated by inputting the content image selected by the content image selection unit 101 to the gaze feature learning model selected by the gaze feature learning model selection unit 116. The gaze feature image generation unit 117 outputs the generated gaze feature image to the content feature amount calculation unit 105A.

[0083] The gaze feature learning model DB118 stores the gaze feature learning model. For example, the gaze feature learning model associated with identification information that uniquely indicates the gaze feature learning model is stored in the gaze feature learning model DB118. The gaze feature learning model is generated, for example, by an external learning server, acquired by the detection support device 100 via an external input device or an input means, and stored in the analysis method DB113. The gaze feature learning model DB118 may store a model according to the type of content image. Thereby, even if the expert's way of looking at the content image differs depending on the type of content image, a model according to the type can be selected, and the gaze feature can be estimated with higher accuracy.

[0084] The content feature amount calculation unit 105A calculates a content feature amount by applying an image feature to the gaze feature image. The content feature amount in this embodiment is, for example, a statistical amount that statistically indicates image features in the gaze feature image.

[0085] For example, when contrast is selected as the image feature, the content feature amount is a value indicating what kind of contrast the expert's gaze forms in the content image. The gaze feature image used here is an image generated by the gaze feature image generating unit 117. The image feature used here is an image feature selected by the image feature selecting unit 104.

[0086] The content feature amount may be the degree of the gaze feature, in which case the content feature amount is the gaze feature image itself.

[0087] Fig. 7 is a flowchart showing an example of the operation of the process performed by the detection support device 100A according to the present embodiment. The processes shown in steps S23 and S25 to S28 in Fig. 7 are similar to the processes shown in steps S13 and S15 to S18 in Fig. 2, and therefore the description thereof will be omitted.

[0088] Step S20: The detection support device 100A outputs the content image selected by the content image selection unit 101 to the gaze characteristic image generation unit 117. Step S21: Detection support device 100A selects a gaze feature learning model by gaze feature learning model selection unit 116. Gaze feature learning model selection unit 116 selects a gaze feature learning model by referring to gaze feature learning model DB 118, and outputs the selected gaze feature learning model to gaze feature image generation unit 117. Step S22: Detection support device 100A generates a gaze feature image by gaze feature image generation unit 117. Gaze feature image generation unit 117 generates a gaze feature image by estimating gaze features in the content image selected in step S20 using the gaze feature learning model selected in step S21. Gaze feature image generation unit 117 outputs the generated gaze feature image to content feature amount calculation unit 105A. Step S24: Detection support device 100A calculates a content feature amount in the content image by content feature amount calculation unit 105A. Content feature amount calculation unit 105A calculates the content feature amount by calculating the image feature selected in step S23 in the gaze feature image generated in step S22.

[0089] As described above, in the detection support device 100A of the second embodiment, the content feature amount calculation unit 105A calculates the content feature amount by applying the gaze feature indicating the feature of the gaze of the person viewing the content image to the content image. As a result, the detection support device 100A of the second embodiment has the same effect as the above-mentioned effect.

[0090] Moreover, the detection support device 100A of the second embodiment further includes a gaze feature image generation unit 117. The gaze feature image generation unit 117 generates a gaze feature image in which the gaze feature of each pixel in a content image corresponds to the position coordinates of the pixel. The content feature amount calculation unit 105A calculates the content feature amount using the gaze feature image. As a result, the detection support device 100A of the second embodiment has the same effects as those described above.

[0091] In addition, in the detection support device 100A of the second embodiment, the gaze feature is estimated using a gaze feature learning model generated by executing machine learning using learning data in which a content image is associated with the gaze feature performance in the content image. As a result, the detection support device 100A of the second embodiment can estimate the gaze feature of an expert in a content image with higher accuracy based on the past performance.

[0092] 8 is a block diagram showing an example of the configuration of a learning device 200 according to an embodiment. The learning device 200 is a device that generates a gaze feature learning model. The learning device 200 includes, for example, a learning content image acquisition unit 201, a gaze information acquisition unit 202, a learning gaze characteristic image generation unit 203, a deep learning unit 204, a learning content image DB 205, a gaze information memory unit 206, a gaze characteristic image memory unit 207, and a gaze characteristic learning model DB 208.

[0093] The learning content image acquisition unit 201 acquires a learning content image. The learning content image is learning data used when performing machine learning on an estimation model, and is information to be input (set) to the input layer of the estimation model.

[0094] The learning content image acquisition unit 201 acquires a set of learning content images selected by a user or the like according to the amount of learning from a plurality of learning content images stored in the learning content image DB 205. The method of selection by the user or the like may be any method. The learning content image acquisition unit 201 outputs the acquired learning content images to the learning gaze characteristic image generation unit 203.

[0095] The gaze information acquisition unit 202 acquires gaze information (gaze feature). The gaze information is information about the gaze of an expert with respect to a learning content image, for example, information indicating a time series change in the gaze of an expert visually viewing the learning content image. The gaze information acquisition unit 202 selects gaze information corresponding to the learning content image from a plurality of visual features stored in the gaze information storage unit 206, for example. The gaze information acquisition unit 202 outputs the acquired gaze information to the learning gaze feature image generation unit 203.

[0096] The learning gaze characteristic image generating unit 203 generates a learning gaze characteristic image by applying gaze information to the learning content image. The method of generating the learning gaze characteristic image is similar to the method of generating the gaze characteristic image by the gaze characteristic image generating unit 117, and therefore the description thereof will be omitted. The learning gaze characteristic image generating unit 203 outputs the generated learning gaze characteristic image to the deep learning unit 204 and stores it in the gaze characteristic image storage unit 207.

[0097] The deep learning unit 204 generates a gaze feature learning model by performing learning (deep learning) using the learning gaze feature image as learning data. The deep learning unit 204 stores the generated gaze feature learning model in the gaze feature learning model DB 208.

[0098] Fig. 9 is a flowchart showing an example of the operation of the processing performed by the learning device 200 according to this embodiment. Fig. 9 shows the flow of the operation of the processing for generating a gaze feature learning model by deep learning. Step S50: The study device 200 acquires study content images by the study content image acquisition unit 201. It is preferable that the study content image acquisition unit 201 acquires a large number of study content images. This is because, in general, a good study effect can be obtained by preparing a large number of study data (study content images) with a wide variety of images. Step S51: The study device 200 acquires, by the line-of-sight information acquisition unit 202, line-of-sight information corresponding to the study content image. Step S52: The learning device 200 generates a learning gaze characteristic image by using the learning gaze characteristic image generating unit 203 to apply the gaze information acquired in step S51 to the learning content image acquired in step S50. Step S53: Learning device 200 determines whether or not learning gaze characteristic images have been generated for all of the learning content images acquired in step S50. If learning device 200 has generated learning gaze characteristic images for all of the learning content images, it executes the process shown in step S54. If learning device 200 has not generated learning gaze characteristic images for all of the learning content images, it returns to the process shown in step S51. Step S54: The learning device 200 generates a gaze feature learning model by performing deep learning using the learning gaze feature image as learning data.

[0099] Fig. 10 is a flowchart showing an example of the operation of the process performed by the learning device 200 according to this embodiment. Fig. 10 shows the flow of the operation of the process of generating a new learning model by performing transfer learning on a gaze feature learning model by deep learning (hereinafter, also simply referred to as a learning model). Step S60: The learning device 200 configures the input layer and output layer of the estimation model by the deep learning unit 204. The estimation model is a deep learning model with a multi-layer structure of intermediate layers (pooling layer and convolution layer). Information on each pixel in the learning content image is input to the input layer. The output layer is a fully connected layer that performs normalization. This output layer is configured to output a decimal value between "1" and "0". Step S61: The deep learning unit 204 determines whether to generate a new learning model using deep learning, or to generate a new learning model by transfer learning using an existing general-purpose learning model. The deep learning unit 204 performs such a determination, for example, when the detection support device 100A selects a learning model.

[0100] For example, consider a case where the deep learning unit 204 generates a gaze feature learning model in a situation where a large amount of learning content images can be prepared. In this case, gaze information of an expert is acquired for each learning content image, and a set of correct answer contents (learning data) is generated. After that, the deep learning unit 204 trains a deep learning model (estimation model) by machine learning using the set of learning content images and the set of correct answer contents, that is, generates a gaze feature learning model by new learning. On the other hand, when the deep learning unit 204 generates a gaze feature learning model in a situation where a large amount of learning content images cannot be prepared, the deep learning unit 204 generates the gaze feature learning model by transfer learning of a gaze feature learning model corresponding to another learning content image that has already been generated by deep learning. Note that whether or not a large amount of learning content images can be prepared is determined according to, for example, the number of learning content images stored in the learning content image DB 205 or a selection operation by the user. When generating a gaze feature learning model by new learning, the deep learning unit 204 executes the process shown in step S65. When generating a gaze feature learning model by transfer learning, the learning device 200 executes the process shown in step S62.

[0101] Step S62: The deep learning unit 204 selects a predetermined learning model from among the learning models stored in the gaze feature learning model DB 208. For example, for a set of learning content images selected by a user, the deep learning unit 204 selects a learning model that has been trained for another set of learning content images. The deep learning unit 204 acquires the selected learning model as a deep learning model to be used for transfer learning. Step S63: The deep learning unit 204 extracts, from the deep learning model read in step S62 for use in transfer learning, the layers from the input layer to the intermediate layer (adaptation layer) designated by the user or in advance, as a transfer learning model. Then, the deep learning unit 204 extracts the intermediate layers subsequent to the adaptation layer from the deep learning model, connects them to the adaptation layer of the transfer learning model, and connects the output layer, thereby configuring a deep learning model for transfer learning. Step S64: The deep learning unit 204 performs optimization processing of the weight parameters of the layers of each network so that, when a pixel determined to have a high degree of attention based on gaze information of an expert in a learning content image is input to the input layer of the learning target model (the above deep learning model for transfer learning or the above deep learning model), a numerical value close to "1" indicating a high degree of attention is output from the output layer. In addition, the deep learning unit 204 performs optimization processing so that, when a pixel determined to have a low degree of attention of an expert in a learning content image is input to the input layer of the learning target model, a numerical value close to "0" indicating a low degree of attention is output from the output layer. In other words, the deep learning unit 204 performs machine learning of class classification on the learning content image, and generates a gaze feature image as a learning result.

[0102] At this time, the deep learning unit 204 may input a set of learning content images different from the learning content and a set of correct answer data which is gaze information of an expert on those images to the generated learning model, and perform a learning test on the generated learning model. In this case, when the deep learning unit 204 inputs a collection of learning content images into the learning model, if the numerical value output from the output layer is equal to or greater than a predetermined first threshold and the numerical value output from the output layer is equal to or less than a predetermined second threshold, the deep learning unit 204 stores this learning model in the gaze feature learning model DB 208 and sets it as the gaze feature learning model. On the other hand, if, in the above learning test, the value output by the output layer of the learning model for a pixel on which the expert's gaze is focused is less than a preset first threshold, or the value output by the output layer of the model to be learned for a pixel on which the inspector's gaze is not likely to be focused is equal to or greater than a preset second threshold, the deep learning unit 204 does not store this learning model in the gaze feature learning model DB 208, and re-learns the learning model.

[0103] Step S65: From the learning model generated in step S64, the deep learning unit 204 extracts, as parameters of the learning model, the output parameters of the pooling layer and convolution layer in the intermediate layer of the multi-layer structure, the type of activation function and the output parameters, etc.

[0104] Step S66: The deep learning unit 204 stores the generated learning model and the extracted learning model parameters in the gaze feature learning model DB 208 (registration process).

[0105] <Third embodiment> Next, a third embodiment will be described. This embodiment differs from the above-mentioned embodiment in that a content feature amount is calculated using visual features and gaze features. As a result, the detection support device 100B of this embodiment can perform processing similar to information processing of human visual perception and use human gaze information, and can present more detailed information about how the content image appears. In this embodiment, only configurations different from the above-mentioned embodiment will be described, and configurations similar to those of the above-mentioned embodiment will be given the same reference numerals, and their description will be omitted unless particularly necessary.

[0106] 11 is a block diagram showing an example of the configuration of a detection support device 100B according to the third embodiment. The detection support device 100B includes, for example, a visual feature / gaze feature calculation unit 119 and a content feature amount calculation unit 105B.

[0107] In the following, in this embodiment, the content image will be described as a still image, but similar to the first embodiment, the present invention may be applied to other content images such as moving images and video.

[0108] The visual feature gaze feature calculation unit 119 calculates the visual feature gaze feature by using the visual feature image and the gaze feature image. The visual feature gaze feature is information indicating the degree of both the visual feature and the gaze feature. The visual feature gaze feature calculation unit 119 calculates the visual feature gaze feature by performing an operation between the visual feature and the gaze feature, for example. The operation here includes various logical operations such as logical AND, logical OR, and exclusive OR (XOR) between the visual feature and the gaze feature, winners take all operation, bit operation, arithmetic operation, and the like.

[0109] When performing a calculation between a visual feature and a gaze feature, visual feature / gaze feature calculation section 119 may perform weighting for each feature or for each pixel.

[0110] The visual feature image used by visual feature image calculation unit 119 is an image generated by visual feature image generation unit 103. The visual feature image used by visual feature image calculation unit 119 is an image generated by gaze feature image generation unit 117. Visual feature gaze feature calculation unit 119 outputs the generated visual feature gaze feature to content feature amount calculation unit 105A.

[0111] The content feature amount calculation unit 105B calculates the content feature amount by applying the image feature to the visual feature / gaze feature. The content feature amount in this embodiment is, for example, a statistical amount that statistically indicates the features on the image in the visual feature / gaze feature.

[0112] For example, when contrast is selected as an image feature, the content feature amount is a value indicating what kind of contrast the visual feature / gaze feature forms in the content image. The visual feature / gaze feature used here is information generated by visual feature / gaze feature calculation unit 119. The image feature used here is an image feature selected by image feature selection unit 104. The content feature amount may be the visual feature or the gaze feature itself.

[0113] Fig. 12 is a flowchart showing an example of the operation of the process performed by the detection support device 100B according to this embodiment. The processes shown in steps S31, S32, S36, and S38 to S41 in Fig. 12 are similar to the processes shown in steps S11, S12, S13, and S15 to S18 in Fig. 2, respectively, and therefore their description will be omitted. In addition, the processes shown in steps S33 and S34 in Fig. 12 are similar to the processes shown in steps S21 and S22 in Fig. 2, and therefore their description will be omitted.

[0114] Step S30: The detection support device 100B outputs the content image acquired by the content image selection unit 101 to the visual feature image generation unit 103 and the gaze feature image generation unit 117. Step S35: The detection support device 100B calculates a visual feature gaze feature using the visual feature image generated in S32 and the visual feature image generated in step S34, using the visual feature gaze feature calculation unit 119, and outputs the calculated visual feature gaze feature to the content feature calculation unit 105B. Step S37: Detection support device 100B causes content feature amount calculation unit 105B to calculate content feature amounts by applying the image features to the visual feature / gaze feature calculated in step S35.

[0115] As described above, the detection support device 100B of the third embodiment includes the visual feature gaze feature calculation unit 119. The visual feature gaze feature calculation unit 119 calculates the visual feature gaze feature by using the visual feature image and the gaze feature image. As a result, the detection support device 100B of the third embodiment can perform processing similar to information processing of human visual perception and utilize human gaze information, and can present more detailed information about how the content image appears.

[0116] In addition, a program for realizing all or part of the functions of the detection support device 100 (100A, 100B) of the present invention may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Additionally, "computer system" also includes WWW systems equipped with a homepage providing environment (or display environment). Additionally, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" also includes devices that hold programs for a certain period of time, such as volatile memory (RAM) within computer systems that act as servers or clients when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0117] The above program may also be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above program may also be for realizing part of the above-mentioned functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0118] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0119] 100, 100A, 100B...Detection support device 101…Content image selection section 102...Visual feature selection unit 103...Visual feature image generation unit 104...Image feature selection unit 105, 105A, 105B...Content feature amount calculation unit 106...Analysis method selection section 107…Analysis Department 108…Content Image DB 109...Visual feature DB 110...visual feature image storage unit 111...Image feature DB 112...Content feature amount storage unit 113…Analysis method DB 114…Analysis result storage unit 115...Analysis result output section 116…Gaze feature learning model selection unit 117...Gaze feature image generation unit 118…Gaze feature learning model DB 119...Visual feature and gaze feature calculation unit

Claims

1. A detection support device that supports detection of the presence or absence of an unexpected pattern due to continuity of a pattern in a content image generated by repeatedly arranging a predetermined pattern, the detection support device comprising: a content image acquisition unit for acquiring the content image; a gaze feature image generating unit that estimates gaze features that are characteristics of a gaze when detecting the pattern from the content image using a gaze feature learning model, and generates a gaze feature image that indicates the estimated gaze features for each pixel in the content image; an analysis result output unit that displays the gaze feature image; Equipped with The gaze feature learning model is a trained model that has learned a correspondence between the content image for learning and the gaze feature of a person who detected the pattern from the content image for learning. Detection aids.

2. a visual feature image generating unit for generating a visual feature image representing a visual feature, which is a feature of brightness or color, from the content image; a visual feature gaze feature calculation unit that calculates the visual feature and the gaze feature using the visual feature image and the gaze feature image, and calculates a result of the calculation as a visual feature gaze feature; Further equipped with The analysis result output unit outputs the visual feature / gaze feature as a gaze and visual feature when detecting the pattern in the content image. The detection assistance device according to claim 1 .

3. the visual feature image generating unit generates the visual feature image by further using a recognition index representing an ease of recognition of the content image by a human eye as the visual feature. The detection assistance device according to claim 2 .

4. The recognition indices include at least one of a visual attention model, a gaze prediction model, a saliency model, and a saliency model. The detection assistance device according to claim 3 .

5. A learning device that generates a gaze feature learning model to be used by a detection support device that supports detection of the presence or absence of an unexpected pattern due to the continuity of a pattern in a content image generated by repeatedly arranging a predetermined pattern, the device comprising: the gaze feature learning model is a trained model that has learned a correspondence relationship between the content image for learning and a gaze feature that is a feature of the gaze of a person who detected the pattern from the content image for learning, a deep learning unit configured to be able to select either deep learning or transfer learning using an existing learning model as a method for generating the gaze feature learning model, and to generate the gaze feature learning model using the selected method; A learning device comprising:

6. a learning content image database for storing the learning content image; The deep learning unit determines whether to select deep learning or transfer learning using an existing learning model depending on the number of the content images for learning stored in the learning content image database. The learning device according to claim 5 .

7. A detection support method performed by a computer, which is a detection support device, for supporting detection of the presence or absence of an unexpected pattern due to the continuity of a pattern in a content image generated by repeatedly arranging a predetermined pattern, the method comprising: A content image acquisition unit acquires the content image, a gaze feature image generating unit that estimates gaze features that are characteristics of a gaze when detecting the pattern from the content image using a gaze feature learning model, and generates a gaze feature image that indicates the estimated gaze features for each pixel in the content image; an analysis result output unit displays the gaze feature image; The gaze feature learning model is a trained model that has learned a correspondence between the content image for learning and the gaze feature of a person who detected the pattern from the content image for learning. Detection aid methods.

8. A program for causing a computer to function as the detection support device according to claim 1.

9. A learning method performed by a computer, which is a learning device, for generating a gaze feature learning model used by a detection support device that supports detection of the presence or absence of an unexpected pattern due to the continuity of a pattern in a content image generated by repeatedly arranging a predetermined pattern, the method comprising: the gaze feature learning model is a trained model that has learned a correspondence relationship between the content image for learning and a gaze feature that is a feature of the gaze of a person who detected the pattern from the content image for learning, The deep learning unit is configured to be able to select either deep learning or transfer learning using an existing learning model as a method for generating the gaze feature learning model, and generates the gaze feature learning model using the selected method. How to learn.

10. A program for causing a computer to function as the learning device according to claim 5.

Images