Estimation device, learning device, estimation method, trained model generating method, estimation program, and learning program

Abstract

To facilitate enhancement of the accuracy of visual inspections of tire inner surfaces.SOLUTION: A data processing device 1 includes: a target tire image acquisition unit 101 that acquires a tire image showing a first texture and a second texture appearing on an inner surface of a tire; a target texture area identification unit 102 that identifies positions of a first texture area and a second texture area; a target partial image generation unit 103 that generates a plurality of partial images; a target partial image selection unit 104 that selects a plurality of first partial images and a plurality of second partial images from the plurality of partial images; an estimation unit 105 that estimates appearance quality of the inner surface of the tire visible in each of the plurality of first partial images based on respective outputs of when each of the plurality of first partial images is input to a trained first machine learning model 106, and that also estimates appearance quality of the inner surface of the tire visible in each of the plurality of second partial images based on respective outputs of when each of the plurality of second partial images is input to a trained second machine learning model 107.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an estimation device, a learning device, an estimation method, a method for generating a trained model, an estimation program, and a learning program, and in particular to a technique for estimating and learning whether a tire's appearance is good or bad in tire appearance inspection. [Background technology]

[0002] Conventionally, there has been known a technology for performing visual inspection of tires using machine learning. For example, Patent Document 1 describes a device that generates a plurality of partial images ("cut-out inspection images") from a tire image showing the tire inner surface, and uses these partial images as input for machine learning to estimate and learn whether the appearance of the tire inner surface is good or bad. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-027826 Summary of the Invention [Problem to be solved by the invention]

[0004] However, tire images showing the inside of a tire contain multiple types of patterns transferred from the bladder during the tire manufacturing process, and the multiple partial images generated from these tire images have variations depending on the types of patterns captured. For this reason, it was difficult to improve inspection accuracy using conventional technology, which inputs these partial images into a machine learning model without distinguishing between them.

[0005] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide an estimation device, a learning device, an estimation method, a method for generating a trained model, an estimation program, and a learning program that can easily improve the accuracy of visual inspection of the inner surface of a tire. [Means for solving the problem]

[0006] (1) The estimation device according to the present invention includes a tire image acquisition means for acquiring a tire image showing a first texture and a second texture different from the first texture that appear on the inner surface of the tire, a texture area identification means for identifying the position of a first texture area in which the first texture appears and the position of a second texture area in which the second texture appears, among areas in the tire image, a partial image generation means for generating a plurality of partial images of the same size that show different parts of the tire image, and a tire image estimation means for estimating the first texture from among the plurality of partial images based on the position of the first texture area and the position of a part of the tire image shown by each of the plurality of partial images. and a partial image selection means for selecting a plurality of first partial images that show the second texture, and selecting a plurality of second partial images from the plurality of partial images that show the second texture, based on the position of the second texture area and the position of the portion of the tire image indicated by each of the plurality of partial images; and an estimation means for estimating the quality of the appearance of the tire inner surface shown in each of the first partial images based on the output when each of the plurality of first partial images is input into a trained first machine learning model, and for estimating the quality of the appearance of the tire inner surface shown in each of the second partial images based on the output when each of the plurality of second partial images is input into a trained second machine learning model.

[0007] (2) In the estimation device of (1), the second texture may not appear in any of the plurality of first partial images, and the first texture may not appear in any of the plurality of second partial images.

[0008] (3) In the estimation device of (2), each of the plurality of partial images may overlap with a part of another partial image adjacent to the partial image.

[0009] (4) In the estimation device of (1) to (3), the texture area identification means may perform frequency analysis on the tire image, extract a unique pattern in the tire image having a predetermined frequency characteristic unique to the first texture, and identify the first texture area based on the unique pattern.

[0010] (5) A learning device according to the present invention includes a tire image acquisition means for acquiring a tire image showing a first texture and a second texture different from the first texture that appear on the inner surface of the tire; a texture area identification means for identifying the position of a first texture area in which the first texture appears and the position of a second texture area in which the second texture appears among the areas in the tire image; a partial image generation means for generating a plurality of partial images of the same size that show different parts of the tire image; a partial image selection means for selecting a plurality of first partial images that show the first texture from the plurality of partial images based on the position of the first texture area and the position of a portion of the tire image indicated by each of the plurality of partial images, and selecting a plurality of second partial images that show the second texture from the plurality of partial images based on the position of the second texture area and the position of a portion of the tire image indicated by each of the plurality of partial images; and a learning means for performing learning of a first machine learning model based on the plurality of first partial images and performing learning of a second machine learning model based on the plurality of second partial images.

[0011] (6) In the learning device of (5), the second texture may not appear in each of the plurality of first partial images, and the first texture may not appear in each of the plurality of second partial images.

[0012] (7) In the learning device of (6), each of the plurality of partial images may overlap with a part of another partial image adjacent to the partial image.

[0013] (8) In the learning device of (7), the first machine learning model and the second machine learning model may be unsupervised machine learning models.

[0014] (9) In the learning device of (5) to (8), the texture area identification means may perform frequency analysis on the tire image, extract a unique pattern in the tire image having a predetermined frequency characteristic unique to the first texture, and identify the first texture area based on the unique pattern.

[0015] (10) The estimation method according to the present invention includes a tire image acquisition step of acquiring a tire image showing a first texture and a second texture different from the first texture that appear on the inner surface of the tire; a texture area identification step of identifying the position of a first texture area in which the first texture appears and the position of a second texture area in which the second texture appears, among areas in the tire image; a partial image generation step of generating a plurality of partial images of the same size that show different parts of the tire image; and a partial image generation step of generating a tire image showing the first texture and a second texture that appear on the inner surface of the tire based on the position of the first texture area and the position of a part of the tire image shown by each of the plurality of partial images. The method includes a partial image selection step of selecting a plurality of first partial images that show the second texture, and selecting a plurality of second partial images that show the second texture from among the plurality of partial images based on the position of the second texture area and the position of the portion of the tire image indicated by each of the plurality of partial images; and an estimation step of estimating the quality of the appearance of the tire inner surface shown in each of the first partial images based on the output when each of the plurality of first partial images is input into a trained first machine learning model, and estimating the quality of the appearance of the tire inner surface shown in each of the second partial images based on the output when each of the plurality of second partial images is input into a trained second machine learning model.

[0016] (11) A method for generating a trained model according to the present invention includes a tire image acquisition step for acquiring a tire image showing a first texture and a second texture different from the first texture that appear on the inner surface of a tire; a texture area identification step for identifying the position of a first texture area in which the first texture appears and the position of a second texture area in which the second texture appears, among areas in the tire image; a partial image generation step for generating a plurality of partial images of the same size that show different parts of the tire image; a partial image selection step for selecting a plurality of first partial images that show the first texture from the plurality of partial images based on the position of the first texture area and the position of a portion of the tire image indicated by each of the plurality of partial images, and selecting a plurality of second partial images that show the second texture from the plurality of partial images based on the position of the second texture area and the position of a portion of the tire image indicated by each of the plurality of partial images; and a learning step for performing training of a first machine learning model based on the plurality of first partial images and performing training of a second machine learning model based on the plurality of second partial images.

[0017] (12) An estimation program according to the present invention includes a tire image acquisition means for acquiring a tire image showing a first texture and a second texture different from the first texture that appear on an inner surface of a tire, a texture area identification means for identifying the position of a first texture area in which the first texture appears and the position of a second texture area in which the second texture appears, among areas in the tire image, a partial image generation means for generating a plurality of partial images of the same size that show different parts of the tire image, and a partial image generation means for generating a plurality of partial images showing the first texture that appear among the plurality of partial images based on the position of the first texture area and the position of a part of the tire image that each of the plurality of partial images shows. The program causes a computer to function as a partial image selection means for selecting a number of first partial images and selecting a plurality of second partial images from the plurality of partial images in which the second texture appears based on the position of the second texture region and the position of a portion of the tire image indicated by each of the plurality of partial images, and an estimation means for estimating the quality of the appearance of the tire inner surface appearing in each of the first partial images based on the output when each of the plurality of first partial images is input to a trained first machine learning model, and for estimating the quality of the appearance of the tire inner surface appearing in each of the second partial images based on the output when each of the plurality of second partial images is input to a trained second machine learning model. This program may be stored in a computer-readable information storage medium such as a magneto-optical disk or a semiconductor memory.

[0018] (13) A learning program according to the present invention causes a computer to function as: a tire image acquisition unit that acquires a tire image showing a first texture and a second texture different from the first texture, the second texture being displayed on the tire inner surface; a texture region identification unit that identifies, among regions in the tire image, the position of a first texture region where the first texture is displayed and the position of a second texture region where the second texture is displayed; a partial image generation unit that generates a plurality of partial images of the same size that show different portions of the tire image; a partial image selection unit that selects, from the plurality of partial images, a plurality of first partial images that show the first texture based on the position of the first texture region and the position of a portion of the tire image represented by each of the plurality of partial images, and selects, from the plurality of partial images, a plurality of second partial images that show the second texture based on the position of the second texture region and the position of a portion of the tire image represented by each of the plurality of partial images; and a learning unit that executes training of a first machine learning model based on the plurality of first partial images and executes training of a second machine learning model based on the plurality of second partial images. This program may be stored on a computer-readable information storage medium such as a magneto-optical disk or a semiconductor memory. [Effects of the Invention]

[0019] According to the present invention, it is possible to easily improve the accuracy of visual inspection of the inner surface of a tire. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a block diagram showing a hardware configuration of a data processing device according to an embodiment of the present invention; [Figure 2] FIG. 1 is a diagram showing a circumferential cross section of a tire. [Figure 3] FIG. 2 is a functional block diagram showing an example of functions implemented by a data processing device according to an embodiment of the present invention. [Figure 4] 1A and 1B are diagrams illustrating an example of a tire image and an example of processing performed on the tire image according to an embodiment of the present invention. [Figure 5]10A and 10B are diagrams illustrating an example of a process for generating a plurality of partial images from a tire image and a process for these partial images according to an embodiment of the present invention. [Figure 6] 10A and 10B are diagrams illustrating an example of a tire image according to a modified example of the present invention and a process for the tire image. DETAILED DESCRIPTION OF THE INVENTION

[0021] [1. Embodiment] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0022] [1-1. Hardware configuration for realizing data processing according to one embodiment of the present invention] 1 is a block diagram showing the hardware configuration of a data processing device according to one embodiment of the present invention. As shown in FIG. 1, the data processing device 1 includes a processing device 11, a storage device 12, a display device 13, and an input device 14.

[0023] The processing device 11 includes, for example, a CPU, a GPU, etc., and operates according to a program stored in the storage device 12. As the processing device 11, an FPGA may be used.

[0024] The storage device 12 may be, for example, a ROM, RAM, HDD, SSD, etc., and stores programs executed by the processing device 11, various data to be processed, etc. The storage device 12 includes a main storage device and an auxiliary storage device. For example, the main storage device is a volatile memory such as RAM, and the auxiliary storage device is a non-volatile memory such as ROM, EEPROM, flash memory, or hard disk.

[0025] The display device 13 is, for example, a liquid crystal display or an organic EL display, and displays data stored in the storage device 12, processing results of the processing device 11, and the like.

[0026] The input device 14 is a user interface such as a keyboard or a mouse, and receives an operation input from the operator and inputs a signal indicating the content of the operation input to the processing device 11.

[0027] The data processing device 1 having the above configuration is used for visual inspection of the inner surface of a tire. As the processing device 11 and storage device 12, one or more personal computers, server computers, etc. can be used.

[0028] Fig. 2 is a diagram showing a circumferential cross section of a tire. As shown in Fig. 2, the tire inner surface is a surface of the tire 2 that is in an area 20 in a recessed portion of the tire 2. The tire inner surface can also be said to be a surface of the tire 2 that faces the tire cavity region. The tire cavity region refers to a spatial region surrounded by the tire 2 and the rim (not shown) that is filled with air when the tire 2 mounted on the rim is filled with air. Generally, the tire inner surface is a surface on which an inner liner is provided.

[0029] [1-2. Functions realized by a data processing device according to an embodiment of the present invention] 3 is a functional block diagram showing an example of functions implemented by a data processing device according to an embodiment of the present invention. As shown in FIG. 3, the data processing device 1 includes a target tire image acquisition unit 101, a target texture region identification unit 102, a target partial image generation unit 103, a target partial image selection unit 104, an estimation unit 105, a first machine learning model 106, a second machine learning model 107, a training tire image acquisition unit 108, a training texture region identification unit 109, a training partial image generation unit 110, a training partial image selection unit 111, and a training unit 112. These functions are implemented by the processing device 11 executing a program stored in the storage device 12.

[0030] In addition, the functions of the target tire image acquisition unit 101, target texture area identification unit 102, target partial image generation unit 103, target partial image selection unit 104, estimation unit 105, first machine learning model 106, and second machine learning model 107 in the data processing device 1 correspond to the functions of an estimation device.

[0031] In addition, the functions of the first machine learning model 106, the second machine learning model 107, the learning tire image acquisition unit 108, the learning texture area identification unit 109, the learning partial image generation unit 110, the learning partial image selection unit 111, and the learning unit 112 in the data processing device 1 correspond to the functions of a learning device.

[0032] The data processing device 1 according to this embodiment utilizes a characteristic unique to tires, in that the texture patterns appearing on the tire inner surface are continuous and periodic in the tire circumferential direction, to group multiple partial images generated from a tire image showing the tire inner surface by texture, and perform estimation and learning using separate machine learning models. Here, the present embodiment will be described as an example, but modified examples of the present invention will be described later. First, the details of each function of the data processing device 1 according to this embodiment will be described below.

[0033] [Target tire image acquisition unit] The target tire image acquisition unit 101 acquires a tire image showing the inner surface of the tire.

[0034] 4 is a diagram illustrating an example of a tire image and an example of processing of the tire image according to an embodiment of the present invention. As shown in FIG. 4, the tire image 30 is an image of the tire inner surface on a two-dimensional plane. The tire image 30 is acquired by scanning the tire inner surface in the circumferential direction using, for example, a line sensor camera or the like.

[0035] 4, the direction 41 of the long sides of the tire image 30 corresponds to the tire circumferential direction, and the direction 42 of the short sides corresponds to the tire circumferential direction. The direction 42 corresponds to the tire width direction in the tire tread portion, for example.

[0036] For example, the tire image 30 may show the inner surface of the tire over a range that is greater than one circumference of the tire. For example, a portion in the same circumferential phase as the portion shown at the top of the tire image 30 may also be shown at the bottom of the tire image 30. In this way, it is possible to perform a visual inspection thoroughly around the entire circumference of the tire.

[0037] The tire image 30 may also show parts of the tire, such as the tread, bead, sidewall, and shoulder of the tire inner surface, or may show all of these parts.

[0038] As shown in Figure 4, tire image 30 shows two types of texture: a first texture 21 that is a checkered pattern that combines upward and downward stripes, and a second texture 22 that is a different texture from first texture 21 and has a downward striped pattern. Typically, on the inner surface of a tire, a checkered texture, a striped texture, or a texture without a pattern appears continuously in the circumferential direction of the tire, with the texture switching in the direction perpendicular to the circumferential direction of the tire. The principle behind how texture appears on the inner surface of a tire in this way will be explained below.

[0039] Typically, the tire vulcanization process uses a flexible, bag-like component called a bladder, which is placed inside the tire. During the tire vulcanization process, the tire is expanded from the inside by the bladder, which is filled with gas or the like and inflated to fit the contours of the tire's inner surface, and is heated while being pressed against a mold placed outside the tire. During this process, the uneven pattern on the bladder's surface is transferred to the tire's inner surface.

[0040] Typically, the surface of the bladder has the same periodic uneven pattern continuously in the circumferential direction, but the pattern changes in the direction perpendicular to the circumferential direction. For example, the center of the bladder has no pattern, the area next to it has a striped pattern, and the area next to that has a checkered pattern, and so on. Therefore, the tire inner surface, to which the bladder pattern is transferred, also has the same periodic pattern continuously in the circumferential direction, but the pattern changes in the direction perpendicular to the circumferential direction. For example, the center of the tire has no pattern, the area next to it has a striped pattern, and the area next to that has a checkered pattern, and so on. In this way, the tire inner surface has several textures that show continuous periodic patterns in the circumferential direction, or no pattern, in the direction perpendicular to the circumferential direction.

[0041] In this way, the texture that appears on the inner surface of the tire is due to the uneven pattern applied to the bladder. Therefore, if the type of bladder used differs depending on the type of tire to be manufactured, the type and arrangement of the uneven pattern applied to the bladder will also differ, and the type and arrangement of the texture that appears on the inner surface of the tire will also differ depending on the type of tire.

[0042] Furthermore, even if the same type of bladder is used, different types of tires have different tire sizes and inner tire contours, which means that the range of texture that appears on the tire's inner surface and the characteristics of that pattern also vary depending on the type of tire. For example, if the tire's width is different, the transition position of the bladder's pattern will correspond to different parts of the tire's inner surface, and so the range of texture that appears on the tire will also vary. Furthermore, if the tire's diameter is different, the bladder's expansion rate will be different, so even if the stripe pattern is the same, the pattern period, such as the spacing between each stripe, will be different. Furthermore, for example, if the tire's inner surface contour is different, the way the bladder expands to match that contour will be different, and so the tilt and distortion of the pattern transferred along the curvature of that contour will be different.

[0043] In this way, the type and range of texture that appears on the inner surface of the tire and the characteristics of that pattern differ depending on the type of bladder used and the type of tire being manufactured, but it is possible to know in advance which type of texture appears where on the inner surface of the tire and what the characteristics of that pattern are by, for example, acquiring a tire image showing the inner surface of a sample tire.

[0044] [Target texture area identification part] Next, the target texture area specifying unit 102 specifies the positions of the texture areas that are the areas of the textures that appear in the tire image 30 acquired by the target tire image acquiring unit 101 .

[0045] In the example shown in Figure 4, the tire image 30 contains a first texture 21 with a checkered pattern and a second texture 22 with a downward-sloping striped pattern, and the target texture area identification unit 102 identifies the position of the first texture area 310 in which the first texture 21 is contained and the position of the second texture area 320 in which the second texture 22 is contained, among the areas in the tire image 30.

[0046] In this embodiment, an example of identifying texture regions using frequency analysis will be described as a method for identifying each texture region. For example, the target texture region identifying unit 102 may perform frequency analysis on the tire image 30, extract a first texture specific pattern 210 in the tire image 30 that has a predetermined frequency characteristic specific to the first texture 21, and identify the first texture region 310 based on the first texture specific pattern 210. Details of this processing will be described below.

[0047] First, the target texture region identification unit 102 converts the tire image 30 into the spatial frequency domain to generate a spatial frequency image (not shown). The spatial frequency image is an image that shows spatial variations in shading in the tire image 30 as a frequency component spectrum. One or more peaks appear at specific positions in the spatial frequency image according to the periodicity of the texture. This is called the frequency characteristic. Note that the conversion into the spatial frequency domain may be performed using, for example, a two-dimensional Fourier transform, a two-dimensional fast Fourier transform, a two-dimensional Walsh-Hadamard transform, or a discrete cosine transform.

[0048] Next, the target texture region identification unit 102 performs filtering processing to extract predetermined frequency characteristics specific to the first texture 21 from the spatial frequency image, thereby generating a post-extraction spatial frequency image (not shown). In the example shown in FIG. 4, the upward-sloping stripes (first texture specific pattern 210) in the first texture 21 are a pattern specific to the first texture 21 that is not present in other textures (second texture 22) captured in the tire image 30, and specific frequency characteristics corresponding to these upward-sloping stripes also appear in the spatial frequency image. These frequency characteristics vary depending on the characteristics of the pattern, such as the period and angle of the stripes. However, as described in the description of the target tire image acquisition unit, since the characteristics of the pattern of the tire to be estimated can be known in advance, the frequency characteristics can also be known in advance. In other words, it is possible to prepare in advance a filter that extracts the frequency characteristics of the tire to be estimated. Then, the spatial frequency image can be filtered using this filter to generate a post-extraction spatial frequency image.

[0049] Next, the target texture region identification unit 102 inversely transforms the post-extraction spatial frequency image to generate a post-extraction tire image 31 from which the first texture unique pattern 210, which is a pattern unique to the first texture 21, has been extracted. Here, as described above, the unique pattern of the first texture 21 is a right-upward sloping stripe. Note that the post-extraction tire image 31 and the tire image 30 are images of the same size, and a certain region in the post-extraction tire image 31 corresponds to a region at the same position in the tire image 30.

[0050] Next, the target texture region identification unit 102 extracts each element of the first texture unique pattern 210 by performing binarization processing or the like on the extracted tire image 31. In the example shown in FIG. 4, each element of the first texture unique pattern 210 is, for example, each line of the upward-sloping stripes.

[0051] Next, the target texture region identification unit 102 determines the region obtained by expanding at least one of the elements of the extracted first texture unique pattern 210 as the first texture region 310, and identifies the position of the first texture region 310. For example, the target texture region identification unit 102 may determine the region obtained by expanding each of the upward-sloping stripes, which are elements of the extracted first texture unique pattern 210, in one direction in the tire circumferential direction to their adjacent elements as the first texture region 310. Because the tire is continuous in the circumferential direction and the pattern is also continuous in the tire circumferential direction, the region obtained by expanding the stripes in the tire circumferential direction can be considered to be the texture region of the texture corresponding to the stripe pattern. Note that all or some of the elements of the first texture unique pattern 210 may be expanded in this manner. The expansion direction does not have to be the tire circumferential direction. The expansion direction may be one direction or two directions.

[0052] Alternatively, the target texture region identifying unit 102 may identify the position of the first texture region 310 based on the extent of the extracted first texture unique pattern 210, without being limited to the above. For example, the target texture region identifying unit 102 may determine that the region from the left end of the extracted tire image 31 to the right end of the first texture unique pattern 210 in direction 42 is the boundary of the region, and determine that the region is the first texture region 310. As described above, the tire is continuous in the circumferential direction, and the pattern is also continuous in the tire circumferential direction. Therefore, each region obtained by dividing the tire in direction 42 orthogonal to the tire circumferential direction in this way can be considered to be a texture region.

[0053] In this way, the periodicity and continuity of the pattern are utilized, so that it is less susceptible to the influence of anything other than the pattern being captured, and it is possible to easily and reliably identify the position of the texture area.

[0054] If the second texture 22 has a unique pattern, the target texture region identifying section 102 may also identify the position of the second texture region 320 in the same manner as above.

[0055] Alternatively, if the second texture 22 does not have a unique pattern, the target texture region identifying unit 102 may determine that a region of the post-extraction tire image 31 other than the first texture region 310 is the second texture region 320, and identify the position of the second texture region 320. Alternatively, if the target texture region identifying unit 102 identifies the position of the first texture region 310 based on the extent of the extracted first texture unique pattern 210, the target texture region identifying unit 102 may also identify the position of the second texture region 320 based on the extent of the extracted first texture unique pattern 210. For example, the target texture region identifying unit 102 may determine the region from the right end of the first texture unique pattern 210 to the right end of the post-extraction tire image 31 as the second texture region 320, using the right end of the first texture unique pattern 210 in the direction 42 as the boundary of the region, and identify the position of the second texture region 320.

[0056] The target texture region specifying unit 102 may generate region position information relating to the position of the specified texture region. This region position information is used in processing by the target partial image selecting unit 104, which will be described later.

[0057] For example, the target texture region identification unit 102 may generate a region display image in which the gradation of pixels corresponding to a certain texture region is set to a predetermined value among the multiple pixels that make up the tire image 30. For example, the gradation of pixels corresponding to a first texture region may be set to 255, and a region display image may be generated in which the region with gradation 255 is the first texture region and the other region is the second texture region.

[0058] Alternatively, the target texture region identification unit 102 may generate a region-information-added image in which a region-related channel related to the region is added to the tire image 30. For example, a region-information-added image may be generated in which a region corresponding to pixels set to "1" in the region-related channel is a first texture region, and a region corresponding to pixels set to "0" in the region-related channel is a second texture region.

[0059] [Target Part Image Generation] 5 is a diagram illustrating an example of a process for generating multiple partial images from a tire image according to one embodiment of the present invention, and the processing performed on these partial images. As shown in FIG. 5, the target partial image generation unit 103 generates multiple partial images of the same size, each showing a different portion of the tire image 30. The target partial image generation unit 103 generates the multiple partial images so that any portion of the tire image 30 appears in any of the multiple partial images. The size of the partial images is, for example, 256 pixels in length and width, and can be set appropriately by the designer.

[0060] Explaining this more specifically using Figure 5, the target partial image generation unit 103 generates partial image 601 as a partial image corresponding to portion 5 of tire image 30 at position 501, the leftmost position in the top row, out of multiple positions shown on tire image 30. In addition, the target partial image generation unit 103 generates partial image 602 as a partial image at position 502, the second from the left end in the top row. In the same manner, the target partial image generation unit 103 generates multiple partial images, such as partial image 603 at position 503, partial image 604 at position 504, partial image 605 at position 505, partial image 606 at position 506, partial image 607 at position 507, and so on. The target partial image generation unit 103 generates partial images in the same manner for the second row and beyond.

[0061] The target partial image generating unit 103 may generate partial image position information, which is information about the position of each partial image. This partial image position information is used in processing by the target partial image selecting unit 104, which will be described later.

[0062] As shown in FIG. 5, each of the multiple partial images may be positioned so as to overlap a portion of another partial image adjacent to the partial image. For example, in FIG. 5, partial images 601 and 602 partially overlap by about half. That is, the same image that appears in the right half of partial image 601 appears in the left half of partial image 602. The same applies to partial images 602 and 603, partial images 603 and 604, partial images 604 and 605, partial images 605 and 606, and partial images 606 and 607. Adjacent partial images may partially overlap not only in direction 42 but also in direction 41. The target partial image generation unit 103 may generate partial images using a sliding window method or the like so that adjacent partial images overlap in this manner.

[0063] By generating multiple overlapping partial images in this way, partial images that show a single texture closer to the boundaries of the texture region can be obtained, making it easier to improve inspection accuracy near the boundaries of the texture region. It also makes it easier to prevent missed detection of visual abnormalities at the boundaries of adjacent partial images. It also makes it easier to identify the precise location of visual abnormalities. Furthermore, during learning, which will be described later, it makes it easier to obtain a large amount of learning data from a single tire image. These effects become more pronounced as the degree of overlap increases (the sliding interval decreases).

[0064] [Target image selection section] The plurality of partial images generated in this manner show either a first checkered texture 21 or a second diagonally downward striped texture 22, or both. The target partial image selection unit 104 determines whether or not the first texture is shown in each partial image based on the position of the first texture region 310 and the position of the portion of the tire image 30 indicated by each of the plurality of partial images, and selects, from the plurality of partial images, a plurality of first partial images that show the first texture 21. The target partial image selection unit 104 also determines whether or not the second texture is shown in each partial image based on the position of the second texture region 320 and the position of the portion of the tire image 30 indicated by each of the plurality of partial images, and selects, from the plurality of partial images, a plurality of second partial images that show the second texture 22.

[0065] The above-described region position information may be used for the position of the first texture region 310 and the position of the second texture region 320. Furthermore, the above-described partial image position information may be used for the position of the portion of the tire image 30 indicated by each of the multiple partial images.

[0066] Note that the trained machine learning models used in the estimation process described below are dedicated machine learning models corresponding to each texture. Therefore, in order to improve estimation accuracy, it is desirable that each partial image input to the trained machine learning model contains only a single texture corresponding to that machine learning model.

[0067] Therefore, the target partial image selection unit 104 may select, as the first partial image, a partial image that shows only the first texture 21 and does not show the second texture 22. Similarly, the target partial image selection unit 104 may select, as the second partial image, a partial image that shows only the first texture 21 and does not show the second texture 22.

[0068] 5, for example, partial images 601 to 603 in the top row are selected as first partial images because they only show the first texture 21 with a checkered pattern and do not show the second texture 22 with a downward-sloping stripe pattern. Partial images in the second row and thereafter are selected in the same manner. The partial images selected in this way are grouped into a group 71 of first partial images showing the first texture.

[0069] Similarly, the partial images 605 to 607 in the top row are identified as second partial images because they only show the second texture 22 with downward-sloping stripes and do not show the first texture 21 with a checkered pattern. Partial images in the second row and thereafter are selected in the same manner. The partial images selected in this way are grouped into a group 72 of second partial images showing the second texture.

[0070] [Estimation part] The partial images selected for each texture in this way are input into a dedicated trained machine learning model (trained model) corresponding to each texture, and the appearance of the tire's inner surface is estimated for each partial image.

[0071] The estimation unit 105 estimates the quality of the appearance of the tire inner surface shown in each of the plurality of first partial images based on the output when each of the first partial images is input to the trained first machine learning model 106. Furthermore, the estimation unit 105 estimates the quality of the appearance of the tire inner surface shown in each of the plurality of second partial images based on the output when each of the second partial images is input to the trained second machine learning model.

[0072] Here, a case will be described in which the estimation unit 105 estimates the quality of the appearance of the tire inner surface by an anomaly detection method using a machine learning model such as an autoencoder. In this case, the estimation unit 105 may estimate the quality of the appearance of the tire inner surface for the first texture region 310 based on the error between an input image to the trained first machine learning model 106 and an output image from the trained first machine learning model 106. Similarly, the estimation unit 105 may estimate the quality of the appearance of the tire inner surface for the second texture region 320 based on the error between an input image to the trained second machine learning model 107 and an output image from the trained second machine learning model 107.

[0073] For example, the estimation unit 105 may input each of the multiple first partial images into the trained first machine learning model 106, and estimate the quality of the appearance of the tire inner surface in the region of each first partial image based on the error between each first partial image and an output image when the first partial image is input into the trained first machine learning model 106. Similarly, the estimation unit 105 may input each of the multiple second partial images into the trained second machine learning model 107, and estimate the quality of the appearance of the tire inner surface in the region of each second partial image based on the error between each second partial image and an output image when the second partial image is input into the trained second machine learning model 107.

[0074] Since the position of each first partial image in the tire image 30 is known, for example, if it is estimated that an abnormality appears in the appearance of the inner surface of the tire in a certain first partial image, it is possible to determine which part of the tire image 30 the abnormality is located in. The same applies to the second partial image. Therefore, the estimation unit 105 may create a map showing where the abnormality appears in the tire image 30 based on the estimation results for each first partial image using the trained first machine learning model and the estimation results for each second partial image using the trained second machine learning model.

[0075] [First machine learning model] The first machine learning model 106 is a machine learning model for estimating the quality of the appearance of a first partial image in which a first texture appears. In other words, the first machine learning model 106 is a dedicated machine learning model corresponding to the first texture.

[0076] For example, the first machine learning model 106 may be an unsupervised learning machine learning model. More specifically, the first machine learning model 106 may be an unsupervised learning machine learning model that obtains output data with the same number of dimensions as input data, such as an autoencoder.

[0077] In this case, the first machine learning model 106 has an encoder that reduces the number of dimensions of the data and a decoder that restores the number of dimensions to the original number. For example, when an image is input to the first machine learning model 106, the encoder extracts features with fewer dimensions from the input image, and the decoder restores the features to the original number of dimensions, resulting in an output image with the same number of dimensions as the input image.

[0078] The first machine learning model 106 may be trained using a large number of normal images in which the tire inner surface is free of abnormalities and has a good appearance, so as to reduce the error between the input image and the output image. The trained first machine learning model 106, trained in this manner, will output an image with a small error from the normal image in response to an input normal image. When an image in which an abnormality appears on the tire inner surface is input to this trained first machine learning model 106, an image in which a large error from the input image appears at the location where the abnormality appears is output. Therefore, by comparing the input image and the output image and extracting the difference between them, it is possible to detect abnormalities in the input image and estimate the quality of the tire's appearance.

[0079] The first machine learning model 106 is trained using a plurality of first partial images. Furthermore, the first machine learning model 106 that has been trained receives the plurality of first partial images as input, and outputs an image corresponding to each input image.

[0080] Note that the first machine learning model 106 is not limited to an autodecoder, and may be, for example, a stacked autoencoder, a variational autoencoder, a convolutional autoencoder, or the like.

[0081] Furthermore, the first machine learning model 106 does not necessarily have to be included in the data processing device 1, and may exist outside the data processing device 1. In this case, for example, the control unit 11 may further include a communication unit (not shown), and the estimation unit 105 and the learning unit 112 (described later) may input and output data to the external first machine learning model 106 via the communication unit.

[0082] [Second machine learning model] The second machine learning model 107 is similar to the first machine learning model 106. The second machine learning model 107 is a machine learning model for estimating the quality of the appearance of a second partial image that shows a second texture. In other words, the second machine learning model 107 is a dedicated machine learning model that corresponds to the second texture. The second machine learning model 107 is trained using a plurality of second partial images. Furthermore, the trained second machine learning model 107 receives input of a plurality of second partial images, and outputs an image corresponding to each input image.

[0083] [Learning tire image acquisition section] The processing of the learning tire image acquisition unit 108 is similar to the processing of the target tire image acquisition unit 101 .

[0084] [Learning texture region identification part] The processing of the learning texture region specifying unit 109 is similar to the processing of the target texture region specifying unit 102 .

[0085] [Learning image generation part] The processing of the learning partial image generation unit 110 is the same as that of the target partial image generation unit 103. Note that the number of pixels of the partial images generated by the target partial image generation unit 103 and the partial images generated by the learning partial image generation unit 110 are to match.

[0086] [Learning image selection section] The processing of the learning partial image selection unit 111 is similar to that of the target data generation unit 104 .

[0087] [Study Department] The learning unit 112 performs learning of the first machine learning model 106 based on the plurality of first partial images. The learning unit 112 also performs learning of the second machine learning model 107 based on the plurality of second partial images.

[0088] [1-3. Summary of embodiments of the present invention] As described above, the estimation device (data processing device 1) includes a target tire image acquisition unit 101 that acquires tire images showing a first texture and a second texture that appear on the tire inner surface, a target texture area identification unit 102 that identifies the positions of the first texture area and the second texture area, a target partial image generation unit 103 that generates a plurality of partial images, a target partial image selection unit 104 that selects a plurality of first partial images and a plurality of second partial images from the plurality of partial images, and an estimation unit 105 that estimates the quality of the appearance of the tire inner surface shown in the first partial images based on the output when each of the plurality of first partial images is input to a trained first machine learning model 106, and that estimates the quality of the appearance of the tire inner surface shown in the second partial images based on the output when each of the plurality of second partial images is input to a trained second machine learning model 107. This estimation device can perform estimation using a machine learning model for each texture, making it easier to improve the accuracy of appearance inspection of tire inner surfaces.

[0089] The learning device (data processing device 1) also includes a learning tire image acquisition unit 108 that acquires tire images showing a first texture and a second texture that appear on the tire inner surface, a learning texture area identification unit 109 that identifies the position of the first texture area and the position of the second texture area, a learning partial image generation unit 110 that generates a plurality of partial images, a learning partial image selection unit 111 that selects a plurality of first partial images and a plurality of second partial images from the plurality of partial images, and a learning unit 112 that trains a first machine learning model 106 based on the plurality of first partial images and trains a second machine learning model 107 based on the plurality of second partial images. This learning device allows learning to be performed using a machine learning model for each texture, making it easier to improve the accuracy of visual inspection of tire inner surfaces.

[0090] [2. Modifications] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention.

[0091] For example, the texture region shown in the tire image is not limited to a single, unified region, but may be two or more separate regions. FIG. 6 is a diagram illustrating an example of a tire image according to a modified example of the present invention and the processing performed on the tire image. As shown in FIG. 6, a first texture 23 with downward-sloping stripes may appear at both ends of a tire image 80 in a direction 42, and a second texture 24 without a pattern may appear in the center. In this case, as shown in FIG. 6, the target texture region identifying unit 102 may generate an extracted tire image 81 from which a first texture-specific pattern 230 (diagonal stripes) is extracted, and identify the position of a first texture region 810, which is a set of separate regions, based on the extracted first texture-specific pattern 230.

[0092] Furthermore, the tire image does not necessarily need to show the tire inner surface over an area that is longer than one circumference of the tire, and may show the tire inner surface over an area that is shorter than one circumference of the tire. In this case, in the estimation step, tire images may be acquired in multiple circumferential ranges, and estimation processing may be performed on each tire image to estimate the quality of the appearance of one circumference of the tire.

[0093] Furthermore, the number of textures shown in the tire image is not limited to two, and may be three or more. In this case, for example, the target texture region identification unit 102 may identify the position of each texture region based on the unique pattern of each texture. This process is effective when each texture has a unique pattern not found in other texture regions. Alternatively, the target texture region identification unit 102 may narrow down the regions by sequentially identifying texture regions starting with textures having unique patterns not found in other textures. For example, a case will be described in which the tire image contains three types of textures: a first texture with a checkered pattern, a second texture with a downward-sloping stripe pattern, and a third texture without a pattern. In this case, the target texture region identification unit 102 identifies the first texture region based on the first texture unique pattern (the upward-sloping stripes in the checkered pattern) that is unique to the first texture and not found in the second and third textures. Next, in the region of the tire image other than the first texture region, the target texture region identification unit 102 identifies the second texture region based on the second texture unique pattern (the downward-sloping stripes) that is not found in the third texture. Next, the region of the tire image other than the first texture region is identified as the third texture region. This process is effective when, at each step of sequentially identifying texture regions, there is a texture that has a unique pattern that is not present in the other textures among the remaining textures. The same applies to the learning texture region identifying unit 109.

[0094] Furthermore, the target texture region identifying unit 102 does not necessarily need to identify the position of the texture region using frequency analysis, and may identify the first texture region and the second texture region using, for example, another method that utilizes machine learning, such as pattern matching. The same applies to the learning texture region identifying unit 109.

[0095] Furthermore, each of the plurality of partial images generated by the target partial image generation unit 103 does not necessarily need to overlap with a part of another partial image adjacent to the partial image, and may be adjacent to another adjacent partial image without overlapping. The same applies to the plurality of partial images generated by the training partial image generation unit 110.

[0096] Furthermore, each of the plurality of first partial images does not necessarily have to be one in which the second texture is not reflected at all, but may have the second texture reflected therein. Similarly, each of the plurality of second partial images does not necessarily have to be one in which the first texture is not reflected at all, but may have the first texture reflected therein. For example, the target partial image selection unit 104 may select as the first partial image an image in which the proportion of the area in which the first texture is reflected in the area of ​​the partial image satisfies a predetermined threshold. Similarly, the target partial image selection unit 104 may select as the second partial image an image in which the proportion of the area in which the second texture is reflected in the area of ​​the partial image satisfies a predetermined threshold. The same applies to the learning partial image selection unit 111.

[0097] Furthermore, the first machine learning model 106 does not necessarily have to be an unsupervised learning model, but may also be a supervised learning or semi-supervised learning model. When the first machine learning model 106 is a supervised learning model, learning may be performed using learning data, which is a training dataset in which a label indicating normality or abnormality is assigned to each partial image. In this case, learning may be performed so that when a partial image is input to the first machine learning model 106, the presence or absence of an abnormality in the appearance of the tire inner surface in the partial image is output. The estimation unit 105 may then estimate the quality of the appearance of the tire inner surface based on the output regarding the presence or absence of an abnormality in each first partial image when each of the multiple first partial images is input to the trained first machine learning model 106. When the first machine learning model 106 is a semi-supervised learning model, the dataset may include labeled and unlabeled partial images, and learning may be performed using both of these. In this case, learning may also be performed so that when a partial image is input to the first machine learning model 106, the presence or absence of an abnormality in the appearance of the partial image is output. The estimation unit 105 may then estimate the quality of the appearance of the tire inner surface based on the output regarding the presence or absence of an abnormality in each of the multiple first partial images when each of the first partial images is input to the trained first machine learning model 106. Alternatively, the first machine learning model 106 may be another type of unsupervised machine learning model that detects partial images showing an abnormality by clustering, etc. The same applies to the second machine learning model 107. [Explanation of symbols]

[0098] 1 data processing device, 11 processing device, 12 storage device, 13 display device, 14 input device, 101 target tire image acquisition unit, 102 target texture region identification unit, 103 target partial image generation unit, 104 target partial image selection unit, 105 estimation unit, 106 first machine learning model, 107 second machine learning model, 108 learning tire image acquisition unit, 109 learning texture region identification unit, 110 learning partial image generation unit, 111 learning partial image selection unit, 112 learning unit, 2 tire, 20 tire inner surface, 21 first texture, 22 second texture, 23 first texture, 24 second texture, 210 first texture specific pattern, 230 first texture specific pattern, 30 tire image, 31 extracted tire image, 310 first texture region, 320 second texture region, 41 direction, 42 direction, 5 part, 501 to 507 Positions, 601 to 607, partial images, 71, first group of partial images, 72, second group of partial images, 80, tire image, 81, tire image after extraction, 810, first texture region.

Claims

1. a tire image acquisition means for acquiring a tire image showing a first texture and a second texture different from the first texture, which appear on the inner surface of the tire; a texture area specifying means for specifying, among areas in the tire image, a position of a first texture area in which the first texture is projected and a position of a second texture area in which the second texture is projected; a partial image generating means for generating a plurality of partial images, all of the same size, showing different portions of the tire image; a partial image selection means for selecting, from the plurality of partial images, a plurality of first partial images in which the first texture appears, based on the position of the first texture region and the position of a portion of the tire image indicated by each of the plurality of partial images, and for selecting, from the plurality of partial images, a plurality of second partial images in which the second texture appears, based on the position of the second texture region and the position of a portion of the tire image indicated by each of the plurality of partial images; an estimation means for estimating the quality of the appearance of the tire inner surface shown in each of the plurality of first partial images based on the output when each of the plurality of first partial images is input into a trained first machine learning model, and for estimating the quality of the appearance of the tire inner surface shown in each of the plurality of second partial images based on the output when each of the plurality of second partial images is input into a trained second machine learning model; An estimation device comprising:

2. the second texture is not captured in each of the plurality of first partial images, the first texture is not captured in each of the plurality of second partial images; The estimation device according to claim 1 .

3. Each of the plurality of partial images overlaps with a part of another partial image adjacent to the partial image. The estimation device according to claim 2 .

4. the texture region identifying means performs a frequency analysis on the tire image, extracts a unique pattern in the tire image having a predetermined frequency characteristic unique to the first texture, and identifies the first texture region based on the unique pattern. The estimation device according to any one of claims 1 to 3.

5. a tire image acquisition means for acquiring a tire image showing a first texture and a second texture different from the first texture, which appear on the inner surface of the tire; a texture area specifying means for specifying, among areas in the tire image, a position of a first texture area in which the first texture is projected and a position of a second texture area in which the second texture is projected; a partial image generating means for generating a plurality of partial images, all of the same size, showing different portions of the tire image; a partial image selection means for selecting, from the plurality of partial images, a plurality of first partial images in which the first texture appears, based on the position of the first texture region and the position of a portion of the tire image indicated by each of the plurality of partial images, and for selecting, from the plurality of partial images, a plurality of second partial images in which the second texture appears, based on the position of the second texture region and the position of a portion of the tire image indicated by each of the plurality of partial images; a learning means for learning a first machine learning model based on the plurality of first partial images and for learning a second machine learning model based on the plurality of second partial images; A learning device including:

6. the second texture is not captured in each of the plurality of first partial images, the first texture is not captured in each of the plurality of second partial images; The learning device according to claim 5 .

7. Each of the plurality of partial images overlaps with a part of another partial image adjacent to the partial image. The learning device according to claim 6.

8. the first machine learning model and the second machine learning model are unsupervised machine learning models; The learning device according to claim 7 .

9. the texture region identifying means performs a frequency analysis on the tire image, extracts a unique pattern in the tire image having a predetermined frequency characteristic unique to the first texture, and identifies the first texture region based on the unique pattern. The learning device according to any one of claims 5 to 8.

10. a tire image acquisition step of acquiring a tire image showing a first texture and a second texture different from the first texture, which appear on an inner surface of the tire; a texture area specifying step of specifying, among areas in the tire image, a position of a first texture area in which the first texture is displayed and a position of a second texture area in which the second texture is displayed; a partial image generating step of generating a plurality of partial images, all of the same size, showing different portions of the tire image; a partial image selection step of selecting, from the plurality of partial images, a plurality of first partial images in which the first texture appears, based on the position of the first texture region and the position of a portion of the tire image indicated by each of the plurality of partial images, and selecting, from the plurality of partial images, a plurality of second partial images in which the second texture appears, based on the position of the second texture region and the position of a portion of the tire image indicated by each of the plurality of partial images; an estimation step of estimating the quality of the appearance of the tire inner surface shown in each of the plurality of first partial images based on the output when each of the plurality of first partial images is input into a trained first machine learning model, and estimating the quality of the appearance of the tire inner surface shown in each of the plurality of second partial images based on the output when each of the plurality of second partial images is input into a trained second machine learning model; Estimation methods including:

11. a tire image acquisition step of acquiring a tire image showing a first texture and a second texture different from the first texture, which appear on an inner surface of the tire; a texture area specifying step of specifying, among areas in the tire image, a position of a first texture area in which the first texture is displayed and a position of a second texture area in which the second texture is displayed; a partial image generating step of generating a plurality of partial images, all of the same size, showing different portions of the tire image; a partial image selection step of selecting, from the plurality of partial images, a plurality of first partial images in which the first texture appears, based on the position of the first texture region and the position of a portion of the tire image indicated by each of the plurality of partial images, and selecting, from the plurality of partial images, a plurality of second partial images in which the second texture appears, based on the position of the second texture region and the position of a portion of the tire image indicated by each of the plurality of partial images; a learning step of learning a first machine learning model based on the plurality of first partial images and learning a second machine learning model based on the plurality of second partial images; How to generate a trained model including:

12. a tire image acquisition means for acquiring a tire image showing a first texture and a second texture different from the first texture, which appear on the inner surface of the tire; a texture area specifying means for specifying, among areas in the tire image, a position of a first texture area in which the first texture is projected and a position of a second texture area in which the second texture is projected; a partial image generating means for generating a plurality of partial images, all of the same size, showing different portions of the tire image; a partial image selection means for selecting, from the plurality of partial images, a plurality of first partial images in which the first texture appears, based on a position of the first texture region and a position of a portion of the tire image indicated by each of the plurality of partial images, and selecting, from the plurality of partial images, a plurality of second partial images in which the second texture appears, based on a position of the second texture region and a position of a portion of the tire image indicated by each of the plurality of partial images; an estimation means for estimating the quality of the appearance of the tire inner surface shown in each of the plurality of first partial images based on the output when each of the plurality of first partial images is input into a trained first machine learning model, and for estimating the quality of the appearance of the tire inner surface shown in each of the plurality of second partial images based on the output when each of the plurality of second partial images is input into a trained second machine learning model; An estimation program for making a computer function as a.

13. a tire image acquisition means for acquiring a tire image showing a first texture and a second texture different from the first texture, which appear on the inner surface of the tire; a texture area specifying means for specifying, among areas in the tire image, a position of a first texture area in which the first texture is projected and a position of a second texture area in which the second texture is projected; a partial image generating means for generating a plurality of partial images, all of the same size, showing different portions of the tire image; a partial image selection means for selecting, from the plurality of partial images, a plurality of first partial images in which the first texture appears, based on a position of the first texture region and a position of a portion of the tire image indicated by each of the plurality of partial images, and selecting, from the plurality of partial images, a plurality of second partial images in which the second texture appears, based on a position of the second texture region and a position of a portion of the tire image indicated by each of the plurality of partial images; a learning means for learning a first machine learning model based on the plurality of first partial images and for learning a second machine learning model based on the plurality of second partial images; A learning program for making computers function as.

Images