Method and system for analyzing internal structure of polymeric material

Abstract

To provide a method and a system for analyzing an internal structure of a polymeric material capable of more accurately grasping the internal structure of a polymeric material using image data acquired by an electron microscope.SOLUTION: A computing device 3 performs a preprocessing step S120 on original image data D1 of a polymeric material acquired by an electron microscope 2. An extraction model M, in which a relationship between contrast in image data and a string-like structure is linked, is constructed by supervised machine learning, in the preprocessing step S120, the original image data D1 is data-processed using the extraction model M to generate an extracted image data D3 in which the string-like structure represented by the contrast in the original image data D1 is extracted. In an analysis step S130, a feature amount of the string-like structure using the extracted image data D3, and predetermined characteristics of the polymeric material are grasped on the basis of the feature amount.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a method and system for analyzing the internal structure of a polymeric material, and more particularly to a method and system for analyzing the internal structure of a polymeric material that can more faithfully grasp the internal structure of the polymeric material using image data acquired by an electron microscope. [Background technology]

[0002] A method for understanding the network structure of crosslinked rubber using image data acquired by an electron microscope has been proposed (see Patent Document 1). The network structure of crosslinked rubber is composed of numerous molecular chains that intersect and branch in a complex manner in three dimensions. These molecular chains exist as string-like structures represented by contrast in image data of crosslinked rubber acquired by an electron microscope. Therefore, to accurately understand each molecular chain that makes up the network structure, it is necessary to distinguish minute differences in contrast in this image data and accurately extract the string-like structures represented by that contrast.

[0003] In the method proposed in Patent Document 1, image data acquired by an electron microscope is subjected to noise removal, binarization, and thinning to generate line-converted image data containing a large number of line image data, and the generated line-converted image data is used for analysis. However, this line-converted image data cannot distinguish minute differences in contrast in the original image data due to binarization. Therefore, there is room for improvement in more faithfully understanding the internal structure of polymeric materials using image data acquired by an electron microscope. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-85735 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide a method and system for analyzing the internal structure of a polymeric material that can more faithfully grasp the internal structure of the polymeric material using image data acquired by an electron microscope. [Means for solving the problem]

[0006] The method for analyzing the internal structure of a polymeric material of the present invention, which achieves the above-mentioned object, is a method for analyzing the internal structure of a polymeric material by a computing device using original image data of the polymeric material obtained by an electron microscope, and includes an image acquisition step of acquiring the original image data using the electron microscope, a preprocessing step of creating extracted image data in which a string-like structure represented by contrast in the original image data is extracted, and an analysis step of calculating feature amounts of the string-like structure using the extracted image data and grasping predetermined characteristics of the polymeric material based on the feature amounts, characterized in that an extraction model linking the relationship between the contrast in the image data and the string-like structure is constructed by supervised machine learning, and in the preprocessing step, the extracted image data is created by data processing the original image data using the extraction model.

[0007] The internal structure analysis system for polymeric materials of the present invention has a memory unit that stores original image data of polymeric materials acquired by an electron microscope, and is equipped with a computing device that analyzes the internal structure of the polymeric material using the original image data.The computing device performs a preprocessing step of creating extracted image data in which string-like structures represented by contrast in the original image data are extracted, and an analysis step of calculating feature values ​​of the string-like structures using the extracted image data and grasping predetermined characteristics of the polymeric material based on the feature values.An extraction model that links the relationship between the contrast in the image data and the string-like structure is constructed by supervised machine learning and stored in the memory unit.In the preprocessing step, the original image data is input to the extraction model, and the extracted image data is output by the extraction model. [Effects of the Invention]

[0008] According to the present invention, the extraction model analyzes and evaluates the influence of the contrast on the string-like structures, so that the extracted image data created using the extraction model accurately extracts changes in the shape and size of the string-like structures based on the contrast difference, even if the difference is small in the original image data. Therefore, using this extracted image data in the analysis step is advantageous for more faithfully understanding the internal structure of the polymer material being analyzed. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an explanatory diagram illustrating an embodiment of a system for analyzing the internal structure of a polymer material; [Figure 2] FIG. 1 is an explanatory diagram illustrating image data of a polymer material obtained by an electron microscope. [Figure 3] FIG. 1 is a flowchart showing an example of the procedure of a method for analyzing the internal structure of a polymer material. [Figure 4] FIG. 10 is an explanatory diagram illustrating an example of training data. [Figure 5] FIG. 10 is an explanatory diagram illustrating an example of a segmentation map. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of extracted image data. [Figure 7] FIG. 10 is a flowchart showing an example of a procedure of a modified example. [Figure 8] 10 is an explanatory diagram illustrating an example of a segmentation map of scaled image data whose image size is smaller than that of the original image data. FIG. [Figure 9] FIG. 10 is an explanatory diagram illustrating a segmentation map of scaled image data whose image size is approximately the same as that of the original image data. [Figure 10] 10 is an explanatory diagram illustrating an example of a segmentation map of scaled image data whose image size is larger than that of the original image data. FIG. [Figure 11] 9 is an explanatory diagram illustrating an example of scaled extracted image data created based on the segmentation map of FIG. 8. FIG. [Figure 12]10 is an explanatory diagram illustrating an example of enlarged / reduced extracted image data created based on the segmentation map of FIG. 9. FIG. [Figure 13] 11 is an explanatory diagram illustrating an example of scaled extracted image data created based on the segmentation map of FIG. 10. FIG. [Figure 14] FIG. 10 is an explanatory diagram illustrating a segmentation map of original image data. [Figure 15] FIG. 10 is an explanatory diagram illustrating an example of extracted image data. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the method and system for analyzing the internal structure of a polymer material according to the present invention will be described based on the embodiments shown in the drawings.

[0011] An embodiment of an internal structure analysis system 1 (hereinafter referred to as analysis system 1) illustrated in Fig. 1 is used to carry out an internal structure analysis method for a polymer material illustrated in Fig. 3. In this analysis method, extracted image data D3 illustrated in Fig. 6 is created by extracting a string-like structure represented by contrast (difference in brightness) from image data D1 (hereinafter referred to as original image data D1) of a polymer material illustrated in Fig. 2 acquired by an electron microscope 2. The internal structure of the polymer material is then analyzed using this extracted image data D3.

[0012] An overview of the internal structure analysis method illustrated in FIG. 3 will be described. This internal structure analysis method includes an image acquisition step S110, a preprocessing step S120, and an analysis step S130, which are performed in sequence. In the image acquisition step S110, original image data D1 is acquired using an electron microscope 2. In the preprocessing step S120, the original image data D1 is processed using an extraction model M constructed by supervised machine learning to obtain extracted image data D3 in which the string-like structure of the original image data D1 is extracted. In the analysis step S130, feature quantities of the string-like structure are calculated using this extracted image data D3, and specific characteristics of the polymer material are identified based on these feature quantities. The string-like structure in the extracted image data D3 is specifically the molecular chains that make up the network structure of the polymer material, and only the string-like structure represented by the contrast of the original image data D1 is extracted in the extracted image data D3.

[0013] Examples of polymeric materials include crosslinked rubber (vulcanized rubber), resin, and unvulcanized rubber. These polymeric materials may contain fillers. In this embodiment, crosslinked rubber is used as the polymeric material. This crosslinked rubber is made by crosslinking natural rubber (NR) or synthetic rubbers such as styrene-butadiene rubber (SBR), butadiene rubber (BR), and isoprene rubber (IR) with a crosslinking agent (vulcanizing agent) such as sulfur or an organic peroxide. This crosslinked rubber may contain fillers such as silica or carbon black, as well as additives such as vulcanization accelerators and antioxidants.

[0014] To obtain the original image data D1, an ultrathin section taken from a cross-linked rubber test piece is used as the sample S. The thickness of the sample S is, for example, 50 nm to 300 nm. The cross-linked rubber test piece is prepared by swelling the cross-linked rubber with a polymerizable monomer and then polymerizing the polymerizable monomer in the presence of a polymerization initiator, resulting in a saturated, swollen network structure that is fixed. The polymerizable monomer used to prepare the test piece may be any polymerizable monomer that can swell the cross-linked rubber to a saturated state and polymerize in the presence of the cross-linked rubber. Examples of the polymerizable monomer include methyl methacrylate, styrene monomer, and silicone. The polymerization initiator may be any polymerizable monomer that generates radicals by heat, light, or oxidation-reduction. Examples of the polymerization initiator include benzoyl peroxide, peroxide, and triethylborane. The sample S may also be stained with a stain. Examples of the stain include osmium tetroxide and iodine. When collecting sample S from the test piece, it is advisable to set the magnification of the electron microscope 2 lower than when acquiring the original image data D1, identify a position of the test piece that exhibits a representative cross-linked state, and collect sample S from that position.

[0015] The analysis system 1 includes a calculation device 3 to which original image data D1 of a sample S acquired by an electron microscope 2 is input. The calculation device 3 is configured as a computer, and various data are input and stored, and data processing is performed using this data. Various known computers can be used for the calculation device 3. The calculation device 3 includes a calculation processing unit (CPU) 4, a main storage unit (memory) 5, and an auxiliary storage unit (e.g., HDD) 6.

[0016] Various known transmission electron microscopes can be used for the electron microscope 2. This electron microscope 2 irradiates the sample S with an electron beam and acquires original image data D1 from which the spatial distribution of electrons inside the sample S can be observed based on the intensity of the transmitted or diffracted electron beam. The original image data D1 acquired by the electron microscope 2 is input to the calculation device 3. In this embodiment, two-dimensional original image data D1 is acquired by the electron microscope 2 and used for analysis, but three-dimensional original image data D1 can also be acquired and used for analysis. The electron microscope 2 may acquire a bright-field image formed by a transmitted wave as the original image data D1, or a dark-field image formed by a diffracted wave.

[0017] In the analysis system 1, when original image data D1 acquired by an electron microscope 2 is input to a calculation device 3 and a predetermined program is started and executed by an input unit (keyboard, mouse, etc.) 7, the calculation device 3 executes each data processing instructed by the program. Then, the calculation device 3 outputs the analysis results obtained by executing each data processing to an output unit (display, etc.) 8.

[0018] The original image data D1 shown in Figure 2 was acquired as a grayscale dark-field image of the object, which was made into molecular chains of crosslinked rubber, by an electron microscope 2, and stored in the auxiliary memory unit 6 of the computing device 3. In this original image data D1, the white parts generally represent the molecular chains of crosslinked rubber, and the black parts represent the background, i.e., the embedding resin (mesh part) that fixes the network structure formed by the molecular chains of crosslinked rubber. Some of the white parts of this original image data D1 contain noise.

[0019] In this original image data D1, the string-like structures represented by contrast represent molecular chains. Each image, which is a collection of pixels with similar pixel values, represents an individual string-like structure. In other words, the pixel values ​​of the pixels corresponding to different string-like structures are different between the string-like structures. The difference in contrast (difference in pixel value) between different string-like structures represents the degree of overlap between the string-like structures in the depth direction of the paper in Figure 2, differences in the arrangement positions of the string-like structures, etc. In the original image data D1, the more overlapping string-like structures there are, or the closer the string-like structures are to the front, the larger the pixel value (the brighter the image). Therefore, to extract string-like structures from the original image data D1 with high accuracy, it is necessary to distinguish subtle differences in contrast in the original image data D1.

[0020] However, it is difficult to distinguish subtle differences in contrast in the original image data D1, making it difficult to accurately grasp the state of individual string-like structures (molecular chains). For example, in the area A enclosed by the white square in Figure 2, an image composed of pixels with relatively large pixel values ​​exists, but it is difficult to determine whether this image represents a single thick string-like structure or a state in which multiple thin string-like structures are overlapping. Therefore, in this embodiment, in the preprocessing step S120 before the analysis step S130 is performed, an extraction model M constructed by supervised machine learning is used to create extracted image data D3 in which string-like structures based on the contrast in the original image data D1 are extracted, and this extracted image data D3 is used in the analysis step S130.

[0021] The contents of each step (S110, S120, S130) illustrated in FIG. 3 will be described in detail below.

[0022] In the image acquisition step S110, original image data D1 of the sample S collected from a test piece of crosslinked rubber is acquired using the electron microscope 2. In this step, the acceleration voltage, magnification, etc. of the electron microscope 2 are adjusted to acquire a dark-field image, as the original image data D1, in which diffracted waves of molecular chains of the crosslinked rubber are imaged.

[0023] In the preprocessing step S120, the calculation device 3 processes the original image data D1 using an extraction model M constructed by supervised machine learning, thereby generating extracted image data D3. The extraction model M may be constructed in the preprocessing step S120, or may be constructed in advance before this internal structure analysis method is performed and stored in the auxiliary storage unit 6 of the calculation device 3. The extraction model M of this embodiment is constructed in the preprocessing step S120. Specifically, processed image data D2 is generated by performing image processing such as noise removal and expansion / contraction on the original image data D1 (S121). Next, the extraction model M is constructed by supervised machine learning (S122). Next, the original image data D1 is input to the constructed extraction model M, and extracted image data D3 is generated by the extraction model M (S123).

[0024] In step S121, the arithmetic unit 3 performs image processing such as noise removal and expansion / contraction on the original image data D1 to generate processed image data D2. This image processing should be able to generally remove (reduce) structures other than the string-like structures that are separate from the string-like structures represented by contrast while generally maintaining the contrast (difference between light and dark) in the original image data D1.

[0025] In step S121 of this embodiment, image processing involves performing noise removal and expansion / contraction on the original image data D1. Various known noise removal image processing methods can be used for noise removal. Examples of this noise removal image processing include known noise removal image processing methods such as L1 regularization and L2 regularization. An appropriate method can be selected from these noise removal image processing methods depending on the noise generation status in the original image data D1. By performing this noise removal on the original image data D1, noise can be removed as structures other than string-like structures. Known morphology processing can be used for expansion / contraction. By performing this expansion / contraction on the original image data D1, the contrast (difference between light and dark) in the original image data D1 is largely maintained, and structures other than string-like structures are transformed into string-like structures. For example, structures other than string-like structures that cannot be observed in actual molecular chains are transformed into string-like structures, or portions of string-like structures that are interrupted midway are connected and transformed into a single continuous string-like structure. This step S121 is not essential and may be performed as appropriate depending on the state of noise generation in the original image data D1 and the presence of structures other than string-like structures in the original image data D1 after noise removal.

[0026] In step S122, the arithmetic device 3 executes data processing to construct an extraction model M by supervised machine learning. This extraction model M is constructed by supervised machine learning, links the relationship between contrast in image data and string-like structures, and is a type of computer program that extracts string-like structures represented by contrast in the original image data D1. Various known supervised machine learning methods can be used for the supervised machine learning to construct the extraction model M. A preferred supervised machine learning method is one that performs image segmentation in the process of extracting the string-like structures, and examples of such methods include a convolutional neural network (CNN), a full-layer convolutional network (FCN), SegNet, U-net, and a feature pyramid network (FPN).

[0027] The training data D4 used in supervised machine learning can be image data that can link the contrast in the image data to the string-like structure. For example, this training data D4 can be image data acquired by an electron microscope 2, where the target is a polymeric material of a different type from the polymeric material from which the sample S was collected, and extracted image data in which a clear string-like structure is extracted from the image data. Data suitable for such training data D4 can include the large amount of image data accumulated in the manufacturing and development process of crosslinked rubber and polymeric materials, or image data described in publicly known literature, which are already owned by those skilled in the art. However, if there is no data suitable for training data D4 among the large amount of image data already owned by those skilled in the art, it is necessary to prepare training data D4 separately.

[0028] In this embodiment, training data D4, exemplified in FIG. 4, is separately prepared. The training data D4, exemplified in FIG. 4, is stored in advance in the auxiliary storage unit 6 of the calculation device 3. This training data D4 accumulates image data and mask data (segmentation mask data) assigned to the image data for each sample (1, 2, . . . , n) in the leftmost column in FIG. 4. That is, this training data D4 accumulates masked image data as learning data (labeled learning data) to which a correct answer is assigned for each sample. This training data D4 accumulates two types of image data as masked image data: different polymer image data D5 and its mask data D5m, and similar image data D6 and its mask data D6m.

[0029] The different type of polymer image data D5 is acquired using an electron microscope with the same or different specifications as the electron microscope 2, with the object being a polymer material of a different type from the polymer material from which the sample S was collected. That is, the contrast (difference between light and dark) in this different type of polymer image data D5 represents the string-like structure of the polymer material. The sample for the different type of polymer image data D5 is not limited to one collected from crosslinked rubber, but may also be collected from a polymer material such as resin. Furthermore, when acquiring the different type of polymer image data D5 using an electron microscope, it is not necessary to match the electron microscope settings (acceleration voltage, magnification), imaging environment, etc. to those of the original image data D1.

[0030] The mask data D5m indicates the region in the different polymer image data D5 where the string-like structures exist. A person skilled in the art can generally identify this region based on the numerous image data accumulated during the manufacturing and development process and the numerous image data described in publicly known literature. Therefore, the mask data D5m is created based on the knowledge of such a person skilled in the art. The mask data D5m only needs to indicate the region in the different polymer image data D5 where the string-like structures exist, and the state of the individual string-like structures present in that region (such as size, shape, degree of overlap, and position in the depth direction of the paper) may be unclear.

[0031] The similar image data D6 is captured using a different imaging device than the electron microscope, capturing images of a different object made of non-rubber or non-resin. The imaging device preferably has a resolution greater than that of the electron microscope. Examples of similar image data D6 include image data of retinal blood vessels captured using a fundus image analysis system, and image data of tree branches or concrete cracks captured using a camera. These image data of different objects contain string-like structures that are not identical to the string-like structures of polymeric materials such as rubber or resin, but that can be considered similar to those structures. The degree of similarity between the string-like structures of the different object and the polymeric material can be set arbitrarily, as long as a person skilled in the art can consider the string-like structures of the different object and the polymeric material to be generally similar. For example, the similar image data D6 in Figure 4 is image data of retinal blood vessels. The retina contains various branched blood vessels (retinal arteries and veins), and the structures of these blood vessels can be considered similar to those of the polymeric material. This similar image data D6 can also be collected via the Internet.

[0032] The mask data D6m indicates the region exhibiting the string-like structure of the similar image data D6. The greater the similarity between this string-like structure and the string-like structure of the polymer material, the higher the accuracy of extraction of the string-like structure by the extraction model M. Therefore, it is preferable that the mask data D6m be used to select the portion of the string-like structure present in the similar image data D6 that is most similar to the string-like structure of the polymer material. Furthermore, multiple mask data D6m may be created for one similar image data D6.

[0033] The number of samples (n) in the training data D4 should be as large as possible without causing overlearning through machine learning. If the number of samples is small, the number of samples can be increased by using image data obtained by scaling or randomly sampling the prepared different-type polymer image data D5 or similar image data D6. Using image data obtained by resizing the different-type polymer image data D5 or similar image data D6 as training data D4 in this way expands the training data D4, which is advantageous for understanding the various shapes and sizes of molecular chains present in actual crosslinked rubber. In particular, preparing multiple different-type polymer image data D5 requires collecting a sample from a polymer material for each different-type polymer image data D5, acquiring image data of the sample using an electron microscope 2, and creating mask data D5m that faithfully reflects the subtle differences in contrast of the acquired image data. Therefore, preparing a single different-type polymer image data D5, resizing the image of the different-type polymer image data D5, and using multiple scaled different-type polymer image data with different image sizes as training data D4 is advantageous in reducing the labor required to prepare the training data D4.

[0034] When both the different polymer image data D5 and the similar image data D6 are accumulated as training data D4, the greater the similarity between the different polymer image data D5 and the similar image data D6, the better. Therefore, histogram matching may be performed so that the histogram of the similar image data D6 matches the histogram of the different polymer image data D5. When multiple different polymer image data D5 are accumulated as training data D4, the histogram of the similar image data D6 may be matched to the histogram of a representative different polymer image data D5 among the different polymer image data D5, or to a histogram calculated as a representative value of the histograms of the different polymer image data D5. Thus, the greater the similarity between the different polymer image data D5 and the similar image data D6, the more advantageous it is for improving the accuracy of the extraction model M constructed in step S122.

[0035] The extraction model M is constructed using U-net, a well-known machine learning technique that uses training data D4. U-net is a type of fully convolutional network and has an encoder and a decoder that are skip-connected to each other. The encoder gradually compresses the information of the input original image data D1 and is adjusted to extract features of the original image data D1. The decoder is adjusted to enlarge the features extracted by the encoder to the original image size and output extracted image data D3.

[0036] In more detail, extraction model M associates the relationship between the contrast in the image data and the region where the string-like structure exists, and analyzes and evaluates the degree of influence of the contrast on the region where the string-like structure exists. Extraction model M also associates the relationship between the region where the string-like structure exists in the image data and the string-like structure, and analyzes and evaluates the degree of influence of differences in the shape and size of that region on the string-like structure. Therefore, extraction model M indirectly associates the relationship between the contrast in the image data and the string-like structure, and can extract changes in the shape and size of the string-like structure for each degree of contrast difference. Since the associated string-like structure can generally be regarded as the string-like structure of a polymer material, extraction model M associates the relationship between the contrast in the image data and the string-like structure of the polymer material. Therefore, extraction model M can extract changes in the shape and size of the string-like structure of the polymer material for each degree of contrast difference in the image data.

[0037] In step S123, the original image data D1 is input to the constructed extraction model M, and extracted image data D3 is created by the extraction model M. Specifically, the original image data D1 is input to the encoder of the extraction model M. Next, the decoder of the extraction model M generates a segmentation map D7, an example of which is shown in FIG. 5. Next, the decoder of the extraction model M outputs extracted image data D3, an example of which is shown in FIG. 6, created based on the segmentation map D7.

[0038] In the segmentation map D7 shown in Figure 5, the white parts indicate areas where string-like structures exist in the original image data D1. This segmentation map D7 is data generated during data processing of the original image data D1 using the extraction model M. In this segmentation map D7, the areas where string-like structures exist and the background are each separated as separate objects based on the contrast in the original image data D1.

[0039] In the extracted image data D3 shown in Figure 6, the white parts represent the string-like structures that are expressed in contrast in the original image data D1, i.e., the molecular chains that make up the network structure of cross-linked rubber. In this extracted image data D3, appropriate string-like structures corresponding to the shape and size of each region in the segmentation map D7 are clearly depicted.

[0040] In the analysis step S130, the calculation device 3 executes data processing to analyze the internal structure of the crosslinked rubber using the extracted image data D3. In this data processing, the feature amount of the string-like structure (molecular chain) of the extracted image data D3 is calculated, and predetermined properties of the crosslinked rubber are determined based on the feature amount.

[0041] For example, at least one of the distribution density of molecular chains and the proportion of molecular chains by length is calculated as a feature quantity. The feature quantities that can be calculated include variations in the size of the mesh portion of the network structure formed by the molecular chains, the number of branch points in a single molecular chain, and the branch length from the branch point to the branch end. Using feature quantities such as the distribution density of molecular chains and variations in the size of the mesh portion of the network structure as indicators, the uniformity of the crosslinking state of the crosslinked rubber can be grasped. Using feature quantities such as the proportion of molecular chains by length, the number of branch points in a single molecular chain, and the branch length from the branch point to the branch end as indicators, the tensile properties (elongation) of the crosslinked rubber, for example, can be grasped.

[0042] The heterogeneity of the network structure can also be calculated as a feature quantity. For example, a dispersion index indicating the distribution and length dispersion of molecular chains can be calculated as a feature quantity. This dispersion index uses so-called arithmetic dispersion, standard deviation, etc. In addition, the enhanced image data D4 can be divided into multiple regions, and the dispersion index of the molecular chain distribution density and the representative value (median, average, etc.) of the molecular chain length for each region can be calculated as a feature quantity. Note that the heterogeneity of the crosslinked network structure can also be calculated as a feature quantity using known persistent homology analysis. In this way, calculating the heterogeneity of the crosslinked network structure as a feature quantity provides a useful index for improving the physical properties of vulcanized rubber, such as viscoelasticity and tensile properties.

[0043] As described above, according to this embodiment, the extraction model M constructed by machine learning using training data D4 analyzes and evaluates the influence of contrast in the original image data D1 on the string-like structure of the polymer material. Therefore, the extracted image data D3 created using the extraction model M accurately extracts changes in the shape and size of the string-like structure based on contrast differences in the original image data D1, even if the difference is small. Using this extracted image data D3 in the analysis step S130 is advantageous for more faithfully understanding the internal structure of the polymer material being analyzed.

[0044] In the extracted image data D3, in which the string-like structure is extracted with high accuracy, the string-like structure, which is closely related to understanding the characteristics of the crosslinked rubber, becomes clear, and therefore the feature quantities of the string-like structure can be understood with higher accuracy in the analysis step S130 using the extracted image data D3, which is advantageous for accurately understanding the characteristics of the polymer material that is the object of analysis.

[0045] In the procedure of the modified example of this embodiment shown in Fig. 7, step S124 is added to the procedure shown in Fig. 3. Furthermore, in this procedure, step S123 described above is changed to steps S123a and S123b in accordance with the addition of step S124. Steps S123a and S123b are processes that use an extraction model M, just like step S123 described above. However, steps S123a and S123b differ in that the extraction model M is used to create multiple scaled extracted image data D8a, D8b, and D8c, and string-like structures of different thicknesses (width dimensions in the direction perpendicular to the extension direction of the string-like structures) are extracted to create extracted image data D3d, as shown in Fig. 15. Specifically, multiple scaled image data D1a, D1b, and D1c are generated by scaling the original image data D1 to different image sizes (S124). Next, multiple scaled extracted image data D8a, D8b, D8c are generated by processing the scaled image data D1a, D1b, D1c using the extraction model M (S123a). Next, one extracted image data D3d is generated using the generated scaled extracted image data D8a, D8b, D8c (S123b). The contents of each step (S124, S123a, S123b) in the procedure of this modified example are described in detail below.

[0046] In step S124, the arithmetic unit 3 performs image processing to change the image size of the original image data D1, thereby generating multiple pieces of enlarged or reduced image data D1a, D1b, and D1c with different image sizes. The number of enlarged or reduced image data D1a to D1c generated is, for example, between two and five. In this embodiment, the enlarged or reduced image data D1a to D1c includes enlarged or reduced image data D1a with an image size smaller than that of the original image data D1, enlarged or reduced image data D1b with the same image size as that of the original image data, and enlarged or reduced image data D1c with an image size larger than that of the original image data D1. The magnification of the image size of the enlarged or reduced image data D1a with respect to the image size (original image size) of the original image data D1 is set to a value within a range of 0.5 times or more and less than 1 time. The magnification of the image size of the enlarged or reduced image data D1c with respect to the image size (original image size) of the original image data D1 is set to a value within a range of more than 1 time and less than 2 times. The scaled image data D1b may have an image size that can be considered the same as the image size of the original image data D1, or the magnification of the image size of the scaled image data D1b with respect to the image size of the original image data D1 (original image size) may be set within a range of 0.7 to 1.5 times, different from the magnification of the image size of the scaled image data D1a and D1c. However, it is optimal that the scaled image data D1b has the same image size as the original image data D1, and that the multiple expanded image data D1 to D1c include image data with an image size that is magnified equally to the image size of the original image data D1.

[0047] In step S123a, each of the scaled image data D1a to D1c is input to extraction model M, and each of the scaled extracted image data D8a, D8b, and D8c is output from extraction model M. Specifically, each of the scaled image data D1a to D1c is input to the encoder of extraction model M. Next, the decoder of extraction model M generates segmentation maps D7a, D7b, and D7c shown in FIGS. 8 to 10. Next, the decoder of extraction model M outputs scaled extracted image data D8a, D8b, and D8c shown in FIGS. 11 to 13, which are created based on each of the segmentation maps D7a to D7c.

[0048] In each of the segmentation maps D7a to D7c illustrated in Figures 8 to 10, the white portions indicate the regions in which string-like structures exist in each of the scaled image data D1a to D1c. Comparing the segmentation maps D7a to D7c, the smaller the image size, the larger each region becomes and the fewer the total number of regions, while the larger the image size, the smaller each region becomes and the more the total number of regions. The expansion or contraction of a region does not mean that the region expands or contracts in the direction in which the string-like structures exist in that region, but rather in a direction perpendicular to the extension direction.

[0049] In each of the scaled extracted image data D8a to D8c illustrated in Figures 11 to 13, the white portions indicate string-like structures that are expressed by the contrast in each of the scaled extracted image data D1a to D1c. Comparing each of the scaled extracted image data D8a to D8c, the larger the image size, the larger the thickness of the string-like structures (the width dimension in the direction perpendicular to the extension direction) and the fewer the overall number of string-like structures, and the larger the image size, the smaller the thickness of the string-like structures and the greater the overall number of string-like structures.

[0050] For example, area B enclosed by a white square in Figures 8 to 13 indicates the same area in each of segmentation maps D7a to D7c and each of scaled extracted image data D8a to D8c. Of the regions present in area B, segmentation map D7a is the largest and segmentation map D7c is the smallest. Accordingly, the thickness of each string-like structure present in area B is the largest in scaled extracted image data D8a and the smallest in scaled extracted image data D8c.

[0051] In step S123b, the arithmetic unit 3 executes data processing to create extracted image data D3d using the generated scaled extracted image data D8a-D8c. Specifically, the original image data D1 on which the scaled extracted image data D8a-D8c are based is input to the decoder of the extraction model M without changing the image size. Next, the decoder of the extraction model M outputs a segmentation map D7d as shown in FIG. 14. Next, string-like structures are generated based on each of the scaled extracted image data D8a-D8c so as to fit into each region of the segmentation map D7d, and the extracted image data D3d as shown in FIG. 15 is created.

[0052] Examples of methods for creating extracted image data D3d using the scaled extracted image data D8a-D8c include combining the data and using a new extraction model constructed by machine learning using the data as training data. In the data combining method, the largest pixel value among the pixels at the same position as the pixel in the scaled extracted image data D8a-D8c corresponding to each region of the segmentation map D7d is selected. This results in extracted image data D3d from which a string-like structure matching each region of the segmentation map D7d is extracted. In the method using a new extraction model, a new extraction model is constructed by machine learning using the segmentation maps D7a-D7b and the scaled extracted image data D8a-D8c as new training data different from the training data D4. This machine learning can be the same as the machine learning (U-net) used in step S123a. Next, the original image data D1 is input to the new extraction model, and extracted image data D3d is output from the new extraction model.

[0053] In the segmentation map D7d shown in Fig. 14, the white parts indicate areas where string-like structures exist in the original image data D1. In the extracted image data D3d shown in Fig. 15, the white parts indicate string-like structures that are expressed by the contrast of the original image data D1. In this extracted image data D3d, string-like structures of different thicknesses are extracted.

[0054] As described above, in this modified example, the image size of the original image data D1 is changed, and multiple scaled image data D1a-D1c with different image sizes are input into the extraction model M to obtain scaled extracted image data D8a-D8c, which are then used to create a single piece of extracted image data D3d. Because string-like structures of different thicknesses (width dimensions) are extracted from each of the scaled extracted image data D8a-D8c, string-like structures of different thicknesses are extracted in the extracted image data D3d created using these scaled extracted image data. Therefore, because this extracted image data D3d extracts string-like structures that are more in line with the internal structure of the actual polymer material, using this extracted image data D3d in the analysis step S130 allows for a more faithful understanding of the internal structure of the polymer material being analyzed.

[0055] It is advisable to add the extracted image data D3, D3d created in the above-described embodiments and modifications, and the segmentation maps D7, D7d obtained in the process of creating them, to the training data D4 and expand the training data D4 with each analysis. This allows the next and subsequent analyses to use the extraction model M constructed by machine learning using the training data D4 expanded with the addition of such data, thereby enabling the string-like structures to be extracted with higher accuracy.

[0056] The extraction model M constructed in step S122 does not need to be reconstructed unless the training data D4 is changed. Therefore, the constructed extraction model M can be stored in advance in the auxiliary storage unit 6 of the calculation device 3, and the pre-stored extraction model M can be used in subsequent analyses, thereby omitting step S122.

[0057] When constructing the extraction model M, it is advisable to select appropriate data as the training data D4 to be used depending on the state of the original image data D1. For example, if the contrast of the original image data D1 is clear and the area where the string-like structure exists is as clear as the segmentation map, there is no need to generate a segmentation map, and only the similar image data D6 can be selected as the training data D4. Furthermore, if a large number of segmentation maps and extracted image data obtained by repeatedly performing this embodiment or its modified examples have been added to the training data D4, it is possible to select only those segmentation maps and extracted image data that have been added as the training data D4.

[0058] When inputting the original image data D1 into the extraction model M, the calculation device 3 may perform image processing on the original image data D1 to increase the similarity with the training data D4. Specifically, a representative value of a predetermined parameter of the training data D4 is calculated. Next, adjusted image data is generated by adjusting the predetermined parameter of the original image data D1 to the calculated representative value. Next, the adjusted image data is processed using the extraction model M to create extracted image data D3. The predetermined parameter can be selected from parameters that indicate the characteristics of the image data, but is preferably a histogram. Therefore, when inputting the original image data D1 into the extraction model M, the calculation device 3 performs histogram matching to generate adjusted image data so that the histogram of the original image data D1 matches the representative value of the histogram of the training data D4, and the generated adjusted image data is input into the extraction model M.

[0059] The present invention is not limited to a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention.

[0060] The present disclosure encompasses the following inventions. Invention 1: 1. A method for analyzing the internal structure of a polymer material, using original image data of the polymer material acquired by an electron microscope, by a computing device to analyze the internal structure of the polymer material, an image acquisition step of acquiring the original image data using the electron microscope; a pre-processing step of creating extracted image data in which string-like structures represented by contrast in the original image data are extracted; an analysis step of calculating a feature amount of the string-like structure using the extracted image data and grasping a predetermined property of the polymer material based on the feature amount; A method for analyzing the internal structure of polymeric materials, in which an extraction model linking the relationship between contrast in image data and string-like structures is constructed using supervised machine learning, and in the preprocessing step, the extracted image data is created by processing the original image data using the extraction model. Invention 2: A method for analyzing the internal structure of a polymer material according to Invention 1, in which a crosslinked rubber is swelled to fix a network structure formed by molecular chains, and this crosslinked rubber is used as the polymer material, and the molecular chains are extracted as the string-like structure in the enhanced image data. Invention 3: The method for analyzing the internal structure of a polymer material according to invention 1 or 2, wherein the pre-processing step involves performing image processing on the original image data to generate processed image data in which structures other than the string-like structures are reduced, and then processing the processed image data using the extraction model to create the extracted image data. Invention 4: A method for analyzing the internal structure of a polymer material according to any one of Inventions 1 to 3, in which, in the preprocessing step, representative values ​​of predetermined parameters of the training data used in the supervised machine learning are calculated, and adjusted image data is generated in which the predetermined parameters of the original image data are adjusted to the representative values, and the extracted image data is created by data processing the adjusted image data using the extraction model. Invention 5: A method for analyzing the internal structure of a polymer material described in any one of Inventions 1 to 4, in which, in the pre-processing step, image processing is performed to change the image size of the original image data to generate multiple scaled image data with different image sizes, each of the scaled image data is processed using the extraction model to generate multiple scaled extracted image data, and one extracted image data is created using each of the generated scaled extracted image data. Invention 6: A method for analyzing the internal structure of a polymer material according to any one of Inventions 1 to 5, in which a different object made of non-rubber and non-resin is photographed, and similar image data having a string-like structure is used as training data for the supervised machine learning. Invention 7: A method for analyzing the internal structure of a polymer material according to any one of Inventions 1 to 6, in which a polymer material of a different type from the polymer material is photographed and image data of a different type of polymer obtained by an electron microscope is used as training data for the supervised machine learning. Invention 8: The method for analyzing the internal structure of a polymer material according to Invention 7, which cites Invention 6, wherein in constructing the extraction model, a histogram of the similar image data is approximated to a histogram of the different polymer image data. Invention 9: A method for analyzing the internal structure of a polymer material according to any one of Inventions 1 to 8, wherein the analysis step calculates at least one of the distribution density of the string-like structures and the proportion of the string-like structures by size as the characteristic quantity. Invention 10: A method for analyzing the internal structure of a polymer material described in any one of Inventions 1 to 8, in which the analysis step calculates a dispersion index indicating the dispersion state of at least one of the distribution of the string-like structures and the size of the string-like structures as the feature. Invention 11: A system for analyzing the internal structure of a polymer material, comprising a storage unit for storing original image data of a polymer material acquired by an electron microscope, and a computing device for analyzing the internal structure of the polymer material using the original image data, The arithmetic device executes a preprocessing step of creating extracted image data in which string-like structures represented by contrast in the original image data are extracted, and an analysis step of calculating feature amounts of the string-like structures using the extracted image data and grasping predetermined characteristics of the polymer material based on the feature amounts, An internal structure analysis system for polymeric materials in which an extraction model linking the relationship between contrast in image data and string-like structures is constructed using supervised machine learning and stored in the memory unit, and in the preprocessing step, the original image data is input to the extraction model, and the extracted image data is output by the extraction model. [Explanation of symbols]

[0061] 1. Internal structure analysis system 2. Electron microscope 3 Computing device 4. Processing unit 5 Main memory 6 Auxiliary storage 7 Input section 8 Output section D1 Original image data D1a~D1c Enlarged / reduced image data D2 processed image data D3, D3d Extracted image data D4 Training data D5 Different polymer image data D6 Similar image data D7, D7a-D7d Segmentation maps D8a~D8c Scaled extracted image data

Claims

1. 1. A method for analyzing the internal structure of a polymer material, using original image data of the polymer material acquired by an electron microscope, by a computing device to analyze the internal structure of the polymer material, an image acquisition step of acquiring the original image data using the electron microscope; a pre-processing step of creating extracted image data in which string-like structures represented by contrast in the original image data are extracted; an analysis step of calculating a feature amount of the string-like structure using the extracted image data and grasping a predetermined property of the polymer material based on the feature amount; A method for analyzing the internal structure of polymeric materials, in which an extraction model linking the relationship between contrast in image data and string-like structures is constructed using supervised machine learning, and in the preprocessing step, the extracted image data is created by processing the original image data using the extraction model.

2. 2. The method for analyzing the internal structure of a polymer material according to claim 1, wherein a crosslinked rubber is swollen to fix a network structure formed by molecular chains, and this crosslinked rubber is used as the polymer material, and the molecular chains are extracted as the string-like structure in the enhanced image data.

3. The method for analyzing the internal structure of a polymer material according to claim 1, wherein the pre-processing step involves performing image processing on the original image data to generate processed image data in which structures other than the string-like structures are reduced, and then creating the extracted image data by processing the processed image data using the extraction model.

4. The method for analyzing the internal structure of a polymer material according to claim 1, wherein the preprocessing step calculates representative values ​​of predetermined parameters of the training data used in the supervised machine learning, generates adjusted image data in which the predetermined parameters of the original image data are adjusted to the representative values, and creates the extracted image data by data processing the adjusted image data using the extraction model.

5. The method for analyzing the internal structure of a polymer material described in claim 1, wherein in the pre-processing step, image processing is performed to change the image size of the original image data to generate multiple scaled image data with different image sizes, each of the scaled image data is processed using the extraction model to generate multiple scaled extracted image data, and one extracted image data is created using each of the generated scaled extracted image data.

6. A method for analyzing the internal structure of a polymer material as described in claim 1, in which the subject of photography is a different object made of non-rubber and non-resin, and similar image data having a string-like structure is used as training data for the supervised machine learning.

7. A method for analyzing the internal structure of a polymer material as described in claim 1 or 6, in which a polymer material of a different type from the polymer material is photographed, and image data of a different type of polymer obtained by an electron microscope is used as training data for the supervised machine learning.

8. The method for analyzing the internal structure of a polymer material according to claim 7, which cites claim 6, wherein in constructing the extraction model, a histogram of the similar image data is approximated to a histogram of the different polymer image data.

9. 3. The method for analyzing the internal structure of a polymer material according to claim 1, wherein the analysis step calculates at least one of the distribution density of the string-like structures and the proportion of the string-like structures by size as the feature amount.

10. The method for analyzing the internal structure of a polymer material according to claim 1 or 2, wherein the analysis step calculates a dispersion index indicating the dispersion state of at least one of the distribution of the string-like structures and the size of the string-like structures as the feature.

11. A system for analyzing the internal structure of a polymer material, comprising a storage unit for storing original image data of a polymer material acquired by an electron microscope, and a computing device for analyzing the internal structure of the polymer material using the original image data, The arithmetic device executes a preprocessing step of creating extracted image data in which string-like structures represented by contrast in the original image data are extracted, and an analysis step of calculating feature amounts of the string-like structures using the extracted image data and grasping predetermined characteristics of the polymer material based on the feature amounts, An internal structure analysis system for polymeric materials in which an extraction model linking the relationship between contrast in image data and string-like structures is constructed using supervised machine learning and stored in the memory unit, and in the preprocessing step, the original image data is input to the extraction model, and the extracted image data is output by the extraction model.

Images