A structure extraction method for non-stationary texture structural attributes

Abstract

The present invention discloses a structure extraction method for non-stationary texture structural attributes, which extracts the structural information of texture examples whose structural attributes in non-stationary textures are presented as feature contours and element distributions, and obtains a texture structure map of the non-stationary texture. Specific steps: 1) For texture examples whose structural attributes are presented as feature contours, obtain the boundary line of the example feature contour structure, determine other feature contour points within the neighborhood of the boundary line, and thus obtain a texture structure map; 2) For texture examples whose structural attributes are presented as element distributions, perform local image block matching based on element example blocks, extract texture elements within the blocks, and thus obtain a texture structure map. In texture synthesis and texture migration tasks, the structural attributes of non-stationary textures have always been a research difficulty. Compared with the prior art, the present invention can extract the structural attributes of non-stationary texture images, provide a new solution for texture-related tasks, and has certain application prospects.

Description

Technical Field

[0001] The present invention relates to the field of texture synthesis for non-stationary textures, and in particular to a structure extraction method for non-stationary texture structural attributes. Background Art

[0002] Texture synthesis analyzes an input texture image and generates a new, larger-scale target texture image. The target texture must be consistent with the input texture image in both visual and structural features. In the field of computer graphics, example-based texture synthesis has achieved excellent results in visual features, but structural features remain a significant challenge, especially for non-stationary textures with large-scale structural features.

[0003] With the continuous development of deep learning, deep learning technology began to be applied to example-based texture synthesis in 2016, and deep learning-based texture synthesis methods began to flourish. As a key technology in the field of computer graphics, texture synthesis can provide texture content-related support for texture mapping, texture stitching, texture reconstruction, texture recognition, and other applications.

[0004] Existing deep learning-based texture synthesis techniques lack the ability to analyze and utilize the structural properties of non-stationary textures, and neural network models struggle to accurately extract the structural features of non-stationary texture images. While some researchers have investigated structural information, most focus on a small number of specific texture structures. This results in high computational complexity, long inference times, and models that typically only address specific structural issues within a single texture, such as rust textures, resulting in poor scalability. Summary of the Invention

[0005] The purpose of this invention is to provide a method for extracting structural properties from non-stationary textures, thereby addressing the difficulty in extracting and learning structural features from non-stationary textures during texture synthesis. This method can handle the two largest types of texture examples, extracting them in less than one minute. This method effectively addresses the difficulty in acquiring structural features in texture tasks, providing a more effective solution for texture synthesis and texture structure transfer for non-stationary textures, and has promising applications.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A structure extraction method for non-stationary texture structural attributes is characterized by performing structure extraction by category based on different structural attributes presented by non-stationary texture examples. The method specifically includes the following steps:

[0008] Step 1: Structure extraction of texture samples with characteristic contour structures

[0009] 1.1: For texture examples whose structural attributes are represented by characteristic contours, the size of the Gaussian blur kernel is determined based on the Euclidean distance between the characteristic contours in the texture examples. The texture examples are then convolved with a Gaussian filter with a hyperparameter blur kernel to remove noise information from the texture examples. The convolution operation is then repeated with a Gaussian filter with a fixed blur kernel size of 5×5 to obtain the filtered texture examples.

[0010] 1.2: Calculate the gradient magnitude and gradient direction of the texture example and perform non-maximum suppression on the gradient magnitude along the gradient direction. Non-maximum suppression compares the pixel value of the center pixel with the pixel values of the two surrounding pixels along the corresponding gradient direction. If the pixel value of the center pixel is the maximum value, the original pixel value of the center pixel is retained; otherwise, it is set to 0.

[0011] 1.3: Set the hyperparameter high and low thresholds to detect and connect the edges of the feature contour structure: pixels less than the set low threshold are set to 0, pixels greater than the set high threshold are set to 1, and pixels less than the set high threshold, greater than the set low threshold, and connected to pixels greater than the set high threshold are set to 1. All pixels set to 1 are further defined as boundary pixels of the feature contour structure;

[0012] 1.4: Traverse the boundary pixels of the characteristic contour structure, search for other boundary pixels within the set hyperparameter Euclidean distance threshold from the 8-neighborhood direction, and reset the values of all pixels between the two boundary pixels to 1 along the search direction. Define all pixels set to 1 as the characteristic contour structure; an image composed of pixel values 0 and pixel values 1 is a texture structure map of a texture example with a characteristic contour structure;

[0013] Step 2: Structure extraction of texture samples with element distribution structure

[0014] 2.1: For texture examples whose structural properties are presented as element distributions, classify them into different categories based on the size and color differences of the texture elements, and provide an element example for each category of elements. That is, for each category of texture elements in the texture example, arbitrarily intercept an image block containing a single texture element of this category;

[0015] 2.2: Based on the provided element example, traverse the local area of the texture example that is equal to the element example and calculate the similarity value between the element example and the local area. The local area block with a similarity value higher than the given similarity threshold is defined as the initial element block; the similarity threshold setting is inconsistent for different categories. The similarity threshold is a hyperparameter that needs to be adjusted. Set the similarity threshold to 0.15-0.4;

[0016] 2.3: In the initial element blocks, the element blocks with an overlap greater than the set overlap threshold of 0.5 are screened: if the two element blocks belong to the same category, the element block with the higher similarity value is retained; otherwise, the element block with the larger area is retained. The element blocks that are finally retained are uniformly marked as target element blocks;

[0017] 2.4: Use the saliency detection model trained on 10,553 images in the DUTS-TR dataset to extract the texture elements contained in the target element block. The extracted texture elements are stitched together according to their original absolute positions in the element block to obtain a texture structure map of the texture example with the element distribution structure.

[0018] Calculate the gradient magnitude and gradient direction of the texture sample as described in step 1.2. The specific steps are:

[0019] i) convolving the texture sample with the convolution array in the Sobel filter to obtain the horizontal gradient value and the vertical gradient value;

[0020] ii) For the gradient values in the horizontal and vertical directions, the gradient magnitude of the texture example is calculated using the square root of the sum of squares, and the gradient direction of the texture example is calculated using the inverse tangent function.

[0021] As described in step 1.4, search for other boundary pixels within the set hyperparameter Euclidean distance threshold from the 8 neighborhood directions. The specific steps are:

[0022] i) For the texture example with tilted characteristic contour structure, we first search along the directions of 45°, 135°, 225°, and 315°, and then supplement the search along the directions of 0°, 90°, 180°, and 270°;

[0023] ii) For texture images with horizontal or vertical contour lines, it is only necessary to search from the directions of 0°, 90°, 180°, and 270° respectively.

[0024] The similarity value between the element example and the local area is calculated in step 2.2. The specific steps are:

[0025] i) subtracting the pixel mean from the element example block and the local area block involved in the calculation and dividing by the pixel variance;

[0026] ii) Multiplying the pixel values at the same position of the element example block and the local area block respectively and then accumulating the sum, and further obtaining the similarity value between the element example block and the local area block.

[0027] As described in step 2.3, the element blocks whose overlap degree between the two element blocks is higher than the set overlap threshold of 0.5 are screened. The overlap degree is calculated as the ratio of the intersection of the two element blocks to the union of the two element blocks.

[0028] Compared with the existing technology, the present invention can extract structural properties in non-stationary texture images, and extract structural features in the two types of textures that account for the largest proportion in texture images under light user interaction: feature contour structure and element distribution structure, without the need for complex calculations or time-consuming model training based on a single texture. That is, structural features can be extracted for any texture example in no more than 1 minute. Therefore, it has certain application prospects in the fields of texture synthesis and texture structure migration of non-stationary textures. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flowchart of the present invention;

[0030] Figure 2 This is a flow chart of feature contour structure extraction of the present invention;

[0031] Figure 3 This is a flow chart for extracting the element distribution structure of the present invention;

[0032] Figure 4 This is an example result diagram of texture of an embodiment of the invention. DETAILED DESCRIPTION

[0033] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0034] The present invention proposes a structure extraction method for non-stationary texture structural attributes. It proposes corresponding structure extraction schemes for the two types of texture structural attributes that account for the largest proportion in non-stationary texture datasets. With minimal human interaction, the method does not require complex calculations or time-consuming model training, and can obtain the structural features of texture examples in just one minute. The method specifically includes the following steps:

[0035] Step 1: Structure extraction of texture samples with characteristic contour structures

[0036] Step 2: Structure extraction of texture examples with element distribution structure.

[0037] See Figure 1 and Figure 4 , combining steps 1 and 2 to complete the structural attribute extraction of the input non-stationary texture example, and obtain a texture structure map containing texture structural attributes, specifically including:

[0038] S100-S140: Extracting the characteristic contour structure of non-stationary texture examples with structural attributes such as the vein contour in leaf texture, see Figure 2 ;

[0039] S200-S240: Extracting the element distribution structure in non-stationary texture examples whose structural attributes are element distribution, such as the feather spot distribution in peacock feather texture, see Figure 3 .

[0040] According to the common structural properties in existing non-stationary textures, the present invention summarizes two common structural types: feature contour structure and element distribution structure. For any input non-stationary texture example, the structural type to which the structural properties contained in the texture example belong is determined, and the corresponding structure extraction method is selected to complete the structure extraction, and further obtain the texture structure diagram of the non-stationary texture. Starting from the structural properties of the non-stationary texture, the present invention extracts the structural features in the non-stationary texture under lightweight user interaction. In the extraction stage, there is no need to perform complex mathematical calculations and time-consuming model operations on the texture example. The extraction of feature contour structure or element distribution structure operations can be completed within 1 minute. The structural features of non-stationary textures can play a role of structural supervision in non-stationary texture synthesis and texture structure migration tasks, and have certain application prospects.

[0041] Example 1

[0042] See Figure 2 In step 1 of this embodiment, a study is conducted on a texture whose structural attribute is a characteristic contour structure in an input non-stationary texture example, and the characteristic contour structure of the texture is extracted using methods such as Gaussian filtering, edge detection, and neighborhood search. The specific steps are as follows:

[0043] S100: For a texture example whose structural attributes are presented as characteristic contours, such as veins in a leaf texture example, determine the size of a Gaussian blur kernel according to the Euclidean distance between the characteristic contours in the texture example.

[0044] S110: performing a convolution operation on the texture example using a Gaussian filter with a hyperparameter blur kernel to remove noise information in the texture example, and then performing convolution again using a Gaussian filter with a fixed blur kernel size of 5×5 to obtain a filtered texture example.

[0045] S120: Calculate the gradient magnitude and gradient direction of the texture example: 1) Convolve the texture example using the convolution array in the Sobel filter to obtain the horizontal gradient value and the vertical gradient value; 2) For the horizontal and vertical gradient values, calculate the gradient magnitude of the texture example using the square root of the sum of squares, and calculate the gradient direction of the texture example using the inverse tangent function. Perform non-maximum suppression on the gradient magnitude along the gradient direction: Compare the pixel value of the center pixel with the pixel values of the two surrounding pixels along the corresponding gradient direction. If the pixel value of the center pixel is the maximum value, retain the original pixel value of the center pixel; otherwise, set it to 0.

[0046] S130: Set the hyperparameter high and low thresholds to detect and connect the edges of the feature contour structure: set the pixel points smaller than the set low threshold to 0, set the pixel points larger than the set high threshold to 1, set the pixel points smaller than the set high threshold, larger than the set low threshold and connected to the pixel points larger than the set high threshold to 1, and further define all the pixel points set to 1 as the boundary pixel points of the feature contour structure.

[0047] S140: Traverse the boundary pixels of the characteristic contour structure, search for other boundary pixels within the set hyperparameter Euclidean distance threshold from the 8-neighborhood direction, and reset the values of all pixels between the two boundary pixels to 1 along the search direction. All pixels set to 1 are defined as the characteristic contour structure. An image composed of pixels with pixel values of 0 and 1 is a texture structure image of a texture example with a characteristic contour structure. The specific steps for searching for other boundary pixels within the set hyperparameter Euclidean distance threshold from the 8-neighborhood direction are: 1) For texture examples with tilted characteristic contour structures, first search along the directions of 45°, 135°, 225°, and 315°, and then supplement the search along the directions of 0°, 90°, 180°, and 270°; 2) For texture images with horizontal or vertical contour lines, only the directions of 0°, 90°, 180°, and 270° need to be searched respectively.

[0048] See Figure 3 In step 2 of this embodiment, a study is conducted on textures whose large-scale structural properties are represented by element distribution structures in the input non-stationary texture examples. Methods such as element example selection, element block matching, and target element detection are used to extract the element distribution structure of the texture. The specific steps are as follows:

[0049] S200: For a texture example whose structural attributes are presented as element distribution, such as the distribution relationship of different texture elements on the texture of a peacock feather, the texture elements are divided into different categories according to their size and color differences, and an element example is provided for each category of elements. That is, an image block containing a single texture element of this category is arbitrarily intercepted for each category of texture elements in the texture example.

[0050] S210: Based on the provided element example, traverse the local area of the texture example that is equal to the size of the element example, and calculate the similarity value between the element example and the local area. The local area block with a similarity value higher than a given similarity threshold is defined as the initial element block. The similarity threshold settings for different categories are inconsistent. The similarity threshold is a hyperparameter that needs to be adjusted. In this embodiment, the lower limit of the similarity threshold is set to 0.15, and the upper limit of the similarity threshold is set to 0.4. The specific steps for calculating the similarity value between the element example and the local area are: 1) subtract the pixel mean of the element example block and the local area block involved in the calculation, and divide by the pixel variance; 2) multiply the pixel values at the same position of the element example block and the local area block, and then accumulate and sum them to further obtain the similarity value between the element example block and the local area block.

[0051] S220: In the initial element blocks, the element blocks whose overlap degree between two element blocks is higher than the set overlap threshold value of 0.5 are screened: if the two element blocks belong to the same category, the element block with the higher similarity value is retained; otherwise, the element block with a larger area is retained, and the element blocks finally retained are uniformly marked as target element blocks; the overlap degree is calculated as the ratio of the intersection of the two element blocks to the union of the two element blocks.

[0052] S230: Utilize the saliency detection model to extract texture elements contained within the target element block. The saliency detection model utilizes the U²-Net framework, trained on the open-source DUTS-TR dataset using open-source code and default parameters. After training, the target element blocks are fed into the saliency detection model one by one, along with the corresponding texture elements within the target element block.

[0053] S240: stitching the extracted texture elements according to the original absolute positions of the target element blocks to obtain a texture structure map of a texture example having an element distribution structure. The original absolute positions are the coordinate positions of the target element blocks in the texture example.

[0054] The above is only a further explanation of the present invention and is not intended to limit the present invention. Any equivalent implementation of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for extracting structure attributes from non-stationary textures, characterized in that structure extraction is performed by category based on the different structural attributes presented by non-stationary texture examples. The method specifically comprises the following steps: Step 1: Structure extraction of texture samples with characteristic contour structures 1.1: For texture examples whose structural attributes are represented by characteristic contours, the size of the Gaussian blur kernel is determined based on the Euclidean distance between the characteristic contours in the texture examples. The texture examples are then convolved with a Gaussian filter with a hyperparameter blur kernel to remove noise information from the texture examples. The convolution operation is then repeated with a Gaussian filter with a fixed blur kernel size of 5×5 to obtain the filtered texture examples. 1.2: Calculate the gradient magnitude and gradient direction of the texture example and perform non-maximum suppression on the gradient magnitude along the gradient direction. Non-maximum suppression compares the pixel value of the center pixel with the pixel values of the two surrounding pixels along the corresponding gradient direction. If the pixel value of the center pixel is the maximum value, the original pixel value of the center pixel is retained; otherwise, it is set to 0. 1.3: Set the hyperparameter high and low thresholds to detect and connect the edges of the feature contour structure: pixels less than the set low threshold are set to 0, pixels greater than the set high threshold are set to 1, and pixels less than the set high threshold, greater than the set low threshold, and connected to pixels greater than the set high threshold are set to 1. All pixels set to 1 are further defined as boundary pixels of the feature contour structure; 1.4: Traverse the boundary pixels of the characteristic contour structure, search for other boundary pixels within the set hyperparameter Euclidean distance threshold from the 8-neighborhood direction, and reset the values of all pixels between the two boundary pixels to 1 along the search direction. Define all pixels set to 1 as the characteristic contour structure; an image composed of pixel values 0 and pixel values 1 is a texture structure map of a texture example with a characteristic contour structure; Step 2: Structure extraction of texture samples with element distribution structure 2.1: For texture examples whose structural properties are presented as element distributions, classify them into different categories based on the size and color differences of the texture elements, and provide an element example for each category of elements. That is, for each category of texture elements in the texture example, arbitrarily intercept an image block containing a single texture element of this category; 2.2: Based on the provided element example, traverse the local area of the texture example that is equal to the element example in size, and calculate the similarity value between the element example and the local area. The local area block with a similarity value higher than the given similarity threshold is defined as the initial element block; set the similarity threshold to 0.15-0.4; 2.3: In the initial element blocks, the element blocks with an overlap greater than the set overlap threshold of 0.5 are screened: if the two element blocks belong to the same category, the element block with the higher similarity value is retained; otherwise, the element block with the larger area is retained. The element blocks that are finally retained are uniformly marked as target element blocks; 2.4: Use the saliency detection model trained on 10,553 images in the DUTS-TR dataset to extract the texture elements contained in the target element block. The extracted texture elements are stitched together according to their original absolute positions in the element block to obtain a texture structure map of the texture example with the element distribution structure.

2. The method for extracting non-stationary texture structural attributes according to claim 1, wherein the step of calculating the gradient magnitude and gradient direction of the texture example in step 1.2 comprises the following steps: i) convolving the texture sample with the convolution array in the Sobel filter to obtain the horizontal gradient value and the vertical gradient value; ii) For the gradient values in the horizontal and vertical directions, the gradient magnitude of the texture example is calculated using the square root of the sum of squares, and the gradient direction of the texture example is calculated using the inverse tangent function.

3. The structure extraction method for non-stationary texture structural attributes according to claim 1, characterized in that: As described in step 1.4, search for other boundary pixels within the set hyperparameter Euclidean distance threshold from the 8 neighborhood directions. The specific steps are: i) For the texture example with tilted characteristic contour structure, we first search along the directions of 45°, 135°, 225°, and 315°, and then supplement the search along the directions of 0°, 90°, 180°, and 270°; ii) For texture images with horizontal or vertical contour lines, it is only necessary to search from the directions of 0°, 90°, 180°, and 270° respectively.

4. The structure extraction method for non-stationary texture structural attributes according to claim 1, characterized in that: The similarity value between the element example and the local area is calculated in step 2.

2. The specific steps are: i) subtracting the pixel mean from the element example block and the local area block involved in the calculation and dividing by the pixel variance; ii) Multiplying the pixel values at the same position of the element example block and the local area block respectively and then accumulating the sum, and further obtaining the similarity value between the element example block and the local area block.

5. The structure extraction method for non-stationary texture structural attributes according to claim 1, characterized in that: As described in step 2.3, the element blocks whose overlap degree between the two element blocks is higher than the set overlap threshold of 0.5 are screened. The overlap degree is calculated as the ratio of the intersection of the two element blocks to the union of the two element blocks.

Images