A method for generating paleontological images based on neural networks

Abstract

This invention discloses a method for generating paleontological images based on neural networks, including acquiring paleontological image datasets and non-paleontological image datasets; preprocessing the paleontological image datasets and non-paleontological image datasets; performing viewpoint image segmentation on the preprocessed paleontological image datasets and non-paleontological image datasets; training a generative adversarial network (GAN) based on the segmentation results; using the trained GAN to generate a first-view image, and then stitching it with a second-view image obtained from the viewpoint image segmentation to generate a new paleontological image. This invention, by performing viewpoint image segmentation on the paleontological image datasets and non-paleontological image datasets and then training a GAN, can generate clear adversarial images while ensuring more complete image information.

Description

Technical Field

[0001] This invention relates to the field of paleontological image processing technology, and specifically to a method for generating paleontological images based on neural networks. Background Technology

[0002] Paleontologists are organisms that lived on Earth in ancient times, existing throughout geological periods of Earth's history. Most paleontologists are now extinct; only their remains, artifacts, and traces of life have been preserved by nature under specific conditions, undergoing petrification to form fossils. Paleontological fossils, as preserved traces of ancient organisms, are non-renewable natural heritage. Through paleontological datasets, humans can obtain information about the Earth's ecological environment in the distant past. Furthermore, paleontological datasets can effectively interface paleontology and data with computer science and database programming. With the development of the internet and the advancement of artificial intelligence, using machine learning techniques to study paleontological identification is also effective.

[0003] Currently established paleontology databases mainly fall into two categories. One category consists of collection and management databases, which contribute significantly more to popular science than to scientific research, and whose data quality is difficult to guarantee. The other category comprises research databases, whose research content mainly includes biogeography, paleoecology, and morphological evolution analysis. However, neither of these databases can meet the requirements of computer vision research. Summary of the Invention

[0004] To address the aforementioned shortcomings in the existing technology, this invention provides a method for generating paleontological images based on neural networks.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0006] A method for generating paleontological images based on neural networks includes the following steps:

[0007] S1. Obtain paleontological image datasets and non-paleontological image datasets;

[0008] S2. Preprocess the paleontological image dataset and the non-paleontological image dataset;

[0009] S3. Perform viewpoint image segmentation on the preprocessed paleontological image dataset and the non-paleontological image dataset, and train the generative adversarial network Style GAN based on the segmentation results.

[0010] S4. Use the trained generative adversarial network Style GAN to generate a first-view image, and stitch it with the second-view image obtained from the view image segmentation in step S3 to generate a new paleontological image.

[0011] Optionally, step S2 specifically includes the following sub-steps:

[0012] S21. The Frequency Tuned algorithm is used to perform Gaussian smoothing with a window size of 5×5 on the paleontological image dataset and the non-paleontological image dataset to remove high-frequency information in the image data.

[0013] S22. The Luminance Contrast algorithm is used to perform global contrast enhancement processing on the paleontological image dataset and the non-paleontological image dataset respectively.

[0014] Optionally, the formula for calculating the saliency of a pixel in the Frequency Tuned algorithm is:

[0015]

[0016] Among them, I μ This is the arithmetic mean of the image pixels. This is a Gaussian blur applied to the original image, where ||.|| is the Euclidean distance.

[0017] Optionally, the formula for calculating the saliency of a pixel in the Luminance Contrast algorithm is as follows:

[0018]

[0019] Among them, I k Let I be the pixel value of pixel k in image I. i This is the grayscale value.

[0020] Optionally, step S3 specifically includes the following sub-steps:

[0021] S31. Perform left-view image L, middle-view image C, and right-view image R segmentation on the preprocessed paleontological image dataset and the non-paleontological image dataset respectively to obtain n sets of three-view image datasets corresponding to the paleontological image dataset and the non-paleontological image dataset.

[0022] S32. Randomly select a set of three-view images from the three-view image dataset corresponding to the paleontological image dataset and the non-paleontological image dataset. Combine the left-view image L and the right-view image R into a pair of samples (L,R), and take the middle-view image C as the true value C of the pair of samples.

[0023] S33. Repeat the above operation to process the remaining (n-1) groups of three-view images in the three-view image dataset to obtain n pairs of samples (L,R) and n true values ​​C.

[0024] S34. Randomly divide the n pairs of samples (L,R) into a training sample set and a test sample set according to the proportion. Use the training sample set to train the constructed Generative Adversarial Network Style GAN, optimize the parameters of the Generative Adversarial Network Style GAN, and obtain the trained Generative Adversarial Network Style GAN. Use the test set to test the generator G of all trained Generative Adversarial Network Style GANs, evaluate the image inpainting performance of generator G, and select the optimal generator G.

[0025] Optionally, step S4 specifically includes:

[0026] The samples (L,R) in the test sample set are sequentially input into the optimal generator G to obtain the first-view image A. The first-view image A is then stitched to the left and right sides of the ground truth C to generate a new paleontological image.

[0027] The present invention has the following beneficial effects:

[0028] 1. This invention designs a method for generating a dataset, which can be used to generate other visual datasets.

[0029] 2. This invention is the first to collect and generate a paleontological image dataset, which can help researchers design artificial intelligence technologies for automatic paleontological identification. This paleontological dataset can be used to design and evaluate new network models, helping paleontologists to identify trace fossils more quickly.

[0030] 3. This invention trains a generative adversarial network, Style GAN, by performing viewpoint image segmentation on paleontological image datasets and non-paleontological image datasets, thereby generating clear adversarial images and ensuring more complete image information. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating a method for generating paleontological images based on neural networks, as described in an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram illustrating the process of extracting a dataset of paleontological images in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram illustrating the manual extraction of images from a thesis in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram illustrating the removal of manually marked image outlines in an embodiment of the present invention;

[0035] Figure 5 This is a schematic diagram of the acquisition process of non-paleontological image datasets in an embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of the preprocessing process for the paleontological image dataset in an embodiment of the present invention;

[0037] Figure 7 This is a schematic diagram illustrating the preprocessing process of non-paleontological image datasets in an embodiment of the present invention;

[0038] Figure 8 This is a schematic diagram of the training process of the Generative Adversarial Network Style GAN in an embodiment of the present invention. Detailed Implementation

[0039] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0040] like Figure 1 As shown, this embodiment of the invention provides a method for generating paleontological images based on neural networks, including the following steps S1 to S4:

[0041] S1. Obtain paleontological image datasets and non-paleontological image datasets;

[0042] In an optional embodiment of the present invention, this invention for the first time collects an image dataset of paleontological trace fossils (hereinafter referred to as the paleontological image dataset), preprocesses the original images through an attention mechanism—saliency detection—to highlight the fossils in the paleontological images, and generates paleontological images through a neural network model. The newly generated paleontological dataset can effectively combine paleontology and computer vision techniques from the fields of deep learning.

[0043] The difficulties in identifying paleontological images under a microscope include: (1) Paleontological remains are almost non-distributed in strata, making it difficult to obtain complete paleontological trace fossil samples. Therefore, neural network methods that require a large number of training images and are commonly used in convolution methods seem unsuitable here. (2) Microscopic images are complex, with various minerals filling the inside and outside of the sample. In addition, the deformation of paleontological remains caused by replacement and compaction during the formation of rocks strongly interferes with image recognition. Therefore, the identification of paleontological textures becomes difficult, and it is necessary to select a suitable texture classification method. (3) Rock specimens can only display two-dimensional cross-sections of three-dimensional paleontological remains. Selecting different angles and positions from the same sample will result in very different features in the two-dimensional image.

[0044] This invention is the first to collect a paleontological image dataset, collecting trace fossil images from published papers as positive samples and using web crawling technology to crawl rock images from search engines as negative samples.

[0045] This invention extracts images from datasets containing paleontological images using code. The extraction process is as follows: Figure 2 As shown.

[0046] The first step is to obtain paleontology papers. Based on the images extracted by Algorithm 1, the papers are screened to extract those containing paleontology images, and then the papers are downloaded.

[0047]

[0048] The second step is to extract paleontological images, such as... Figure 3 As shown. Select papers with soil traces and whose themes are related to paleontology. Within these papers, extract images containing paleontological features, retaining the captions and descriptions related to the extracted images as the image names. Place images with and without outline annotations in different folders.

[0049] The third step is to remove the manually marked outlines from the image. Use Photoshop tools to remove the marks, such as... Figure 4 As shown.

[0050] Fourth step, record the information. Add the information saved in step two into the file according to the prescribed format.

[0051] This invention collects non-paleontological datasets in four steps, including web crawling, deduplication, screening, and recording, as follows: Figure 5 As shown in the image. The recorded information can be used during the screening process.

[0052] The first step is to crawl images from search engines. A search engine-based image crawling tool was developed to differentiate images at their source. This tool allows users to specify search keywords or page numbers for images and download them to a specified folder. The crawling algorithm is as follows.

[0053]

[0054] The second step is to use code to remove duplicates. Based on the image information recorded in the file, duplicate images are deleted.

[0055] The third step is manual screening. Manual screening must adhere to two principles: the image must be natural with a single, uniform background and contain only a single stone. A folder containing the data from before the manual screening should be retained, and images that do not meet these principles should be deleted one by one.

[0056] The fourth step is to record the data.

[0057] S2. Preprocess the paleontological image dataset and the non-paleontological image dataset;

[0058] In an optional embodiment of the present invention, step S2 specifically includes the following sub-steps:

[0059] S21. The Frequency Tuned algorithm is used to perform Gaussian smoothing with a window size of 5×5 on the paleontological image dataset and the non-paleontological image dataset to remove high-frequency information in the image data.

[0060] S22. The Luminance Contrast algorithm is used to perform global contrast enhancement processing on the paleontological image dataset and the non-paleontological image dataset respectively.

[0061] Specifically, due to the complexity of the image patterns, the paleontological morphology is influenced by diagenesis, various compaction processes, and severe metasomatism. The paleontological patterns are also significantly affected by mineral lines. Image preprocessing is a crucial step. After initially collecting a certain amount of raw data, to improve the quality of the dataset and accelerate feature extraction in the neural network model, we derived two new image datasets using image saliency algorithms (Frequency-tuned (FT) and Luminance Contrast (LC) algorithms).

[0062] Since most trace fossils are footprints of ancient vertebrates, the size and depth of which are significantly different from those of other organisms and are closely related to the geological background, the LC algorithm is used to highlight global contrast and the FT algorithm to display overall information.

[0063] The Fourier Transform (FT) algorithm is a classic visual saliency detection algorithm. As is well known, images can be divided into low-frequency and high-frequency components in the frequency domain. The low-frequency component reflects the overall information of the image, such as the outlines of objects and basic constituent regions. The high-frequency component reflects the detailed information of the image, such as the texture of objects. The saliency detection of the Fourier Transform algorithm primarily utilizes information from the low-frequency component. The saliency of a pixel can be calculated using the following formula:

[0064]

[0065] Among them, I μ It is the arithmetic mean of the image pixels. This applies a Gaussian blur to the original image. The algorithm uses Lab space, where ||.|| represents the Euclidean distance.

[0066] The Color Levitational (LC) algorithm is another visual saliency detection algorithm. Its basic idea is to calculate the global contrast of a pixel across the entire image, that is, to use the sum of the color distances between that pixel and all other pixels in the image as the saliency value of that pixel. The calculation formula is:

[0067]

[0068] Among them, I k Let I be the pixel value of pixel k in image I. i This is the grayscale value.

[0069] In actual calculations, the FT algorithm uses Gaussian smoothing with a window size of 5×5 to discard high-frequency information. Furthermore, if the LC algorithm directly uses the above formula to calculate saliency values, the time complexity is too high. Therefore, histogram optimization is used to improve the time complexity of the LC algorithm. The processed image is as follows. Figure 6 As shown in Figure 7.

[0070] S3. Perform viewpoint image segmentation on the preprocessed paleontological image dataset and the non-paleontological image dataset, and train the generative adversarial network Style GAN based on the segmentation results.

[0071] In an optional embodiment of the present invention, step S3 specifically includes the following sub-steps:

[0072] S31. Perform left-view image L, middle-view image C, and right-view image R segmentation on the preprocessed paleontological image dataset and the non-paleontological image dataset respectively to obtain n sets of three-view image datasets corresponding to the paleontological image dataset and the non-paleontological image dataset.

[0073] S32. Randomly select a set of three-view images from the three-view image dataset corresponding to the paleontological image dataset and the non-paleontological image dataset. Combine the left-view image L and the right-view image R into a pair of samples (L,R), and take the middle-view image C as the true value C of the pair of samples.

[0074] S33. Repeat the above operation to process the remaining (n-1) groups of three-view images in the three-view image dataset to obtain n pairs of samples (L,R) and n true values ​​C.

[0075] S34. Randomly divide the n pairs of samples (L,R) into a training sample set and a test sample set according to the ratio. Use the training sample set to train the constructed Generative Adversarial Network (GAN) Style GAN, optimize the parameters of the GAN Style GAN, and obtain the trained GAN Style GAN. Use the test set to test the generator G of all trained GAN Style GANs, evaluate the image inpainting performance of generator G, and select the optimal generator G. The process is as follows. Figure 8 As shown.

[0076] S4. Use the trained generative adversarial network Style GAN to generate a first-view image, and stitch it with the second-view image obtained from the view image segmentation in step S3 to generate a new paleontological image.

[0077] In an optional embodiment of the present invention, step S4 specifically includes:

[0078] The samples (L,R) in the test sample set are sequentially input into the optimal generator G to obtain the first-view image A. The first-view image A is then stitched to the left and right sides of the ground truth C to generate a new paleontological image.

[0079] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0083] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for generating paleontological images based on neural networks, characterized in that, Includes the following steps: S1. Obtain paleontological image datasets and non-paleontological image datasets; S2. Preprocess the paleontological image dataset and the non-paleontological image dataset; S3. Perform viewpoint image segmentation on the preprocessed paleontological image dataset and the non-paleontological image dataset, and train the generative adversarial network Style GAN based on the segmentation results. Step S3 details It includes the following steps: S31. Perform left-view image L, middle-view image C, and right-view image R segmentation on the preprocessed paleontological image dataset and the non-paleontological image dataset respectively to obtain n sets of three-view image datasets corresponding to the paleontological image dataset and the non-paleontological image dataset. S32. Randomly select a set of three-view images from the three-view image dataset corresponding to the paleontological image dataset and the non-paleontological image dataset. Form a pair of samples (L,R) with the left-view image L and the right-view image R, and take the middle-view image C as the true value C of the pair of samples. S33. Repeat the above operation to process the remaining n-1 groups of three-view images in the three-view image dataset to obtain n pairs of samples (L, R) and n true values ​​C; S34. Randomly divide the n pairs of samples (L, R) into a training sample set and a test sample set according to the proportion. Use the training sample set to train the constructed Generative Adversarial Network Style GAN, optimize the parameters of the Generative Adversarial Network Style GAN, and obtain the trained Generative Adversarial Network Style GAN. Use the test set to test the generator G of all trained Generative Adversarial Network Style GANs, evaluate the image inpainting performance of generator G, and select the optimal generator G. S4. Use the trained generative adversarial network Style GAN to generate a first-view image, and then stitch it with the second-view image obtained by segmenting the view image in step S3 to generate a new paleontology image. Step S4 specifically includes: The samples (L, R) in the test sample set are sequentially input into the optimal generator G to obtain the first-view image A. The first-view image A is then stitched to the left and right sides of the ground truth C to generate a new paleontological image.

2. The paleontological image generation method based on neural networks according to claim 1, characterized in that, Step S2 specifically includes the following sub-steps: S21. The Frequency Tuned algorithm is used to perform Gaussian smoothing with a window size of 5×5 on the paleontological image dataset and the non-paleontological image dataset to remove high-frequency information in the image data. S22. The Luminance Contrast algorithm is used to perform global contrast enhancement processing on the paleontological image dataset and the non-paleontological image dataset respectively.

3. The paleontological image generation method based on neural networks according to claim 2, characterized in that, The formula for calculating the saliency of a pixel in the Frequency Tuned algorithm is as follows: in, This is the arithmetic mean of the image pixels. This is a Gaussian blur applied to the original image, where || is the Euclidean distance.

4. The paleontological image generation method based on neural networks according to claim 2, characterized in that, The formula for calculating the saliency of a pixel in the Luminance Contrast algorithm is as follows: in, For image The pixel value of pixel k in the middle. This is the grayscale value.

Images