Method, device, computer equipment and storage medium for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculation

Abstract

The embodiment of the present invention relates to the field of jewelry identification technology and information technology, and discloses a method for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculation, the method comprising: obtaining a pearl image of a pearl to be classified; the pearl image contains the surface growth line characteristics of the pearl to be classified; the pearl image is input into a preset first classification model to obtain a first classification result; the first classification model is obtained by training a convolutional neural network; the pearl image is input into a second classification model to obtain a second classification result; the second classification model is obtained by training a deep residual network based on pearl image samples; the pearl image is input into a third classification model to obtain a third classification result; the third classification model is obtained by training a time series model based on pearl image samples; according to the first classification result, the second classification result and the third classification result, the target category is determined; the target category includes seawater pearls and freshwater pearls. The embodiment of the present invention realizes accurate identification of pearls.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of jewelry identification technology and informatization, and specifically to a method, device, computer equipment and storage medium for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculations. Background Art

[0002] Whether used for jewelry decoration or medicinal purposes, there is a big difference in the value of saltwater pearls and freshwater pearls, so the identification of pearl categories is of great significance.

[0003] At present, the methods commonly used to identify and distinguish seawater and freshwater pearls mainly include naked eye observation, irradiation, elemental method, synchrotron radiation X-ray and cathode luminescence CL imaging. Naked eye observation mainly uses a gem microscope to observe the gloss and texture of the pearl surface. This method requires personnel training and consumes a lot of manpower. At present, it mainly relies on a microscope with an eyepiece of 10X and an objective lens of 1-4X, that is, a gem microscope with a maximum magnification of 40X. This method is easier to distinguish colored pearls, but white pearls are difficult to distinguish. Therefore, white pearls are mainly tested by other auxiliary means. Irradiation method, predecessors mainly used Co 60 The irradiation method is used to identify seawater and freshwater medicinal pearls. It is found that freshwater pearls are more likely to appear black, thus distinguishing seawater and freshwater pearls. The element method mainly uses ICP-MS and XRF testing. The ICP-MS destructive / non-destructive method is used to obtain the types and content / relative content of trace elements in seawater and freshwater. Previous researchers believed that seawater pearl powder is rich in Na, Sr, K, Mg and B, while freshwater pearl powder is rich in Mn, Ba, Fe and Cr; compared with freshwater pearls, the content of arginine in seawater pearls is lower and the content of tyrosine is higher; the total amount of trace elements in freshwater cultured pearls is significantly less than that in seawater cultured pearls; the content of trace elements in freshwater cultured pearls is relatively regular, with a relatively fixed content sequence, namely Sr>Mn>Ba>Ni>Cr>Pb; while the content of trace elements in seawater cultured pearls varies greatly. Except that each sample is the most enriched in Sr element, there is no obvious content order. XRF is used to test the peak area (content) of Sr / Ca elements in pearls to determine the freshwater / seawater source of pearls. Synchrotron X-ray and cathodoluminescence CL imaging. Previous researchers have used X-ray transmission imaging and fluorescence dual-mode pearl detection. The cathode luminescence characteristics can basically distinguish freshwater and seawater cultured pearls with similar appearances. In addition, thermal analysis, image recognition technology and texture detection are also used to identify seawater and freshwater.

[0004] Most of the above methods for distinguishing seawater and freshwater pearls belong to conventional gemological and mineralogical testing technologies, including naked eye observation, irradiation, elemental method, synchrotron radiation X-ray and cathode luminescence CL imaging. Among them, naked eye observation is a subjective judgment, which requires trained experienced inspectors to make judgments. The training cycle is long. Practice has proved that not all pearls can be accurately judged by the naked eye. Previous experience shows that for experienced inspectors, the accuracy of naked eye judgment of white pearls of the same size is about 60-70%. Based on the fact that pearls are precious samples in daily testing, irradiation and LA-ICP-MS are destructive testing methods, so they are not really used in daily jewelry testing. At present, the most widely used method in daily testing is XRF fluorescence testing element method, which mainly uses Sr / Ca ratio and ROI area calculation. This method has a certain theoretical basis and its accuracy is greater than 80%. However, for many pearl layers with thinner pearls or sea areas close to land, the Sr / Ca ratio is not within the distinguishable threshold range, and the accuracy of distinguishing such pearls is low. As for the testing methods of synchrotron radiation and cathode luminescence, no scholar has conducted a large amount of data verification, and its accuracy still needs to be further verified. In addition, these two devices are large-scale instruments with high testing costs, long testing time, and low efficiency. They are not really widely used in the field of jewelry testing. Therefore, a method that can quickly and accurately identify pearl categories is urgently needed. Summary of the invention

[0005] In view of the above problems, an embodiment of the present invention provides a method for distinguishing between seawater and freshwater pearls based on deep learning and mathematical calculations, which is used to solve the problem of low accuracy and low efficiency in the identification of pearl categories in the prior art.

[0006] According to one aspect of an embodiment of the present invention, a method for distinguishing seawater pearls from freshwater pearls based on deep learning and mathematical calculation is provided, the method comprising:

[0007] Acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl to be classified;

[0008] Inputting the pearl image into a preset first classification model to obtain a first classification result; the first classification model is obtained by inputting a convolutional neural network training according to a preset pearl image sample; the first classification result includes one of seawater pearls and freshwater pearls;

[0009] Inputting the pearl image into a preset second classification model to obtain a second classification result; the second classification model is obtained by inputting a deep residual network training according to a preset pearl image sample; the second classification result includes one of seawater pearls and freshwater pearls;

[0010] Input the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained according to a preset pearl image sample input into a time series model; and the third classification result includes one of seawater pearls and freshwater pearls;

[0011] According to the first classification result, the second classification result and the third classification result, a target category of the pearls to be classified is determined; the target category includes one of seawater pearls and freshwater pearls.

[0012] In an optional manner, the step of obtaining pearl images of pearls to be classified further comprises:

[0013] Using a magnification system with a preset magnification, the curved spherical pearl to be classified is photographed from at least three different directions to obtain at least three pearl images taken from different directions that include growth line features, halo color features, and body color features;

[0014] Wherein, the magnification system includes an electronic microscope, a CCD and a chassis spring bracket;

[0015] The electron microscope device comprises an electronic fixed-focus macro lens and a light source, wherein the light source is arranged at the electronic fixed-focus macro lens and is used to provide a light source for the pearls to be classified; the electronic fixed-focus macro lens is set to a fixed magnification and is used to obtain images of the pearls to be classified;

[0016] The spring bracket is arranged below the electronic fixed-focus macro lens and is used to clamp the pearls to be classified. The spring bracket can move away from or approach the lens direction of the electronic fixed-focus macro lens according to a preset step length. The CCD is electrically connected to the electronic fixed-focus macro lens and is used to acquire and transmit the pearl image taken by the electronic fixed-focus macro lens.

[0017] In an optional manner, the first classification model includes a first initial classification model and a second initial classification model connected in series; the convolutional neural network is an Inception algorithm; a pearl image sample includes three sample images of a sample pearl in different directions and a first sample label, and the first sample label includes an imitation pearl label and a pearl label; the sample image belonging to the pearl label also carries a second sample label, and the second sample label includes a seawater pearl label and a freshwater pearl label;

[0018] Before inputting the pearl image into a preset first classification model to obtain a first classification result, the method further comprises:

[0019] Extracting RGB values ​​of pixels in the pearl image sample, and converting the RGB values ​​into HSL values;

[0020] Inputting the pearl image into the Inception algorithm for training, calculating a first loss according to a first sample label and a first loss function, adjusting the parameters of the convolutional neural network according to the first loss, and obtaining a trained first initial classification model;

[0021] Determine the prediction results of the trained first initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; calculate whether the HSL values ​​of the pearl image samples classified as pearls are within the HSL threshold range of pearls, and whether the HSL values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls;

[0022] If it is not within the HSL threshold range, continue to train the trained first initial classification model according to the pearl image sample; if it is within the HSL threshold range, perform a blind sample test on the trained first initial classification model to obtain a trained first initial classification model;

[0023] The pearl image samples belonging to the pearl label are input into the Inception algorithm for training, and the second loss is calculated according to the second sample label and the second loss function, and the parameters of the convolutional neural network are adjusted according to the second loss until a second initial classification model is obtained.

[0024] In an optional manner, the second classification model includes a third initial classification model and a fourth initial classification model; before inputting the pearl image into the preset second classification model to obtain the second classification result, the method includes:

[0025] Extracting RGB values ​​of pixels in the pearl image sample;

[0026] Inputting the pearl image sample into a deep residual network, calculating a third loss according to the first sample label and a third loss function, adjusting the parameters of the deep residual network according to the third loss, and obtaining a trained third initial classification model; wherein the deep residual network includes a 1*1 convolution kernel, an N*N convolution kernel, and a 1*1 convolution kernel connected in sequence, wherein N is greater than 1;

[0027] Determine the prediction results of the trained third initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculate whether the RGB values ​​of the pearl image samples classified as pearls are within the RGB threshold range of pearls, and whether the RGB values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls;

[0028] If it is not within the RGB threshold range, continue to train the trained third initial classification model according to the pearl image sample; if it is within the RGB threshold range, perform a blind sample test on the trained third initial classification model to obtain a trained third initial classification model;

[0029] The RGB values ​​of the pearl image samples belonging to the pearl label are input into the deep residual network for training, and the fourth loss is calculated according to the second sample label and the fourth loss function, and the parameters of the deep residual network are adjusted according to the fourth loss until a fourth initial classification model is obtained.

[0030] In an optional manner, the time series model integrates the algorithms after Transfromer and KAN; before inputting the pearl image into a preset third classification model to obtain the third classification result, the method includes:

[0031] Inputting the pearl image sample into the Transfromer+KAN algorithm to obtain a plurality of first input features of the pearl image sample;

[0032] Inputting the plurality of first input features into the Transfromer+KAN algorithm to obtain a first prediction result;

[0033] Calculating a fifth loss according to the first prediction result, the first sample label and a fifth loss function, and adjusting parameters of the Transfromer+KAN algorithm according to the fifth loss until a fifth initial classification model is obtained;

[0034] Input the pearl image samples belonging to the pearl label into the Transfromer+KAN algorithm to obtain multiple second input features;

[0035] Inputting the plurality of second input features into the KAN algorithm to obtain a second prediction result;

[0036] A fifth loss is calculated according to the second prediction result, the second sample label and the sixth loss function, and the parameters of the Transfromer+KAN algorithm are adjusted according to the sixth loss until a sixth initial classification model is obtained.

[0037] In an optional manner, determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result includes:

[0038] A weighted sum is performed according to the first classification model, the second classification model and the third classification model, and the weights corresponding to the first classification model, the second classification model and the third classification model, to obtain the target category of the pearls to be classified.

[0039] In an optional manner, the first classification model, the second classification model and the third classification model are stored in a cloud server;

[0040] The step of obtaining pearl images of pearls to be classified comprises:

[0041] Uploading the pearl images of the pearls to be classified, which are taken by the microscope, to the cloud server;

[0042] After determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result, the method further includes:

[0043] The cloud server sends the target category to the user terminal.

[0044] According to another aspect of an embodiment of the present invention, a device for distinguishing seawater pearls from freshwater pearls based on deep learning and mathematical calculation is provided, comprising:

[0045] An acquisition module is used to acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl to be classified;

[0046] A first classification module is used to input the pearl image into a preset first classification model to obtain a first classification result; the first classification model is obtained by inputting a convolutional neural network training according to a preset pearl image sample; the first classification result includes one of seawater pearls and freshwater pearls;

[0047] A second classification module is used to input the pearl image into a preset second classification model to obtain a second classification result; the second classification model is trained by inputting a deep residual network according to a preset pearl image sample; the second classification result includes one of seawater pearls and freshwater pearls;

[0048] A third classification module is used to input the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained according to a preset pearl image sample input into a time series model; and the third classification result includes one of seawater pearls and freshwater pearls;

[0049] The target classification module is used to determine the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result; the target category includes one of seawater pearls and freshwater pearls.

[0050] According to another aspect of an embodiment of the present invention, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0051] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the method for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculations.

[0052] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction, and when the executable instruction is executed on a computer device, the computer device executes the operation of the method for distinguishing between seawater and freshwater pearls based on deep learning and mathematical calculations.

[0053] The embodiment of the present invention obtains a pearl image including the growth line characteristics, halo color and body color of the pearl through an amplification system, identifies the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl, and sets three different types of classification models according to the characteristics of the pearl, respectively identifies the image from different angles, realizes local and global accurate judgment of the image, and determines the final pearl category according to the classification results of the three, so as to effectively improve the accuracy of distinguishing seawater pearls from freshwater pearls.

[0054] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0056] Figure 1 A schematic diagram of a process for distinguishing seawater pearls from freshwater pearls based on deep learning and mathematical calculations provided by an embodiment of the present invention is shown;

[0057] Figure 2 A schematic diagram of the structure of a device for distinguishing seawater pearls from freshwater pearls based on deep learning and mathematical calculations provided by an embodiment of the present invention is shown;

[0058] Figure 3 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Figure 1 The flowchart of the method for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculation provided by an embodiment of the present invention is shown, and the method is executed by a computer device. The computer device can be a cloud server, etc., and the embodiment of the present invention does not make specific restrictions. In a specific implementation, for example, the computer device can be a cloud server, and the amplification system and the user terminal interact with the cloud server for data. Figure 1 As shown, the method comprises the following steps:

[0060] Step 110: Acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line features, halo color features and body color features of the pearl to be classified.

[0061] Among them, the surface growth pattern characteristics of pearls refer to some fine stripes or lines on the surface of pearls. These lines are traces of sediment deposition during the formation of pearls. Pearl growth patterns are an important feature of pearl quality, which will affect the glossiness and surface texture of pearls. High-quality pearl growth patterns should be uniform, delicate and layered, showing beautiful textures and colors under light. Pearl body color characteristics: also known as body color, also known as background color, is the color produced by the selective absorption of white light by pearls, which depends on the various pigments and trace metal elements contained in the pearls themselves. Pearl iridescence characteristics: It is a drifting rainbow color formed on the surface or subsurface of the pearl, which is added to its body color and is observed from the light reflected from the surface of the pearl.

[0062] In the embodiment of the present invention, a magnification system with a preset magnification can be used to shoot the curved sphere pearls to be classified from at least three different directions to obtain at least three pearl images taken from different directions containing growth line features, halo color features, and body color features. Specifically, the magnification system in the embodiment of the present invention includes an electron microscope, a CCD (sensor), and a chassis spring bracket. Among them, the electron microscope includes an electronic fixed-focus macro lens and a light source. Since the light source contains coaxial, side light, and annular light paths (since it is necessary to obtain the surface growth line features, halo color features, and body color features of the pearl, different light sources are required to cooperate to obtain them); CCD is for signal transmission, and the spring bracket is for pushing the pearls to be tested to a fixed position to meet the requirements that pearls of different sizes can present clear textures at a fixed magnification (which can be any fixed magnification of 20-1000X). Among them, the light source is arranged near the electronic fixed-focus macro lens, which is used to provide a light source for the pearls to be classified; the spring bracket is arranged below the lens of the electron microscope, which is used to clamp the pearls to be classified, and it can move away from or approach along the lens direction of the electron microscope according to a preset step length; the electron microscope is set to a fixed magnification, and the CCD is electrically connected to the electron microscope, which is used to obtain and transmit the image taken by the electron microscope. Since pearls have different sizes, the lens of the electron microscope is set to a fixed magnification. Therefore, by setting the distance between the spring bracket and the lens, the pearls to be classified can be directly placed at the focal length of the electron microscope, so that the pearl image of the pearl to be classified can be quickly obtained. And for pearls of different sizes to be classified, the spring bracket can be adjusted to a predetermined position, so that the pearls to be classified can be quickly set at the focal length, avoiding the problem of low efficiency caused by frequent focusing.

[0063] Among them, a fixed magnification between 20X-1000X of the microscope can be used to shoot the pearls to be classified from three different directions, and after removing the surface reflection, a pearl image of the pearl to be classified showing clear pearl surface textures and halo color and body color is obtained. After the pearl image of the pearl to be classified is obtained, the microscope communicates with the cloud server and uploads the captured pearl image of the pearl to be classified to the cloud server.

[0064] The cloud server stores a first classification model, a second classification model and a third classification model for identifying the pearls to be classified. Each model is written into a program of the language cloud server through a computer language, such as Python.

[0065] Step 120: Input the pearl image into a preset first classification model to obtain a first classification result. The first classification result includes one of seawater pearls and freshwater pearls.

[0066] In the embodiment of the present invention, in order to improve the classification accuracy of the pearls to be classified, the first classification model is divided into a first initial classification model and a second initial classification model. The first classification model includes the first initial classification model and the second initial classification model connected in series. The first initial classification model is used to determine whether the pearl image of the pearl to be classified is a pearl or a pearl imitation; the second initial classification model is used to determine whether the pearl to be classified determined as a pearl by the first initial classification model is a seawater pearl or a freshwater pearl. Among them, the convolutional neural network is an Inception algorithm, which processes the input image by connecting multiple convolution kernels of different sizes in parallel, and then splices these feature maps of different scales to capture the multi-scale features in the image. This structure enables the Inception model to extract features at different scales, thereby enhancing the representation ability of the model. Although the Inception model itself does not directly involve local processing of the image, its design concept and implementation method have an important influence on the local feature extraction of the image. Therefore, considering the important role of the Inception algorithm in the local feature extraction of the pearl image, the embodiment of the present invention adopts the Inception model as the first classification model.

[0067] Wherein, the first classification model is obtained by pre-selecting and inputting the convolutional neural network training according to the preset pearl image samples. A pearl image sample includes three sample images of a sample pearl in different directions and a first sample label, wherein the first sample label includes an imitation pearl label and a pearl label; the sample image belonging to the pearl label also carries a second sample label, wherein the second sample label includes a seawater pearl label and a freshwater pearl label. Wherein, the specific acquisition process of the pearl image sample includes: collecting 5,000 seawater pearls, 5,000 freshwater pearls, and 5,000 pearl imitations from different sources, including seawater pearls of different colors in multiple regions around the world and pearl imitations on multiple markets. The surface texture of the sample is observed using a microscope with a magnification of 20X-1000X, and different pearls or "imitation pearls" are photographed, and "freshwater pearls", "seawater pearls", and "imitation pearls" are respectively used as a class of labels.

[0068] Specifically, the first classification model is trained as follows:

[0069] Step 0020: extracting the RGB values ​​of the pixels in the pearl image sample and converting the RGB values ​​into HSL values, wherein all the pearl image samples are resized to 100*100, and the RGB values ​​of the pixels in the image are extracted and converted into HSL values.

[0070] Step 0021: Input the pearl image samples into the Inception algorithm for training, calculate the first loss according to the first sample label and the first loss function, adjust the parameters of the convolutional neural network according to the first loss, and obtain a trained first initial classification model. The first initial classification model is used to classify whether the pearl to be classified is a pearl or an imitation pearl. Among them, the convolutional network of the Inception algorithm model takes a 3*3 convolution kernel, and inputs the pearl image samples of "imitation pearl" and "pearl" into the convolutional network of the Inception algorithm model for training.

[0071] In the embodiment of the present invention, the validation set data is used to train the neural network on the images with the category labels, and the first loss function is the cross entropy loss function. When the parameters are adjusted according to the first loss function, the model is also verified by the validation set. When the loss function Loss of the model drops to the threshold, the first loss and the validation set training accuracy are greater than 90%, the model is considered to have converged, and the image classification model M is obtained. Art-Pearl . The first loss function is expressed as: L Art-Pearl = Wherein, y represents the true label of the pearl image sample, and ŷ represents the model prediction result; when y=1, it means that the pearl image sample belongs to the positive class (pearl), and when y=0, it means that the pearl image sample belongs to the negative class (imitation pearl).

[0072] Wherein, after training, in order to improve the accuracy of the model, the embodiment of the present invention also combines the HSL value of the pearl image to perform mathematical calculations to determine whether the first initial classification model after training is completed, and a blind sample test can be performed after the training is completed. Specifically, the prediction results of the first initial classification model after training for the sample are determined, and the prediction results include pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculate whether the HSL value of the pearl image sample classified as pearl is within the HSL threshold range of the pearl, and whether the HSL value of the pearl image classified as imitation pearl is within the threshold range of the imitation pearl. If it is not within the HSL threshold range, continue to train the first initial classification model after training according to the pearl image sample; if it is within the HSL threshold range, perform a blind sample test on the first initial classification model after training to obtain the trained first initial classification model. Wherein, the HSL thresholds of imitation pearls and pearls can be obtained by supervised learning, respectively, by annotating and statistically obtaining a large number of known pearl or imitation pearl samples.

[0073] In an embodiment of the present invention, untrained labeled "imitation pearls" and "pearls" pictures are used to verify the generalization ability. When the accuracy of the blind sample reaches more than 95%, the model results can be used to classify "imitation pearls" and "pearls" to obtain the final first initial classification model.

[0074] Step 0022: Input the pearl image samples belonging to the pearl label into the Inception algorithm for training, calculate the second loss according to the second sample label and the second loss function, and adjust the parameters of the convolutional neural network according to the second loss until a second initial classification model is obtained. The second initial classification model is used to predict whether the pearl to be classified is a seawater pearl or a freshwater pearl.

[0075] Among them, all the pearl image samples classified as "pearls" by the first initial classification model in the aforementioned step 0021 are further classified into "seawater pearls" and "freshwater pearls", and the second classification model is trained. The specific method is consistent with steps 0020 and 0021 and will not be repeated here. In this way, the second initial classification model Msea-culture1 is trained.

[0076] After obtaining the first initial classification model and the second initial classification model, the pearl image is input into the preset first classification model. The first initial classification model determines whether the pearl to be classified is a "pearl" or an "imitation pearl". When it is a "pearl", the second initial classification model performs classification prediction to obtain the first classification result of whether the pearl to be classified is a "seawater pearl" or a "freshwater pearl".

[0077] Step 130: Input the pearl image into a preset second classification model to obtain a second classification result; the second classification model is trained by inputting a preset pearl image sample into a deep residual network.

[0078] Wherein, the second classification model includes a third initial classification model and a fourth initial classification model. The third initial classification model is used to distinguish whether the pearl to be classified is a pearl or an imitation pearl. The fourth initial classification model is used to distinguish whether the pearl to be classified is a seawater pearl or a freshwater pearl. Wherein, the training sample of the second classification model in this step can be the same as the pearl image sample in step 120, or can be collected separately. Wherein, the deep residual network is a resnet residual network, wherein considering that the growth texture in the pearl image is identified in the embodiment of the present invention, after analysis, the resnet residual network has a better effect on image texture recognition, so the resnet residual network is selected for training.

[0079] Specifically, the second classification model is trained as follows:

[0080] Step 0030: extracting the RGB values ​​of the pixels in the pearl image sample.

[0081] Among them, all pearl image samples are resized to 224*224, and the RGB values ​​of the pixels in the pearl image samples are extracted.

[0082] Step 0031: Input the pearl image sample into the deep residual network, calculate the third loss according to the first sample label and the third loss function, adjust the parameters of the deep residual network according to the third loss, and obtain the trained third initial classification model.

[0083] The deep residual network includes a 1*1 convolution kernel, an N*N convolution kernel, and a 1*1 convolution kernel connected in sequence, where N is greater than 1. In the embodiment of the present invention, in a normal convolutional neural network, a branch structure is added, first through a 1*1 convolution kernel, then through a 3*3 convolution kernel, and finally through a 1*1 convolution kernel, the pearl image samples with "imitation pearls" and "pearls" are input into the convolutional network of the resnet residual network for training, and the validation set data is used to train the neural network for the pearl image samples with category labels, so that the network output is no longer F （x） , but F(x)+x In order to reduce the number of parameters, the following gradient update formula is used to update the parameters during training to prevent the gradient from disappearing: Gradient update formula: in, represents the third loss function, H(x) represents the residual network output, and F(x) represents the normal network output. After training, in order to improve the accuracy of the model, the embodiment of the present invention also performs mathematical calculations in combination with the RGB values ​​of the pearl image to determine whether the training of the trained third initial classification model is completed, and determines the prediction results of the trained third initial classification model for the sample, the prediction results include pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculate whether the RGB values ​​of the pearl image samples classified as pearls are within the RGB threshold range of pearls, and whether the RGB values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls; if not within the RGB threshold range, continue to train the trained third initial classification model according to the pearl image samples; if within the RGB threshold range, perform a blind sample test on the trained third initial classification model to obtain a trained third initial classification model. Among them, after the training is completed, untrained but labeled "imitation pearls" and "pearls" pictures are used to verify the generalization ability. When the accuracy of the blind sample reaches more than 95%, the final third initial classification model is obtained, and the model results can be used to classify "imitation pearls" and "pearls".

[0084] Step 0032: Input the pearl image samples belonging to the pearl label into the deep residual network for training, calculate the fourth loss according to the second sample label and the fourth loss function, and adjust the parameters of the deep residual network according to the fourth loss until a fourth initial classification model is obtained.

[0085] Among them, all the pearl image samples classified as "pearls" by the third initial classification model in the aforementioned step 0031 are further classified into "saltwater pearls" and "freshwater pearls", and the fourth classification model is trained. The specific method is consistent with steps 0030 and 0031, and will not be repeated here. The fourth initial classification model is trained in this way.

[0086] In an embodiment of the present invention, after obtaining the third initial classification model and the fourth initial classification model, the pearl image is input into the preset third classification model, and the third initial classification model determines whether the pearl to be classified is a "pearl" or an "imitation pearl". When it is a "pearl", the fourth initial classification model performs classification prediction to obtain a second classification result of whether the pearl to be classified is a "seawater pearl" or a "freshwater pearl".

[0087] Step 140: inputting the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained by inputting preset pearl image samples into a time series model.

[0088] The time series model includes an algorithm after integrating Transfromer and KAN. By adopting the Transfromer+KAN algorithm, pearl images can be classified and identified in a global perspective.

[0089] In the embodiment of the present invention, the specific training process of the time series model includes: Step 0040: inputting the pearl image sample into the Transfromer+KAN algorithm to obtain multiple first input features of the pearl image sample.

[0090] The Transfromer algorithm divides the pearl image sample into blocks, then performs position encoding, and learns the relationship between image blocks through the self-attention mechanism, extracts deeper features through the feedforward network, and obtains multiple different first input features X 1 , X 2 , X 3 , X 4 .

[0091] Step 0041: Input the plurality of first input features into the Transfromer+KAN algorithm to obtain a first prediction result. The Transfromer+KAN algorithm implements the image classification task by performing nonlinear processing on the input features, performing spline parameterization, signal summation, and performing summation operations on nodes. Specifically, the Transfromer+KAN algorithm is trained by the following steps: Nonlinear processing: Inputting a plurality of different input features X 1 , X2 , X 3 , X 4 Perform nonlinear processing: Parametric spline, the nodes of the network structure only sum the incoming signals without any linear transformation: Where n is the input dimension, . Perform function approximation: For the pearls and imitation pearls that need to be distinguished, the input-output pair {xi, yi} is composed to find the function f(x), so that for all data points yi≈f(xi), if the appropriate single variable function φq,p, Фq is found, the activation function is used to process the network. Among them, the activation function formula is: Among them, the activation function The pre-activation value is simply expressed as , The post-activation value is .

[0092] The activation value of a neuron is the sum of all incoming activation values, as shown in the following formula: in, Represents the function matrix corresponding to the Lth KAN layer. Given an output vector x_0∈R^n0, the output given by the following formula can be used to distinguish pearl image samples as pearls or imitation pearls: .

[0093] Step 0042: Calculate the fifth loss according to the first prediction result, the first sample label and the fifth loss function, and adjust the parameters of the Transfromer+KAN algorithm according to the fifth loss until a fifth initial classification model is obtained. The fifth loss function may be a cross entropy loss function. After the above iterative training, the fifth initial classification model for distinguishing pearls from imitation pearls is obtained.

[0094] In the manner of steps 0040-0042, the pearl image samples belonging to the pearl label are input into the time series model to continue iterative training, and the sixth initial classification model capable of classifying seawater pearls and freshwater pearls is obtained, including steps 0043-0045. Step 0043: Input the pearl image samples belonging to the pearl label into the Transfromer+KAN algorithm to obtain multiple second input features. Step 0044: Input the multiple second input features into the Transfromer+KAN algorithm to obtain a second prediction result. Step 0045: Calculate the fifth loss according to the second prediction result, the second sample label and the sixth loss function, and adjust the parameters of the Transfromer+KAN algorithm according to the sixth loss until the sixth initial classification model is obtained. The specific training process is consistent with the above training process and will not be repeated here. After obtaining the trained fifth initial classification model and the sixth initial classification model, the pearl image is input into the preset fifth classification model to determine whether the pearl to be classified is a "pearl" or an "imitation pearl". When it is a "pearl", the sixth initial classification model performs classification prediction again to obtain a second classification result of whether the pearl to be classified is a "seawater pearl" or a "freshwater pearl".

[0095] Step 150: Determine the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result.

[0096] Wherein, the target category includes one of seawater pearls and freshwater pearls. In an embodiment of the present invention, a weighted sum is performed according to the first classification model, the second classification model and the third classification model, and the weights corresponding to the first classification model, the second classification model and the third classification model, respectively, to obtain the target category of the pearl to be classified; wherein, the weight is determined by back propagation iteration of mathematical logic on the first classification model, the second classification model and the third classification model according to the blind sample pearl. wherein, after the first classification model, the second classification model and the third classification model are trained respectively, the training results of the three models are iterated by back propagation to determine their respective weights. wherein, after obtaining the target category, the cloud server sends the target category to the user terminal. thereby, by amplifying the interaction between the system and the cloud server and the user terminal, the category of the pearl can be conveniently and accurately identified.

[0097] The embodiment of the present invention obtains a pearl image containing the growth line features of the pearl through an amplification system, identifies the surface growth line features of the pearl, and sets three different types of classification models to identify the image from different angles respectively. The local field of view is increased by the Inception algorithm, the resnet residual network focuses on the image texture, and the time series model in the present application increases the global field of view, thereby achieving accurate local and global judgment of the image, and determining the final pearl category based on the classification results of the three, thereby effectively improving the accuracy of distinguishing seawater pearls from freshwater pearls. Figure 2 The schematic diagram of the structure of the device for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculation provided by an embodiment of the present invention is shown. Figure 2 As shown, the device 200 includes:

[0098] The acquisition module 210 is used to acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl to be classified;

[0099] A first classification module 220 is used to input the pearl image into a preset first classification model to obtain a first classification result; the first classification model is obtained by inputting a convolutional neural network training according to a preset pearl image sample; the first classification result includes one of seawater pearls and freshwater pearls;

[0100] The second classification module 230 is used to input the pearl image into a preset second classification model to obtain a second classification result; the second classification model is trained by inputting a deep residual network according to a preset pearl image sample; the second classification result includes one of seawater pearls and freshwater pearls;

[0101] A third classification module 240 is used to input the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained according to a preset pearl image sample input into a time series model; and the third classification result includes one of seawater pearls and freshwater pearls;

[0102] The target classification module 250 is used to determine the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result; the target category includes one of seawater pearls and freshwater pearls.

[0103] In an optional manner, the method of obtaining the pearl image of the pearl to be classified further includes: using a magnification system with a preset magnification to photograph the curved sphere pearl to be classified from at least three different directions to obtain at least three pearl images taken from different directions containing growth line features, halo color features and body color features; wherein the magnification system includes an electron microscope, a CCD and a chassis spring bracket; the electron microscope includes an electronic fixed-focus macro lens and a light source, the light source is arranged at the electronic fixed-focus macro lens, and is used to provide a light source for the pearl to be classified; the electronic fixed-focus macro lens is set to a fixed magnification, and is used to obtain the image of the pearl to be classified; the spring bracket is arranged below the electronic fixed-focus macro lens, and is used to clamp the pearl to be classified, and the spring bracket can move away from or close to the lens direction of the electronic fixed-focus macro lens according to a preset step length; the CCD is electrically connected to the electronic fixed-focus macro lens, and is used to obtain and transmit the pearl image taken by the electronic fixed-focus macro lens.

[0104] In an optional manner, the first classification model includes a first initial classification model and a second initial classification model connected in series; the convolutional neural network is an Inception algorithm; a pearl image sample includes three sample images of a sample pearl in different directions and a first sample label, and the first sample label includes an imitation pearl label and a pearl label; the sample image belonging to the pearl label also carries a second sample label, and the second sample label includes a seawater pearl label and a freshwater pearl label. The device 200 includes:

[0105] A first extraction module is used to extract the RGB value of the pixel in the pearl image sample and convert the RGB value into an HSL value;

[0106] A first training module is used to input the pearl image sample into the Inception algorithm for training, calculate a first loss according to a first sample label and a first loss function, adjust the parameters of the convolutional neural network according to the first loss, and obtain a first initial classification model after training;

[0107] The first mathematical calculation module is used to determine the prediction results of the trained third initial classification model for samples, wherein the prediction results include pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculate whether the HSL values ​​of the pearl image samples classified as pearls are within the HSL threshold range of pearls, and whether the HSL values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls; if not within the HSL threshold range, continue to train the trained third initial classification model according to the pearl image samples; if within the HSL threshold range, perform a blind sample test on the trained third initial classification model to obtain a trained third initial classification model;

[0108] The second training module is used to input the HSL value of the pearl image sample belonging to the pearl label into the Inception algorithm for training, and calculate the second loss according to the second sample label and the second loss function, and adjust the parameters of the convolutional neural network according to the second loss until the second initial classification model is obtained. In an optional manner, the second classification model includes a third initial classification model and a fourth initial classification model; before the pearl image is input into the preset second classification model to obtain the second classification result, the device 200 includes:

[0109] A second extraction module is used to extract the RGB values ​​of pixels in the pearl image sample;

[0110] a third training module, for inputting the pearl image samples into a deep residual network, and calculating a third loss according to the first sample label and a third loss function, and adjusting the parameters of the deep residual network according to the third loss to obtain a trained third initial classification model; wherein the deep residual network includes a 1*1 convolution kernel, an N*N convolution kernel, and a 1*1 convolution kernel connected in sequence, wherein N is greater than 1; a second mathematical calculation module, for determining the prediction results of the trained third initial classification model for samples, wherein the prediction results include pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculating whether the RGB values ​​of the pearl image samples classified as pearls are within the RGB threshold range of pearls, and whether the RGB values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls; if not within the RGB threshold range, continuing to train the trained third initial classification model according to the pearl image samples; if within the RGB threshold range, performing a blind sample test on the trained third initial classification model to obtain a trained third initial classification model;

[0111] The fourth training module is used to input the RGB values ​​of the pearl image samples belonging to the pearl label into the deep residual network for training, calculate the fourth loss according to the second sample label and the fourth loss function, and adjust the parameters of the deep residual network according to the fourth loss until a fourth initial classification model is obtained.

[0112] In an optional manner, the time series model integrates the algorithms after Transfromer and KAN; the device 200 includes: a fifth training module, which is used to input the pearl image sample into the Transfromer+KAN algorithm to obtain multiple first input features of the pearl image sample; input the multiple first input features into the Transfromer+KAN algorithm to obtain a first prediction result; calculate a fifth loss according to the first prediction result, the first sample label and the fifth loss function, and adjust the parameters of the Transfromer+KAN algorithm according to the fifth loss until a fifth initial classification model is obtained;

[0113] The sixth training module is used to input the pearl image samples belonging to the pearl label into the Transfromer+KAN algorithm to obtain multiple second input features; input the multiple second input features into the KAN algorithm to obtain a second prediction result; calculate the fifth loss according to the second prediction result, the second sample label and the sixth loss function, and adjust the parameters of the Transfromer+KAN algorithm according to the sixth loss until a sixth initial classification model is obtained.

[0114] In an optional manner, determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result includes:

[0115] A weighted sum is performed according to the first classification model, the second classification model and the third classification model, and the weights corresponding to the first classification model, the second classification model and the third classification model, to obtain the target category of the pearls to be classified.

[0116] In an optional manner, the first classification model, the second classification model and the third classification model are stored in a cloud server. The step of obtaining pearl images of pearls to be classified comprises:

[0117] Uploading the pearl images of the pearls to be classified, which are taken by the microscope, to the cloud server;

[0118] After determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result, the method further includes:

[0119] The cloud server sends the target category to the user terminal. The embodiment of the present invention obtains a pearl image containing the growth line characteristics of the pearl through an amplification system, identifies the surface growth line characteristics of the pearl, and sets three different types of classification models to identify the image from different angles respectively. The local field of view is increased by the Inception algorithm, the resnet residual network focuses on the image texture, and the time series model in this application increases the global field of view, so that the image is accurately judged locally and globally, and the final pearl category is determined according to the classification results of the three, so that the accuracy of distinguishing seawater pearls from freshwater pearls can be effectively improved.

[0120] Figure 3 The schematic diagram of the structure of the computer device provided by the embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0121] like Figure 3 As shown, the computer device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0122] The processor 402, the communication interface 404, and the memory 406 communicate with each other via the communication bus 408. The communication interface 404 is used to communicate with other devices such as a client or other server network elements. The processor 402 is used to execute the program 410, which can specifically execute the relevant steps in the above-mentioned method embodiment for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculations.

[0123] Specifically, the program 410 may include program code including computer executable instructions.

[0124] Processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs. Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk storage.

[0125] The program 410 may be specifically called by the processor 402 to enable the computer device to perform the following operations:

[0126] Acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl to be classified;

[0127] Inputting the pearl image into a preset first classification model to obtain a first classification result; the first classification model is obtained by inputting a convolutional neural network training according to a preset pearl image sample; the first classification result includes one of seawater pearls and freshwater pearls;

[0128] Inputting the pearl image into a preset second classification model to obtain a second classification result; the second classification model is obtained by inputting a deep residual network training according to a preset pearl image sample; the second classification result includes one of seawater pearls and freshwater pearls;

[0129] Input the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained according to a preset pearl image sample input into a time series model; and the third classification result includes one of seawater pearls and freshwater pearls;

[0130] According to the first classification result, the second classification result and the third classification result, a target category of the pearls to be classified is determined; the target category includes one of seawater pearls and freshwater pearls.

[0131] In an optional manner, the step of obtaining pearl images of pearls to be classified further comprises:

[0132] The curved spherical pearls to be classified are photographed from at least three different directions using a magnification system with a preset magnification, to obtain at least three pearl images taken from different directions containing growth line features, halo color features and body color features; wherein the magnification system includes an electron microscope, a CCD and a chassis spring bracket;

[0133] The electron microscope device comprises an electronic fixed-focus macro lens and a light source, wherein the light source is arranged at the electronic fixed-focus macro lens and is used to provide a light source for the pearls to be classified; the electronic fixed-focus macro lens is set to a fixed magnification and is used to obtain images of the pearls to be classified;

[0134] The spring bracket is arranged below the electronic fixed-focus macro lens and is used to clamp the pearls to be classified. The spring bracket can move away from or approach the lens direction of the electronic fixed-focus macro lens according to a preset step length. The CCD is electrically connected to the electronic fixed-focus macro lens and is used to acquire and transmit the pearl image taken by the electronic fixed-focus macro lens.

[0135] In an optional manner, the first classification model includes a first initial classification model and a second initial classification model connected in series; the convolutional neural network is an Inception algorithm; a pearl image sample includes three sample images of a sample pearl in different directions and a first sample label, and the first sample label includes an imitation pearl label and a pearl label; the sample image belonging to the pearl label also carries a second sample label, and the second sample label includes a seawater pearl label and a freshwater pearl label;

[0136] Before inputting the pearl image into a preset first classification model to obtain a first classification result, the method further comprises:

[0137] Extracting RGB values ​​of pixels in the pearl image sample, and converting the RGB values ​​into HSL values;

[0138] Inputting the pearl image into the Inception algorithm for training, calculating a first loss according to a first sample label and a first loss function, adjusting the parameters of the convolutional neural network according to the first loss, and obtaining a trained first initial classification model;

[0139] Determine the prediction results of the trained first initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; calculate whether the HSL values ​​of the pearl image samples classified as pearls are within the HSL threshold range of pearls, and whether the HSL values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls;

[0140] If it is not within the HSL threshold range, continue to train the trained first initial classification model according to the pearl image sample; if it is within the HSL threshold range, perform a blind sample test on the trained first initial classification model to obtain a trained first initial classification model;

[0141] The pearl image samples belonging to the pearl label are input into the Inception algorithm for training, and the second loss is calculated according to the second sample label and the second loss function, and the parameters of the convolutional neural network are adjusted according to the second loss until a second initial classification model is obtained.

[0142] In an optional manner, the second classification model includes a third initial classification model and a fourth initial classification model; before inputting the pearl image into the preset second classification model to obtain the second classification result, the method includes:

[0143] Extracting RGB values ​​of pixels in the pearl image sample;

[0144] Inputting the pearl image sample into a deep residual network, calculating a third loss according to the first sample label and a third loss function, adjusting the parameters of the deep residual network according to the third loss, and obtaining a trained third initial classification model; wherein the deep residual network includes a 1*1 convolution kernel, an N*N convolution kernel, and a 1*1 convolution kernel connected in sequence, wherein N is greater than 1;

[0145] Determine the prediction results of the trained third initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculate whether the RGB values ​​of the pearl image samples classified as pearls are within the RGB threshold range of pearls, and whether the RGB values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls;

[0146] If it is not within the RGB threshold range, continue to train the trained third initial classification model according to the pearl image sample; if it is within the RGB threshold range, perform a blind sample test on the trained third initial classification model to obtain a trained third initial classification model;

[0147] The RGB values ​​of the pearl image samples belonging to the pearl label are input into the deep residual network for training, and the fourth loss is calculated according to the second sample label and the fourth loss function, and the parameters of the deep residual network are adjusted according to the fourth loss until a fourth initial classification model is obtained.

[0148] In an optional manner, the time series model integrates the algorithms after Transfromer and KAN; before inputting the pearl image into a preset third classification model to obtain the third classification result, the method includes:

[0149] Inputting the pearl image sample into the Transfromer+KAN algorithm to obtain a plurality of first input features of the pearl image sample; inputting the plurality of first input features into the Transfromer+KAN algorithm to obtain a first prediction result;

[0150] Calculating a fifth loss according to the first prediction result, the first sample label and a fifth loss function, and adjusting parameters of the Transfromer+KAN algorithm according to the fifth loss until a fifth initial classification model is obtained;

[0151] Input the pearl image samples belonging to the pearl label into the Transfromer+KAN algorithm to obtain multiple second input features;

[0152] Inputting the plurality of second input features into the KAN algorithm to obtain a second prediction result;

[0153] A fifth loss is calculated according to the second prediction result, the second sample label and the sixth loss function, and the parameters of the Transfromer+KAN algorithm are adjusted according to the sixth loss until a sixth initial classification model is obtained.

[0154] In an optional manner, determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result includes:

[0155] A weighted sum is performed according to the first classification model, the second classification model and the third classification model, and the weights corresponding to the first classification model, the second classification model and the third classification model, to obtain the target category of the pearls to be classified.

[0156] In an optional manner, the first classification model, the second classification model and the third classification model are stored in a cloud server;

[0157] The step of obtaining pearl images of pearls to be classified comprises:

[0158] Uploading the pearl images of the pearls to be classified, which are taken by the microscope, to the cloud server;

[0159] After determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result, the method further includes:

[0160] The cloud server sends the target category to the user terminal.

[0161] The embodiment of the present invention obtains a pearl image including the growth line characteristics, halo color and body color of the pearl through an amplification system, identifies the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl, and sets three different types of classification models according to the characteristics of the pearl, respectively identifies the image from different angles, realizes local and global accurate judgment of the image, and determines the final pearl category according to the classification results of the three, so as to effectively improve the accuracy of distinguishing seawater pearls from freshwater pearls.

[0162] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on a computer device, the computer device executes the method for distinguishing between seawater and freshwater pearls based on deep learning and mathematical calculation in any of the above method embodiments.

[0163] The executable instructions can be specifically used to cause the computer device to perform the following operations:

[0164] Acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl to be classified;

[0165] Inputting the pearl image into a preset first classification model to obtain a first classification result; the first classification model is obtained by inputting a convolutional neural network training according to a preset pearl image sample; the first classification result includes one of seawater pearls and freshwater pearls;

[0166] Input the pearl image into a preset second classification model to obtain a second classification result; the second classification model is trained by inputting a deep residual network according to preset pearl image samples; the second classification result includes one of seawater pearls and freshwater pearls; input the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained by inputting a time series model according to preset pearl image samples; the third classification result includes one of seawater pearls and freshwater pearls;

[0167] According to the first classification result, the second classification result and the third classification result, a target category of the pearls to be classified is determined; the target category includes one of seawater pearls and freshwater pearls.

[0168] In an optional manner, the step of obtaining pearl images of pearls to be classified further comprises:

[0169] Using a magnification system with a preset magnification, the curved spherical pearl to be classified is photographed from at least three different directions to obtain at least three pearl images taken from different directions that include growth line features, halo color features, and body color features;

[0170] Wherein, the magnification system includes an electronic microscope, a CCD and a chassis spring bracket;

[0171] The electron microscope device comprises an electronic fixed-focus macro lens and a light source, wherein the light source is arranged at the electronic fixed-focus macro lens and is used to provide a light source for the pearls to be classified; the electronic fixed-focus macro lens is set to a fixed magnification and is used to obtain images of the pearls to be classified;

[0172] The spring bracket is arranged below the electronic fixed-focus macro lens and is used to clamp the pearls to be classified. The spring bracket can move away from or approach the lens direction of the electronic fixed-focus macro lens according to a preset step length. The CCD is electrically connected to the electronic fixed-focus macro lens and is used to acquire and transmit the pearl image taken by the electronic fixed-focus macro lens.

[0173] In an optional manner, the first classification model includes a first initial classification model and a second initial classification model connected in series; the convolutional neural network is an Inception algorithm; a pearl image sample includes three sample images of a sample pearl in different directions and a first sample label, and the first sample label includes an imitation pearl label and a pearl label; the sample image belonging to the pearl label also carries a second sample label, and the second sample label includes a seawater pearl label and a freshwater pearl label;

[0174] Before inputting the pearl image into a preset first classification model to obtain a first classification result, the method further comprises:

[0175] Extracting RGB values ​​of pixels in the pearl image sample, and converting the RGB values ​​into HSL values;

[0176] Inputting the pearl image into the Inception algorithm for training, calculating a first loss according to a first sample label and a first loss function, adjusting the parameters of the convolutional neural network according to the first loss, and obtaining a trained first initial classification model;

[0177] Determine the prediction results of the trained first initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; calculate whether the HSL values ​​of the pearl image samples classified as pearls are within the HSL threshold range of pearls, and whether the HSL values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls;

[0178] If it is not within the HSL threshold range, continue to train the trained first initial classification model according to the pearl image sample; if it is within the HSL threshold range, perform a blind sample test on the trained first initial classification model to obtain a trained first initial classification model;

[0179] The pearl image samples belonging to the pearl label are input into the Inception algorithm for training, and the second loss is calculated according to the second sample label and the second loss function, and the parameters of the convolutional neural network are adjusted according to the second loss until a second initial classification model is obtained.

[0180] In an optional manner, the second classification model includes a third initial classification model and a fourth initial classification model; before inputting the pearl image into the preset second classification model to obtain the second classification result, the method includes:

[0181] Extracting RGB values ​​of pixels in the pearl image sample;

[0182] Inputting the pearl image sample into a deep residual network, calculating a third loss according to the first sample label and a third loss function, adjusting the parameters of the deep residual network according to the third loss, and obtaining a trained third initial classification model; wherein the deep residual network includes a 1*1 convolution kernel, an N*N convolution kernel, and a 1*1 convolution kernel connected in sequence, wherein N is greater than 1;

[0183] Determine the prediction results of the trained third initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculate whether the RGB values ​​of the pearl image samples classified as pearls are within the RGB threshold range of pearls, and whether the RGB values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls;

[0184] If it is not within the RGB threshold range, continue to train the trained third initial classification model according to the pearl image sample; if it is within the RGB threshold range, perform a blind sample test on the trained third initial classification model to obtain a trained third initial classification model;

[0185] The RGB values ​​of the pearl image samples belonging to the pearl label are input into the deep residual network for training, and the fourth loss is calculated according to the second sample label and the fourth loss function, and the parameters of the deep residual network are adjusted according to the fourth loss until a fourth initial classification model is obtained.

[0186] In an optional manner, the time series model integrates the algorithms after Transfromer and KAN; before inputting the pearl image into a preset third classification model to obtain the third classification result, the method includes:

[0187] Inputting the pearl image sample into the Transfromer+KAN algorithm to obtain a plurality of first input features of the pearl image sample;

[0188] Inputting the plurality of first input features into the Transfromer+KAN algorithm to obtain a first prediction result;

[0189] Calculating a fifth loss according to the first prediction result, the first sample label and a fifth loss function, and adjusting parameters of the Transfromer+KAN algorithm according to the fifth loss until a fifth initial classification model is obtained;

[0190] Input the pearl image samples belonging to the pearl label into the Transfromer+KAN algorithm to obtain multiple second input features;

[0191] Inputting the plurality of second input features into the KAN algorithm to obtain a second prediction result;

[0192] A fifth loss is calculated according to the second prediction result, the second sample label and the sixth loss function, and the parameters of the Transfromer+KAN algorithm are adjusted according to the sixth loss until a sixth initial classification model is obtained.

[0193] In an optional manner, determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result includes:

[0194] A weighted sum is performed according to the first classification model, the second classification model and the third classification model, and the weights corresponding to the first classification model, the second classification model and the third classification model, to obtain the target category of the pearls to be classified.

[0195] In an optional manner, the first classification model, the second classification model and the third classification model are stored in a cloud server;

[0196] The step of obtaining pearl images of pearls to be classified comprises:

[0197] Uploading the pearl images of the pearls to be classified, which are taken by the microscope, to the cloud server;

[0198] After determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result, the method further includes:

[0199] The cloud server sends the target category to the user terminal.

[0200] The embodiment of the present invention obtains a pearl image including the growth line characteristics, halo color and body color of the pearl through an amplification system, identifies the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl, and sets three different types of classification models according to the characteristics of the pearl, respectively identifies the image from different angles, realizes local and global accurate judgment of the image, and determines the final pearl category according to the classification results of the three, so as to effectively improve the accuracy of distinguishing seawater pearls from freshwater pearls.

[0201] An embodiment of the present invention provides a device for distinguishing seawater pearls from freshwater pearls based on deep learning and mathematical calculations, which is used to execute the above-mentioned method for distinguishing seawater pearls from freshwater pearls based on deep learning and mathematical calculations.

[0202] An embodiment of the present invention provides a computer program, which can be called by a processor to enable a computer device to execute the method of distinguishing seawater and freshwater pearls based on deep learning and mathematical calculations in any of the above method embodiments.

[0203] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are run on a computer, the computer executes the method for distinguishing between seawater and freshwater pearls based on deep learning and mathematical calculations in any of the above-mentioned method embodiments.

[0204] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0205] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0206] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than those expressly recited in each claim.

[0207] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0208] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A method for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculation, characterized in that: The method comprises: Acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl to be classified; The pearl image is input into a preset first classification model to obtain a first classification result; the first classification model is obtained by inputting a convolutional neural network training according to a preset pearl image sample; the first classification result includes one of seawater pearls and freshwater pearls; the first classification model includes a first initial classification model and a second initial classification model connected in series in sequence; the convolutional neural network is an Inception algorithm; a pearl image sample includes three sample images of a sample pearl in different directions and a first sample label, and the first sample label includes an imitation pearl label and a pearl label; the sample image belonging to the pearl label also carries a second sample label, and the second sample label includes a seawater pearl label and a freshwater pearl label; Input the pearl image into a preset second classification model to obtain a second classification result; the second classification model is obtained by inputting a preset pearl image sample into a deep residual network training; the second classification result includes one of seawater pearls and freshwater pearls; the second classification model includes a third initial classification model and a fourth initial classification model; Input the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained by inputting a preset pearl image sample into a time series model; the third classification result includes one of seawater pearls and freshwater pearls; the time series model is an algorithm after integrating Transfromer and KAN; According to the first classification result, the second classification result and the third classification result, a target category of the pearls to be classified is determined; the target category includes one of seawater pearls and freshwater pearls.

2. The method according to claim 1, characterized in that The step of obtaining pearl images of pearls to be classified further comprises: Using a magnification system with a preset magnification, the curved spherical pearl to be classified is photographed from at least three different directions to obtain at least three pearl images taken from different directions that include growth line features, halo color features, and body color features; Wherein, the magnification system includes an electronic microscope, a CCD and a chassis spring bracket; The electron microscope device comprises an electronic fixed-focus macro lens and a light source, wherein the light source is arranged at the electronic fixed-focus macro lens and is used to provide a light source for the pearls to be classified; the electronic fixed-focus macro lens is set to a fixed magnification and is used to obtain images of the pearls to be classified; The spring bracket is arranged below the electronic fixed-focus macro lens and is used to clamp the pearls to be classified. The spring bracket can move away from or approach the lens direction of the electronic fixed-focus macro lens according to a preset step length. The CCD is electrically connected to the electronic fixed-focus macro lens and is used to acquire and transmit the pearl image taken by the electronic fixed-focus macro lens.

3. The method according to claim 1, characterized in that Before inputting the pearl image into a preset first classification model to obtain a first classification result, the method further comprises: Extracting RGB values ​​of pixels in the pearl image sample, and converting the RGB values ​​into HSL values; Inputting the pearl image sample into the Inception algorithm for training, calculating a first loss according to a first sample label and a first loss function, adjusting the parameters of the convolutional neural network according to the first loss, and obtaining a trained first initial classification model; Determine the prediction results of the trained first initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; calculate whether the HSL values ​​of the pearl image samples classified as pearls are within the HSL threshold range of pearls, and whether the HSL values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls; If it is not within the HSL threshold range, continue to train the trained first initial classification model according to the pearl image sample; if it is within the HSL threshold range, perform a blind sample test on the trained first initial classification model to obtain a trained first initial classification model; The pearl image samples belonging to the pearl label are input into the Inception algorithm for training, and the second loss is calculated according to the second sample label and the second loss function, and the parameters of the convolutional neural network are adjusted according to the second loss until a second initial classification model is obtained.

4. The method according to claim 3, characterized in that Before inputting the pearl image into a preset second classification model to obtain a second classification result, the method includes: Extracting RGB values ​​of pixels in the pearl image sample; Inputting the pearl image sample into a deep residual network, calculating a third loss according to the first sample label and a third loss function, adjusting the parameters of the deep residual network according to the third loss, and obtaining a trained third initial classification model; wherein the deep residual network includes a 1*1 convolution kernel, an N*N convolution kernel, and a 1*1 convolution kernel connected in sequence, wherein N is greater than 1; Determine the prediction results of the trained third initial classification model for the samples, the prediction results including pearl image samples classified as pearls and pearl image samples classified as imitation pearls; respectively calculate whether the RGB values ​​of the pearl image samples classified as pearls are within the RGB threshold range of pearls, and whether the RGB values ​​of the pearl images classified as imitation pearls are within the threshold range of imitation pearls; If it is not within the RGB threshold range, continue to train the trained third initial classification model according to the pearl image sample; if it is within the RGB threshold range, perform a blind sample test on the trained third initial classification model to obtain a trained third initial classification model; The RGB values ​​of the pearl image samples belonging to the pearl label are input into the deep residual network for training, and the fourth loss is calculated according to the second sample label and the fourth loss function, and the parameters of the deep residual network are adjusted according to the fourth loss until a fourth initial classification model is obtained.

5. The method according to claim 3, characterized in that: Before inputting the pearl image into a preset third classification model to obtain a third classification result, the method includes: Inputting the pearl image sample into the Transfromer+KAN algorithm to obtain a plurality of first input features of the pearl image sample; Inputting the plurality of first input features into the Transfromer+KAN algorithm to obtain a first prediction result; Calculating a fifth loss according to the first prediction result, the first sample label and a fifth loss function, and adjusting parameters of the Transfromer+KAN algorithm according to the fifth loss until a fifth initial classification model is obtained; Input the pearl image samples belonging to the pearl label into the Transfromer+KAN algorithm to obtain multiple second input features; Inputting the plurality of second input features into the KAN algorithm to obtain a second prediction result; A sixth loss is calculated according to the second prediction result, the second sample label and the sixth loss function, and the parameters of the Transfromer+KAN algorithm are adjusted according to the sixth loss until a sixth initial classification model is obtained.

6. The method according to any one of claims 1 to 5, characterized in that: Determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result includes: A weighted sum is performed according to the first classification model, the second classification model and the third classification model, as well as the weights corresponding to the first classification model, the second classification model and the third classification model, to obtain the target category of the pearls to be classified; wherein the weights are determined by back propagation iteration of mathematical logic on the first classification model, the second classification model and the third classification model based on the blind sample pearls.

7. The method according to any one of claims 1 to 5, characterized in that: The first classification model, the second classification model and the third classification model are stored in a cloud server; The step of obtaining pearl images of pearls to be classified comprises: Uploading the pearl images of the pearls to be classified, which are taken by the microscope, to the cloud server; After determining the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result, the method further includes: The cloud server sends the target category to the user terminal.

8. A device for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculations, characterized in that: The device comprises: An acquisition module is used to acquire a pearl image of the pearl to be classified; the pearl image includes the surface growth line characteristics, halo color characteristics and body color characteristics of the pearl to be classified; The first classification module is used to input the pearl image into a preset first classification model to obtain a first classification result; the first classification model is obtained by inputting a convolutional neural network training according to a preset pearl image sample; the first classification result includes one of seawater pearls and freshwater pearls; the first classification model includes a first initial classification model and a second initial classification model connected in series; the convolutional neural network is an Inception algorithm; a pearl image sample includes three sample images of a sample pearl in different directions and a first sample label, the first sample label includes an imitation pearl label and a pearl label; the sample image belonging to the pearl label also carries a second sample label, and the second sample label includes a seawater pearl label and a freshwater pearl label; A second classification module is used to input the pearl image into a preset second classification model to obtain a second classification result; the second classification model is obtained by inputting a deep residual network training according to a preset pearl image sample; the second classification result includes one of seawater pearls and freshwater pearls; the second classification model includes a third initial classification model and a fourth initial classification model; The third classification module is used to input the pearl image into a preset third classification model to obtain a third classification result; wherein the third classification model is trained according to the preset pearl image sample input into the time series model; the third classification result includes one of seawater pearls and freshwater pearls; the time series model is an algorithm after integrating Transfromer and KAN; The target classification module is used to determine the target category of the pearls to be classified according to the first classification result, the second classification result and the third classification result; the target category includes one of seawater pearls and freshwater pearls.

9. A computer device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the method for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculation as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction, and when the executable instruction is executed on a computer device, the computer device executes the operation of the method for distinguishing seawater and freshwater pearls based on deep learning and mathematical calculation as described in any one of claims 1 to 7.

Images