A multimodal semi-supervised learning jade bracelet intelligent identification and evaluation method

Abstract

This invention discloses a multimodal semi-supervised learning-based intelligent identification and valuation method for jade bracelets, relating to the field of jewelry appraisal. The method includes the following steps: S1 fusing cross-modal features; S2 improving the image encoder based on MobileNet V4; S3 using the MSR algorithm to enhance the image; S4 adding an image multidimensional attention feature fusion module; S5 multimodal feature projection; S6 semi-supervised learning; and S7 model inference and price evaluation. This invention, through multimodal semi-supervised learning technology, solves the problem of existing technologies struggling to balance accuracy, efficiency, and data utilization efficiency, achieving the identification of authenticity, price range prediction, and multimodal data fusion processing of jade bracelets.

Description

Technical Field

[0001] This invention belongs to the field of jewelry appraisal, and specifically refers to a multimodal semi-supervised learning-based intelligent appraisal and valuation method for jadeite bracelets. Background Technology

[0002] As a high-value jewelry commodity, jadeite has long faced the problem of information asymmetry in its market transactions. Traditional appraisal and valuation rely on human experience, which has three major pain points: First, the appraisal process is significantly affected by subjective factors. Different appraisers have individual differences in their judgments of characteristics such as texture, color, and flaws, resulting in a lack of objectivity and consistency in the evaluation results. Second, the appraisal process of institutions is cumbersome and time-consuming. Third, the factors affecting the value of jadeite are complex, covering multiple dimensions such as color, texture, size, and flaws. High-quality labeled data requires professional knowledge, resulting in an extreme scarcity of publicly available datasets with price labels, which affects the application of artificial intelligence technology in this area.

[0003] Existing technologies have limitations in addressing the aforementioned issues: unimodal models rely solely on a single data type, such as images or text, like image classification models based on convolutional neural networks. They cannot fully utilize key information in text descriptions, such as the shape and type of plants, resulting in low accuracy in identification and valuation. Traditional multimodal models employ simple feature concatenation, failing to effectively address cross-modal feature alignment issues and exhibiting high computational complexity, making them difficult to deploy on mobile or lightweight devices. Furthermore, existing methods overly rely on supervised learning, underutilizing large amounts of unlabeled data, leading to a significant decrease in model generalization ability in scenarios where labeled data is scarce. Existing technologies in the field of intelligent jadeite bracelet evaluation suffer from three key shortcomings: First, they lack an efficient multimodal fusion mechanism for jadeite features, failing to effectively extract cross-modal correlation information between image texture and color and text (such as type, color, and size). Second, the combination of semi-supervised learning and contrastive learning is insufficient, failing to fully utilize the contrastive loss function to align the semantic features of images and text, resulting in the underutilization of unlabeled data value and poor model performance in scenarios with few samples. Third, it is difficult to balance inference efficiency and accuracy; traditional deep learning models such as ViT, while achieving high accuracy, have a large number of parameters and long detection times, failing to meet the application requirements of real-time identification and valuation. In summary, existing technologies struggle to balance accuracy, efficiency, and data utilization efficiency, necessitating innovative solutions that integrate multimodal learning, semi-supervised learning, and lightweight design. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention aims to provide a multimodal semi-supervised learning-based intelligent identification and valuation method for jade bracelets. Specific objectives include: replacing human experience with multimodal deep learning algorithms to reduce the influence of subjective factors and output standardized evaluation results to improve the objectivity of identification and valuation; constructing a semi-supervised learning framework, combining a small amount of labeled data with a large amount of unlabeled data to train the model, alleviating the problem of scarce labeled data and reducing data labeling costs; and designing a dedicated feature fusion module and a contrastive loss function to achieve efficient alignment and complementary utilization of image and text features to optimize multimodal fusion efficiency.

[0005] This invention provides a multimodal semi-supervised learning-based intelligent identification and valuation method for jade bracelets, comprising the following steps:

[0006] S1. Fusing cross-modal features: The model structure includes two branches: an image encoder and a text encoder. After the two branches extract image and text features respectively, they are projected into a common feature space to fuse cross-modal features, which prepares for subsequent semi-supervised learning to align cross-modal features. This cross-modal fusion design aims to make full use of image and text information to improve the accuracy and generalization ability of identification and evaluation.

[0007] S2. Improved image encoder based on MobileNet V4: The image encoder is designed based on MobileNet V4, retaining its inverted residual blocks and lightweight convolutional structure to ensure inference speed. On the basis of MobileNet V4 as the backbone network, an illumination robust module and an image multidimensional attention feature fusion module are added to the front and back ends of the backbone network, respectively.

[0008] S3. Image Enhancement Using the MSR Algorithm: When observing a jade bracelet, lighting significantly impacts its color, texture, and other attributes, which in turn directly affect the accuracy of the valuation. Therefore, before inputting the bracelet image into the image encoder model for feature extraction, the Multi-Scale Retinex (MSR) algorithm is used to enhance the image. This compresses the image's dynamic range while maintaining color consistency and detail, thus reducing the impact of lighting on the evaluation results. The MSR algorithm's steps for inputting the jade bracelet's RGB image are as follows:

[0009] S31. Divide the image into channels for processing. Convert the pixel value of each channel from integer (int) to floating-point (float) and then to the logarithmic field.

[0010] S32. Input the Gaussian surround scale C (range: [80-100]), discretize the integral operation, convert it into a summation operation, and determine the value of λ through formulas (2) and (3);

[0011] S33. Use formula (1) to calculate r(x,y) for the red, green and blue channels of the image;

[0012] S34. Transform the r(x,y) values ​​of the three channels from the logarithmic domain to the real domain and stitch them together along the channel dimensions to obtain the reflection image R(x,y);

[0013] S35. After normalizing the reflection image R(x,y), input it into the subsequent image encoder to extract features;

[0014] S4. Adding a Multi-Dimensional Attention Feature Fusion Module: Among various types of bracelets, there is no clear distinction between the internal classifications of their appearance attributes (e.g., there is no clear distinction between "glutinous" and "icy glutinous" types of jadeite). Therefore, the image encoder model needs strong feature extraction capabilities to ensure that the features of each appearance attribute have clear distinguishability. In the MobileNet V4 architecture, we added a multi-dimensional attention feature fusion module to improve the image encoder model's ability to distinguish the internal classifications of attributes, thereby improving the model's estimation accuracy. The multi-dimensional attention feature fusion module consists of three branches, each handling the correlation between different combinations of image dimensions. This design promotes more refined interactions between feature dimensions, thereby capturing richer spatial and channel correlations in the image and focusing on the features of the bracelet's appearance attributes, thus improving the model's estimation accuracy. All three branches use self-attention modules, allowing the model to construct attention to bracelet attributes from three dimensions, enhancing the expressive power of key features such as color distribution and jadeite characteristics.

[0015] S5. Multimodal Feature Projection: After the text and image encoders process the text-image pair, image features v and text features t are obtained respectively. v and t need to be projected into a multimodal space. As shown in formula (4), where N represents the number of price ranges for jade bracelets, and p represents the feature dimension after text encoding. In practical use, N is taken as 8. Furthermore, the two feature vectors are adjusted using a subsequent loss function to align them semantically.

[0016] s = V * T = g V (v)*g T (t) (4)

[0017] Where g V (v) and g T (t) is a linear layer that projects image and text features onto the multimodal feature space, respectively. V and T represent the results of the image and text after passing through the linear layer, and * represents the matrix multiplication operation.

[0018] S6. Semi-supervised learning: The semi-supervised learning process is divided into unsupervised and supervised stages. In unsupervised learning, the contrastive loss function is used to semantically align image features and text features, using text descriptions to filter specific image features. After completing the image feature filtering, supervised learning is needed using the price tag of the jade bracelet to transform the image features into a judgment of the bracelet's price.

[0019] S7. Model Inference and Price Evaluation: In model inference, price evaluation mainly assesses the price range of the jade bracelet. During inference, both text and images are input simultaneously; the text-image pair is processed by a text encoder and an image encoder to output text and image features, thus constructing a multimodal space s. From the multimodal space s, the image features v of each dimension are obtained. i The maximum value of all text features t is ultimately the probability distribution of various price ranges, as shown in formula (11):

[0020]

[0021] The price with the highest probability is determined as the final valuation of the bracelet. If the label with a price of "0" has the highest probability, the image is considered not to be a jade bracelet and cannot be classified or valued. If the label with a price of "1" has the highest probability, it is considered to be a "B+C" type bracelet and its price is assessed as 0. If other prices have the highest probability, it is considered to be a "A" type bracelet, and its price is the price range label with the highest probability.

[0022] Preferably, in step S1, the image encoder uses MobileNet V4 as the backbone network, and adds an illumination robust module and an image multi-dimensional feature fusion module on this basis, so that it can resist the influence of illumination changes and make the extracted features more discriminative.

[0023] Preferably, in step S1, the text encoder uses a pre-trained BERT-Base-Chinese model. First, the text description is split into character-level or word-level tokens using the built-in tokenizer, which are then converted into input IDs, segment IDs, and attention masks before being input into the model. Then, the CLS token of the last hidden layer of the model is selected as the text feature vector, where the feature vector has a dimension of 768. Finally, layer normalization is used to ensure the stability of cross-batch input.

[0024] Preferably, in step S3, the reflected image R(x,y) can be obtained from the original image S according to formula (1). k The result obtained by decomposing (x,y) is:

[0025]

[0026] in, The convolution operation is represented by the center wrapping function T(x,y) as shown in formula (2):

[0027]

[0028] Where C is the Gaussian wrapping scale, and λ is a scale, the values ​​of both must follow the constraints of formula (3):

[0029] ∫∫T(x,y)dxdy=1(3)

[0030] In step S4, the specific operation steps of the multidimensional attention feature fusion module for the input feature map F are as follows:

[0031] S41. On the left branch, the feature map F is directly input into the self-attention module to obtain the feature weight μ1 of size C×H×W. Then the feature weight μ1 is multiplied element-wise with the feature map F to obtain the feature enhancement map f1 of size C×H×W.

[0032] S42. In the middle and right branches, the feature maps F need to be dimensionally transformed first, and then fed into the self-attention module to obtain feature weights μ2 and μ3 of size H×C×W and W×H×C respectively. μ2 and μ3 are then multiplied element-wise with the feature maps F of the corresponding size to obtain feature enhancement maps f2 and f3 of size H×C×W and size W×H×C respectively.

[0033] S43. Perform convolution operations on three feature enhancement maps of different sizes using a 1×1 convolution kernel to make the feature discrimination of each feature enhancement map greater;

[0034] S44. For the feature enhancement maps f2 and f3 of the middle and right branches, dimensional transformation operations need to be performed separately to obtain feature enhancement maps of size C×H×W. Finally, the three-branch feature enhancement maps f1, f2 and f3 of the same size are superimposed on the channel dimension, thereby enabling the image encoder to improve the recognition of internal classification of different attributes from different perspectives.

[0035] Preferably, in step S6, the image-to-text contrast loss function L v2t The calculation is shown in formula (5):

[0036]

[0037] Text-to-image contrast loss function L t2v The calculation is shown in formula (6):

[0038]

[0039] Where i∈{1...N}, j∈{1...p} represent the feature dimensions of the image and text, respectively. The numerators in formulas (5) and (6) are the similarity between the two embeddings in the matching pair, while the denominator is the sum of all embeddings related to the image feature points v. i and text feature points t j Based on the relevant similarity, we finally add the two loss functions as follows to obtain formula (7), i.e., Loss1:

[0040] Loss1=L v2i +L i2v (7)

[0041] Preferably, in step S6, the supervised learning cross-entropy loss function L price As shown in formula (8):

[0042]

[0043] in, Let represent the probability distribution of the image encoder's predictions for each price category, y represent the one-hot encoding of the price tag, and N represent the number of price categories. To fully utilize text features, we calculate the image-to-text cross-entropy loss function L. i2tC As shown in formula (9):

[0044]

[0045] Finally, we add the two loss functions together to get formula (10), which is Loss2:

[0046] Loss2=L price +L i2tC (10)

[0047] The beneficial effects of the technical solution of this invention are as follows: First, the construction of a cross-modal semi-supervised learning framework can make full use of the existing small amount of labeled data and large amount of unlabeled data to train the model, reducing training costs. Second, a multi-dimensional feature fusion module is designed, and a multi-dimensional attention feature fusion module is introduced into MobileNet V4 to enhance the model's discriminative ability to extract features of jade bracelets, which is helpful for subsequent cross-modal semantic feature alignment. Finally, a bidirectional contrastive loss function of image-to-text and text-to-image is proposed to align cross-modal semantic features, enabling multi-modal information to be combined for evaluation. The accuracy of the multi-modal model is 16.2% higher than that of the single-modal model. Attached Figure Description

[0048] Figure 1 This is a flowchart of the workflow of the intelligent identification and valuation model for jade bracelets based on multimodal semi-supervised learning, as described in this invention.

[0049] Figure 2 This is a schematic diagram of the MobileNet model composition of the present invention;

[0050] Figure 3 This is a schematic diagram of the multidimensional attention feature module of the present invention;

[0051] Figure 4 This is a schematic diagram of the self-attention module of the present invention. Detailed Implementation

[0052] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0053] This invention proposes a multimodal semi-supervised learning model comprising an image encoder, a text encoder, and a multimodal fusion module. It achieves authentication and price range prediction for jade bracelets through joint optimization of unsupervised contrastive learning and supervised classification loss. Specifically, the image encoder employs the lightweight neural network MobileNet V4, embedding a multidimensional attention feature fusion module. It enhances cross-dimensional feature interaction through operations such as dimensionality transformation and channel pooling, improving the discriminative power of visual features like color, texture, and shape. The text encoder uses the pre-trained language model BERT-Base-Chinese to segment, encode, and generate semantic feature vectors from the text description. During multimodal fusion, image and text features are mapped to a unified feature space via linear projection. An unsupervised contrastive loss function between the image and text is used to align cross-modal semantic features, and a supervised cross-entropy loss function is then used to classify price ranges. In the inference phase, the probability distribution of price ranges is calculated by fusing image visual features and text semantic features.

[0054] like Figure 1-4 As shown in the figure, this invention provides a multimodal semi-supervised learning-based intelligent identification and valuation method for jade bracelets, including the following steps:

[0055] S1. Cross-modal feature fusion: The model structure includes two branches: an image encoder and a text encoder. These branches extract image and text features respectively, then project them into a common feature space to fuse cross-modal features. This prepares the model for subsequent semi-supervised learning to align cross-modal features. This cross-modal fusion design aims to fully utilize image and text information to improve the accuracy and generalization ability of identification and evaluation. The image encoder uses MobileNet V4 as its backbone network, adding an illumination robustness module and an image multi-dimensional feature fusion module to resist the effects of illumination changes and make the extracted features more discriminative. The text encoder uses a pre-trained BERT-Base-Chinese model. First, the built-in tokenizer splits the text description into character-level or word-level tokens, converting them into input IDs, segment IDs, and attention masks before inputting them into the model. Then, the CLS token from the last hidden layer of the model is selected as the text feature vector, where the feature vector has a dimension of 768. Finally, layer normalization ensures the stability of cross-batch input.

[0056] S2. Improved image encoder based on MobileNet V4: The image encoder is designed based on MobileNet V4, retaining its inverted residual blocks and lightweight convolutional structure to ensure inference speed. On the basis of MobileNet V4 as the backbone network, an illumination robust module and an image multidimensional attention feature fusion module are added to the front and back ends of the backbone network, respectively.

[0057] S3. Image enhancement using the MSR algorithm: When observing a bracelet, lighting has a significant impact on the observation of its color, texture, and other attributes, and the performance of these external attributes directly affects the accuracy of the valuation. Therefore, before inputting the bracelet image into the image encoder model for feature extraction, the multi-scale retina (MSR) algorithm is used to enhance the image, thereby compressing the dynamic range of the image while maintaining the color consistency and detail of the image, so as to reduce the impact of lighting on the evaluation results. The reflected image R(x,y) can be calculated using formula (1) for the original image S. k The result obtained by decomposing (x,y) is:

[0058]

[0059] in, The convolution operation is represented by the center wrapping function F(x,y) as shown in formula (2):

[0060]

[0061] Where C is the Gaussian wrapping scale, and λ is a scale, the values ​​of both must follow the constraints of formula (3):

[0062] ∫∫F(x,y) dxdy=1 (3)

[0063] The MSR algorithm operates as follows for the input RGB image of a jade bracelet:

[0064] S31. Divide the image into channels for processing. Convert the pixel value of each channel from integer (int) to floating-point (float) and then to the logarithmic field.

[0065] S32. Input the Gaussian surround scale C (range: [80-100]), discretize the integral operation, convert it into a summation operation, and determine the value of λ through formulas (2) and (3);

[0066] S33. Use formula (1) to calculate r(x,y) for the red, green and blue channels of the image;

[0067] S34. Transform the r(x,y) values ​​of the three channels from the logarithmic domain to the real domain and stitch them together along the channel dimensions to obtain the reflection image R(x,y);

[0068] S35. After normalizing the reflection image R(x,y), input it into the subsequent image encoder to extract features;

[0069] S4. Adding a multi-dimensional attention feature fusion module for images: Among various types of bracelets, there is no clear distinction between the internal classifications of their appearance attributes (e.g., there is no clear distinction between glutinous jade and icy glutinous jade). Therefore, the image encoder model needs strong feature extraction capabilities to ensure that the features of each appearance attribute have clear distinguishability. In the MobileNet V4 architecture, we added a multi-dimensional attention feature fusion module (e.g., ... Figure 2 As shown), this improves the image encoder model's ability to distinguish internal attribute classifications, thereby enhancing the model's estimation accuracy; the multi-dimensional attention feature fusion module (as shown in the attached diagram) enhances the model's ability to distinguish internal attribute classifications, thereby improving the model's estimation accuracy; Figure 3 The model (as shown) consists of three branches, each handling the correlations between different combinations of image dimensions. This design facilitates more refined interactions between feature dimensions, thereby capturing richer spatial and channel correlations in the image and focusing on the features of the bracelet's appearance attributes, thus improving the model's estimation accuracy. All three branches utilize a self-attention module (as shown in the attached diagram). Figure 4 As shown in the diagram, the model can construct its focus on the bracelet's attributes from three dimensions, enhancing its ability to express key features such as color distribution and texture. The specific operation steps of the multi-dimensional attention feature fusion module for the input feature map F are as follows:

[0070] S41. On the left branch, the feature map F is directly input into the self-attention module to obtain the feature weight μ1 of size C×H×W. Then the feature weight μ1 is multiplied element-wise with the feature map F to obtain the feature enhancement map f1 of size C×H×W.

[0071] S42. In the middle and right branches, the feature map F needs to be dimensionally transformed first, and then fed into the self-attention module to obtain feature weights μ2 and μ3 of size H×C×W and W×H×C respectively. μ2 and μ3 are multiplied element-wise with feature map F to obtain feature enhancement map f2 of size H×C×W and feature enhancement map f3 of size W×H×C respectively.

[0072] S43. Perform convolution operations on three feature enhancement maps of different sizes using a 1×1 convolution kernel to make the feature discrimination of each feature enhancement map greater;

[0073] S44. For the feature enhancement maps f2 and f3 of the middle and right branches, dimensional transformation operations need to be performed separately to obtain feature enhancement maps of size C×H×W. Finally, the three-branch feature enhancement maps f1, f2 and f3 of the same size are superimposed on the channel dimension, thereby enabling the image encoder to improve the recognition of internal classification of different attributes from different perspectives.

[0074] S5. Multimodal Feature Projection: After the text and image encoders process the text-image pair, image features v and text features t are obtained respectively. v and t need to be projected into a multimodal space. Where N represents the number of price ranges for jade bracelets, and p represents the feature dimension after text encoding. The similarity between the two feature vectors is then calculated using a subsequent loss function, as shown in formula (4):

[0075] s = V * T = g V (v)*g T (t) (4)

[0076] Where g V (v) and g T (t) represents the linear layers that project image and text features onto the multimodal feature space, respectively. V and T represent the results of the image and text after passing through the linear layers, and * represents the matrix multiplication operation. In practical use, N has 8 values, corresponding to 0 (non-jade), 1 (jade processed in B+C category), 2 (below 3999 yuan), 3 (4000-6999 yuan), 4 (7000-10000 yuan), 5 (10000-39999 yuan), 6 (40000-69999 yuan), and 7 (70000-100000 yuan).

[0077] S6. Semi-supervised Learning: The semi-supervised learning process is divided into unsupervised and supervised stages. In unsupervised learning, the contrastive loss function is used to semantically align image features and text features, using text descriptions to filter specific image features. After image feature filtering, supervised learning is needed using the price tag of the jade bracelet to transform image features into judgments about the bracelet's price. The image-to-text contrastive loss function L... v2t The calculation is shown in formula (5):

[0078]

[0079] Text-to-image contrast loss function L t2v The calculation is shown in formula (6):

[0080]

[0081] Where i∈{1...N}, j∈{1...p} represent the feature dimensions of the image and text, respectively. The numerators in formulas (5) and (6) are the similarity between the two embeddings in the matching pair, while the denominator is the sum of all embeddings related to the image feature points v. i and text feature points t j Based on the relevant similarity, we finally add the two loss functions as follows to obtain formula (7), i.e., Loss1:

[0082] Loss1=L v2i +L i2v (7)

[0083] Cross-entropy loss function L in supervised learning price As shown in formula (8):

[0084]

[0085] in, Let represent the probability distribution of the image encoder's predictions for each price category, y represent the one-hot encoding of the price tag, and N represent the number of price categories. To fully utilize text features, we calculate the image-to-text cross-entropy loss function L. i2tC As shown in formula (9):

[0086]

[0087] Finally, we add the two loss functions together to get formula (10), which is Loss2:

[0088] Loss2=L price +L i2tC(10) S7, Model Reasoning and Price Evaluation: In model reasoning, price evaluation mainly assesses the price range of the jadeite bracelet (non-jadeite, B+C grade treated jadeite, below 3999 yuan, 4000-6999 yuan, 7000-10000 yuan, 10000-39999 yuan, 40000-69999 yuan, 70000-100000 yuan). During the reasoning process, text and images need to be input simultaneously; after the text-image pair is processed by the text encoder and image encoder, the text and image features are output to construct the multimodal space s, and the image features v of each dimension are obtained from the multimodal space s. i The maximum value of all text features t is ultimately the probability distribution of various price ranges, as shown in formula (11):

[0089]

[0090] The price with the highest probability is determined as the final valuation of the bracelet. If the label with a price of "0" has the highest probability, the image is considered not to be a jade bracelet and cannot be aligned for classification and valuation. If the label with a price of "1" has the highest probability, it is considered to be a "B+C" type bracelet and its price is assessed as 0. If other prices have the highest probability, it is considered to be a "A" type bracelet, and its price is the price range label with the highest probability.

[0091] This invention utilizes multimodal semi-supervised learning technology to achieve authentication of jadeite bracelets, price range prediction, and multimodal data fusion processing. It is applicable to automated price evaluation of jadeite bracelets in scenarios such as jewelry consumption, e-commerce transactions, and appraisal institutions.

Claims

1. A multimodal semi-supervised learning-based intelligent identification and valuation method for jadeite bracelets, characterized in that: Includes the following steps: S1. Fusion of cross-modal features: The model structure contains two branches: an image encoder and a text encoder. After the two branches extract image and text features respectively, they are projected into a common feature space to fuse cross-modal features. S2. Improved image encoder based on MobileNet V4: The image encoder is designed based on MobileNet V4, retaining its inverted residual blocks and lightweight convolutional structure to ensure inference speed. On the basis of MobileNet V4 as the backbone network, an illumination robust module and an image multidimensional attention feature fusion module are added to the front and back ends of the backbone network respectively. S3. Image enhancement using the MSR algorithm: Before inputting the bracelet image into the image encoder model for feature extraction, the image is enhanced using the MSR algorithm; S4. Add a multi-dimensional attention feature fusion module for images: In the structure of MobileNet V4, a multi-dimensional attention feature fusion module is added; the multi-dimensional attention feature fusion module consists of three branches, which respectively handle the correlation between different combinations of image dimensions; S5. Multimodal Feature Projection: After the text-image pair passes through the text and image encoders, image features are obtained separately. v and text features t It is necessary to v and t Projecting to a cross-modal space , N This indicates the price range for jadeite bracelets. p This represents the feature dimension after text encoding, and the two feature vectors are adjusted by a subsequent loss function to align them in the semantic direction, as shown in formula (4): (4) in g V ( v ) and g T ( t ) are linear layers that project image and text features into a multimodal feature space, respectively. V and T This represents the result of an image and text passing through a linear layer. This represents the matrix multiplication operation; S6. Semi-supervised learning: The semi-supervised learning process is divided into an unsupervised stage and a supervised stage. In unsupervised learning, the contrastive loss function is used to semantically align image features and text features, and text descriptions are used to filter specific image features. After the image feature filtering is completed, the price tag of the jade bracelet is used to perform supervised learning on the image model, and the image features are transformed into a judgment on the price of the bracelet. S7. Model Inference and Price Evaluation: In model inference, both text and images need to be input simultaneously. Text-image pairs are processed by text encoders and image encoders to output text and image features, thereby constructing a multimodal space. s From multimodal space s Each dimension of the image features is obtained. v i For all text features t The maximum value of is the final result of the probability distribution of various price ranges, as shown in formula (11): (11) The price with the highest probability is determined as the final valuation of the bracelet. If the label with a price of "0" has the highest probability, the image is considered not to be a jade bracelet and cannot be classified or valued. If the label with a price of "1" has the highest probability, it is considered to be a "B+C" type bracelet and its price is assessed as 0. If other prices have the highest probability, it is considered to be a "A" type bracelet, and its price is the price range label with the highest probability.

2. The intelligent identification and valuation method for jade bracelets based on multimodal semi-supervised learning according to claim 1, characterized in that: In step S1, the image encoder uses MobileNet V4 as the backbone network, and adds an illumination robust module and an image multi-dimensional feature fusion module on this basis, so that it can resist the influence of illumination changes and make the extracted features more discriminative.

3. The intelligent identification and valuation method for jade bracelets based on multimodal semi-supervised learning according to claim 1, characterized in that: In step S1, the text encoder uses a pre-trained BERT-Base-Chinese model. First, the built-in tokenizer splits the text description into character-level or word-level tokens, which are then converted into input IDs, segment IDs, and attention masks before being input into the model. Then, the CLS token of the last hidden layer of the model is selected as the text feature vector, where the feature vector has a dimension of 768. Finally, layer normalization is used to ensure the stability of cross-batch input.

4. The intelligent identification and valuation method for jade bracelets based on multimodal semi-supervised learning according to claim 1, characterized in that: In step S3, when the image is enhanced using the MSR algorithm, the reflected image... R ( x , y The original image can be processed according to formula (1). S k ( x , y The decomposition yielded the following: (1) in, Represents the convolution operation, center-wrap function T ( x , y ) is represented as in formula (2): (2) in, C It is a Gaussian surround scale. λ It is a scale, and the values ​​of both need to follow the constraints of formula (3): (3)。 5. The intelligent identification and valuation method for jade bracelets based on multimodal semi-supervised learning according to claim 1, characterized in that: In step S3, the MSR algorithm operates as follows on the input RGB image of the jade bracelet: S31. Divide the image into channels for processing. Convert the pixel value of each channel from integer to floating-point and then to the logarithmic field. S32, Input Gaussian Encirclement Scale C Discretize the integral operation and convert it into a summation operation, which is then determined by formulas (2) and (3). λ The value; S33. Calculate the red, green, and blue channels of the image using formula (1) to obtain... r ( x , y ); S34, Transfer the three channels r ( x , y The reflection image is obtained by transforming from the logarithmic field to the real field and stitching along the channel dimension. R ( x , y ); S35, in the reflected image R ( x , y After normalization, the image is input into the subsequent image encoder to extract features.

6. The intelligent identification and valuation method for jade bracelets based on multimodal semi-supervised learning according to claim 1, characterized in that: In step S4, for the input feature map F The specific operation steps of the multi-dimensional attention feature fusion module are as follows: S41. On the left branch, the feature map... F Directly inputting into the self-attention module yields a size of Feature weights μ 1. Then, the feature weights are... μ 1 and feature map F Multiplying by elements, we get a size of Feature Enhancement Map f 1; S42. In the middle and right branches, the feature map needs to be processed first. F Perform dimensional transformations separately, then feed them into the self-attention module to obtain sizes of... Feature weights μ 2 and Feature weights μ 3. μ 2 and μ 3. Feature diagrams of corresponding dimensions F Multiplying by elements, we get the following sizes: Feature Enhancement Map f 2 and size is Feature Enhancement Map f 3; S43, Use The convolution kernel performs a convolution operation on three feature enhancement maps of different sizes, making the feature distinction of each feature enhancement map greater; S44. Feature enhancement maps for the middle and right branches f 2 and f 3. Each of these requires a dimensional transformation operation to obtain a size of... The feature enhancement map is then used to finally create a three-branch feature enhancement map of the same size. f 1. f 2 and f 3. Overlaying is performed on the channel dimension, thereby enabling the image encoder to improve the recognition of internal classification of different attributes from different perspectives.

7. The intelligent identification and valuation method for jade bracelets based on multimodal semi-supervised learning according to claim 1, characterized in that: In step S6, the image-to-text contrast loss function L v2t The calculation is shown in formula (5): (5) Contrast loss function from text to image L t2v The calculation is shown in formula (6): (6) in i ∈{1... N }, j ∈{1... p } represent the feature dimensions of the image and text, respectively. The numerators in formulas (5) and (6) are the similarity between the two embeddings in the matching pair, while the denominators are all the features of the image feature points. v i and text feature points t j The relevant similarity is then calculated by adding the two loss functions together, resulting in formula (7). Loss 1: (7)。 8. The intelligent identification and valuation method for jade bracelets based on multimodal semi-supervised learning according to claim 1, characterized in that: In step S6, the supervised learning cross-entropy loss function L price As shown in formula (8): (8) in, This represents the probability distribution of the image encoder's predictions for each price category. y This indicates the one-hot code of the price tag. N This represents the number of price categories. To fully utilize text features, the image-to-text cross-entropy loss function is calculated. L i2tC As shown in formula (9): (9) Adding these two loss functions together, we get formula (10), that is... Loss 2: (10)。

Images