Sketch image retrieval method based on style adaptive residual network

Abstract

The invention relates to a sketch image retrieval method based on a style adaptive residual network, and belongs to the field of computer vision. The method comprises the following steps: firstly, acquiring a fine-grained sketch data set, and performing feature extraction through a style adaptive residual network; in the network, firstly, features with important semantic information are enhanced by using a channel attention mechanism through a style enhancement door module, so that low-response but key global or local features are effectively transmitted. Then, a style enhancement module is used for adaptively adjusting style enhancement features through a feature relation matrix and learnable transformation operation, and it is ensured that the model does not excessively depend on a local mode of a specific style, but matching can be carried out on the basis of overall structure information; finally, a style adaptive residual network is constructed, so that the model can adaptively adjust feature representation, high robustness is kept under sketch input of different styles, and it is ensured that stable and discriminant depth features can be obtained in the matching stage.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a sketch image retrieval method based on a style adaptive residual network. Background Art

[0002] In recent years, fine-grained sketch image retrieval (FG-SBIR) has attracted extensive research attention in the field of computer vision and has had a positive impact on application areas such as intelligent security, e-commerce, and visual content understanding. The FG-SBIR task is at the core of cross-modal visual matching, and its goal is to accurately match user-drawn sketches with target images in large-scale image databases. Unlike traditional image-based retrieval tasks, the uniqueness of FG-SBIR is reflected in the following aspects: (1) Cross-modal characteristics: Sketches and photos belong to different visual modalities. Sketches are usually users' subjective expressions of objects through simplified lines, while photos are visual captures of real-world objects. This modality difference leads to a significant feature representation gap between sketches and photos. (2) Local feature differences: Although sketches retain the main outline of the target object in terms of overall shape, due to the differences in drawing habits of different users, the omission, exaggeration, or deformation of local details makes cross-modal matching more complicated. In addition, since sketches are usually drawn based on user memory, their proportions, shapes, and feature details often deviate from the real image, further increasing the difficulty of matching.

[0003] Benefiting from the powerful capabilities of deep neural networks in learning feature representations, FG-SBIR has made significant progress in sketch-to-image cross-modal feature mapping, local feature alignment, and adaptive style modeling. In recent years, researchers have explored strategies such as attention mechanisms, feature contrast learning, and triplet loss to capture fine-grained local feature differences between sketches and images and improve the robustness of cross-modal matching. However, due to the high subjectivity and diversity of sketch data, adaptively adjusting for local feature differences across different user styles while maintaining global morphological consistency remains a key challenge in FG-SBIR research. Summary of the Invention

[0004] The purpose of the present invention is to provide a sketch image retrieval method based on a style-adaptive residual network, which can adaptively capture the sketch features under different user drawing styles and effectively compensate for the differences between the sketch and the real image in terms of shape, scale, local details, etc.

[0005] To achieve the above objectives, the technical solution of the present invention is: a sketch image retrieval method based on a style adaptive residual network obtains a fine-grained sketch dataset, and extracts features by constructing and training a style adaptive residual network to obtain fine-grained retrieval results; in the style adaptive residual network, a style enhancement gate module is used to enhance features with important semantic information using a channel attention mechanism; and a style enhancement module is used to adaptively adjust the style enhancement features through a feature relationship matrix and a learnable transformation operation.

[0006] The present invention also provides a sketch image retrieval system based on a style adaptive residual network, comprising a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, it can implement any of the method steps described above.

[0007] The present invention also provides a computer-readable storage medium on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, any of the method steps described above can be implemented.

[0008] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes any of the method steps described above.

[0009] Compared with the prior art, the present invention has the following beneficial effects:

[0010] 1. The present invention constructs a fine-grained sketch image retrieval method based on a style-adaptive residual network, which can adaptively capture sketch features under different user drawing styles and effectively compensate for the differences between sketches and real images in terms of shape, scale, local details, etc.

[0011] 2. Effectively enhance features with low response but important semantic information. Through the channel attention mechanism, while maintaining global feature expression, the feature weights are adaptively adjusted, allowing the model to retain more key information when processing sketches of different styles, thereby improving the accuracy of sketch-to-image matching.

[0012] 3. This method adaptively captures the local structural offsets between sketches and photos without the need for additional manual annotation. It uses a self-attention mechanism to calculate the adaptive weights of local features and employs min-max normalization to ensure that sketches of different scales focus on consistent structural information during the matching process, effectively reducing the impact of local deformation on matching accuracy.

[0013] 4. To address the exaggeration of shapes and omission of details caused by user style differences during sketching, a local region alignment strategy and optimal matching method are used to dynamically adjust local features. Unlike traditional matching methods based on fixed local features, this invention uses weighted matching based on the importance of local features. This ensures that even if the local structure of the sketch varies significantly, the corresponding image region can be effectively found, achieving robust cross-modal matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0016] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0017] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0018] The present invention provides a sketch image retrieval method based on a style adaptive residual network. The method obtains a fine-grained sketch dataset and extracts features by constructing and training a style adaptive residual network to obtain fine-grained retrieval results. In the style adaptive residual network, a style enhancement gate module is used to enhance features with important semantic information using a channel attention mechanism. The style enhancement module is used to adaptively adjust the style enhancement features through a feature relationship matrix and a learnable transformation operation.

[0019] The following is a specific implementation process of the present invention.

[0020] like Figure 1 As shown, this embodiment provides a fine-grained sketch image retrieval method based on a style adaptive residual network, which specifically includes the following steps:

[0021] Step S1: Obtain a fine-grained classification dataset, preprocess the data, expand the dataset using data enhancement technology, and input it into the backbone network to extract mid-level features.

[0022] Step S2: Dynamically adjust the activation strength of the feature channel through the channel attention mechanism to enhance those features that may not be activated by ReLU but still have important semantic information, so that low-response but critical global or local features can be effectively transmitted.

[0023] Step S3: Further adaptively adjust the style enhancement features through the feature relationship matrix and learnable transformation operations to ensure that the model does not overly rely on local patterns of a specific style, but is able to match based on the overall structural information.

[0024] Step S4: Perform iterative training according to the specified training parameters, update the model parameters by optimizing the combined loss, continuously save the optimal model according to the verification accuracy, and use the final model to obtain the fine-grained retrieval results.

[0025] In this embodiment, step S1 specifically includes the following steps:

[0026] Step S11: Use a public fine-grained classification dataset to perform data preprocessing to complete label extraction. Through the regular matching mechanism, automatically match the photo file with the corresponding sketch file to prepare for subsequent model training.

[0027] Step S12: Perform data augmentation operations on the collected data set, and use random cropping, random horizontal inversion, random vertical flipping, and random rotation data augmentation methods to process the data set to obtain an enhanced data set.

[0028] Step S13: The mid-level features of the image are extracted through the ResNet-50 model to capture the local structural information in the sketch and photo images to enhance the feature representation ability of the model. The features are normalized to maintain stability and the normalized features are returned. The overall process is shown as follows:

[0029]

[0030] h spatial Represents the spatial coordinate of the feature map in height, corresponding to the dimension from top to bottom of the image. spatial Represents the spatial coordinates of the feature map in width, corresponding to the dimension from left to right of the image. original (h spatial ,w spatial ) represents the spatial position (h spatial ,w spatial ) is extracted by the middle convolutional layer of ResNet-50. normalized (h spatial ,w spatial) represents the normalized feature vector at the same location, used for subsequent calculations and matching. To prevent the denominator from reaching zero, a minimum value ∈ is introduced to enhance stability. channel_count represents the total number of channels in the feature map, that is, the dimension of the feature vector at each spatial location. channel_index represents the channel index currently being traversed, used for channel-by-channel processing in normalization calculations.

[0031] In this embodiment, step S2 specifically includes the following steps:

[0032] Step S21: Sketch feature F in S13 original After extraction, channel-level feature enhancement is first performed. Convolutional layers adjust the channel dimensions and batch normalization is used to normalize the feature distribution. Subsequently, channel weights are calculated using a sigmoid activation function and element-wise multiplied with the original features to achieve dynamic weighting of the feature channels. The weighted features undergo L2 normalization to maintain amplitude consistency and are then flattened and resized.

[0033] F boosted =σ(BN(Conv attention (F original )))×F original

[0034]

[0035] Among them, Conv attention The convolution operation used for channel dimension mapping, usually 1×1 convolution, is used to generate attention features. Where σ represents the sigmoid function, which maps the features to the interval [0, 1], generates channel-level attention weights, and performs element-wise multiplication to weight the generated channel weights with the original features. Represents the feature map after attention enhancement, which retains the original information while highlighting the key channels.

[0036] Step S21: Calculate sketch features and enhanced features The guidance vector G between sketch_guidance This matrix calculates the similarity of the input features using a similarity formula and generates dynamic feature weights through a Softmax transformation. Ultimately, the weight matrix is ​​element-wise weighted with the features, thereby highlighting the important areas of the spatial features while suppressing redundant information.

[0037] G sketch_guidance =Softmax(Sim(F original ,F boosted ))

[0038] F sketch_refined =Gsketch_guidance ⊙F boosted +F boosted

[0039] Among them, Sim(F original ,F boosted ) represents a sketch feature Sketch features with enhanced style ⊙Element-wise multiplication, used to apply guidance weights to enhanced features. Represents the attention weight coefficient of each spatial position, emphasizing the area with higher similarity to the enhanced feature. The sketch feature representation after the final fusion, in which important areas are enhanced and background or irrelevant areas are suppressed, is used as the input of the downstream module.

[0040] Step S23: Step S22 The convolutional layer achieves linear combination of cross-channel features, enabling the model to capture richer semantic information.

[0041] F channel_mapped_1 =Conv1d stage1 (F sketch_refined )

[0042] F channel_mapped_2 =Conv1d stage2 (F channel_mapped_1 +F original )

[0043] Conv1d stage1 Represents the first stage 1D convolution operation, performed on the channel dimension, used to linearly integrate cross-channel features, Conv1d stage2 The second stage convolution operation further transforms the weighted residual features. Represents the output feature after the first convolution, F channel_mapped_2 It represents the final fusion feature, which combines nonlinear mapping and residual information and serves as the input of subsequent modules.

[0044] Step S24: Through the residual learning mechanism, the enhanced features are fused with the original features to effectively maintain the integrity and information consistency of the feature space. This feature fusion method achieves the complementarity of global and local features without adding additional computational overhead, thereby improving the model's discriminative ability in multi-scale feature spaces.

[0045] F residual_enhanced

[0046] =Conv1d stage1 (Conv1d stage1 (F sketch_refined)+F original )+F sketch_refined

[0047] It represents the final fused output features, which have enhanced semantic expression ability and original structural information for subsequent feature matching or discrimination tasks.

[0048] In this embodiment, step S3 specifically includes the following steps:

[0049] Step S31: Obtained from S24 The self-attention mechanism is used to capture the global relationship of local features of the sketch and dynamically adjust the importance of sketch features. Specifically, we first transform the sketch features It is regarded as a set of (spatial_height×spatial_width)×(spatial_height×spatial_width) local feature points, each of which contains channel_count dimension channel information. The similarity between feature points is calculated by dot product operation, and the self-attention matrix is ​​obtained by softmax normalization.

[0050]

[0051] F residual_enhanced '=A sketch_attention F residual_enhanced

[0052] A sketch_attention It reflects the similarity relationship between feature points in different regions of the sketch. sketch_attention Weighted original feature F residual_enhanced , you can get the enhanced sketch feature

[0053] Step S32: weighted feature F generated by S31 residual_enhanced ', further introduces a feature weight normalization mechanism to ensure that the importance of different regions can be treated consistently during the feature matching process. The weighted sum of each feature point is calculated to measure the importance of the feature point, and the feature weight is adjusted through minimum-maximum normalization:

[0054]

[0055] in, Represents the normalized weight of each feature point, ∈ is a very small positive number (usually 1×10 -8) is used to prevent the denominator from being zero. The normalized weights can dynamically adjust the importance of enlarged or reduced areas in the sketch, ensuring that the model will not be biased by imbalanced area proportions during subsequent matching.

[0056] Step S33: Calculate the Euclidean distance between the sketch local features and the real image features using the optimal matching strategy. Since the deformation of the local structure may cause misalignment, the optimal pooling strategy is used to select the optimal match for each sketch area.

[0057] Dis local =CDist(F residual_enhanced_stetch' ,F photo_set )

[0058] Dis min =min(Dis local ,dim=3)

[0059] CDist(.) represents the Euclidean distance calculation function across the sample set, which outputs the pairing distance between each sketch position and all positions of each candidate photo. local Represents the local distance matrix of all feature points between the sketch and the photo. Dis min represents the shortest matching distance between each sketch position and each photo, F photo_set The original features of the image extracted by S13;

[0060] Step S34: To further enhance the robustness of the matching, the adaptive weights are adjusted in step S32. The optimal matching distance is weighted and the final distance aggregation is performed according to different norms (L1 or L2 norm). The weighted distance matrix is ​​defined as:

[0061]

[0062] Dis final The matching distance between the final sketch and each candidate image is used for ranking and retrieval. This dynamic adjustment mechanism allows the model to adaptively adjust the weights of local features based on the importance of the sketch area, thereby maintaining stable matching between sketches of different scales and styles.

[0063] In this embodiment, step S4 specifically includes the following steps:

[0064] Let the model learn the matching relationship between sketch features and positive samples while keeping the feature distance away from negative samples.

[0065]

[0066] Only when the negative sample distance D(S,P -) is less than the positive sample distance D(S,P + ) plus the margin m, the model will be penalized. - )≥D(S,P + )+m, the loss is zero and the model is not updated. Where S is the final feature representation of the sketch image, P + Positive sample image features for sketch semantic matching, P - Negative sample image features that are not related to the sketch, D(.) represents the distance function between any two samples.

[0067]

[0068] M represents the number of modes, which in this invention are photos and sketches. (i) Sketch features of the i-th style; A cross-modal feature transformation network that maps the i-th style to the j-th style; the target term E (j) Semantic target under the j-th style.

[0069] The total loss function is:

[0070]

[0071] Wherein λ is a weight coefficient, which is 0.5 in the present invention.

[0072] Step S42: Perform iterative training according to the specified parameters, and continuously update the gradient according to the overall loss L calculated in step S41 to perform iterative training;

[0073] Step S43: During the inference process, the fine-grained classification result is obtained by combining the multi-granularity prediction results to fully utilize the complementarity of multi-granularity information. The calculation formula is as follows:

[0074]

[0075] Where sk_label_list[i] is the predicted label of the i-th sample. ph_label_list[indices[i][0]]} represents the list of true labels, and sk_num represents the number of samples.

[0076] Step S44: During the training process, the model is verified at a certain iteration interval according to the verification interval flag, and the optimal model is continuously saved. When the number of iterations reaches the preset maximum iteration threshold, the training process ends and the optimal fine-grained retrieval accuracy of the currently specified data set is returned.

[0077] Specifically, this paper proposes a style-adaptive sketch image retrieval method that relies solely on category annotations, eliminating the need for additional manual annotation information. Addressing the subjectivity and diversity of user drawing styles in sketch image retrieval (SBIR) tasks, this study leverages the variations in local structure, shape scale, and detail deformation between sketches and ground-truth images. Unlike other visual matching tasks, sketches often exhibit significant stylistic diversity. Traditional fine-grained recognition tasks typically rely on detailed features in local regions for matching, but this strategy may not be applicable in sketch retrieval. Most existing methods rely on global feature matching or fixed local region alignment strategies to match sketches with ground-truth images. These fixed feature models struggle to adapt to different user drawing styles. To address these issues, this study dynamically enhances potential key region features and adaptively adjusts the regional weights of different style features to compensate for local scale deviations between sketches of different styles. Furthermore, to further compensate for scale distortion and detail distortion that may occur during sketching, a self-attention mechanism is used to calculate the importance weights of local features within the sketches, and an optimal matching strategy is employed for dynamic region alignment. This enables precise alignment and structural compensation of sketches of different styles during the matching phase. The style-adaptive residual network constructed in this study realizes the dynamic capture and feature alignment of fine-grained features, greatly improving the retrieval performance and robustness of the FG-SBIR task.

[0078] The present invention also provides a sketch image retrieval system based on a style adaptive residual network, comprising a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, it can implement any of the method steps described above.

[0079] The present invention also provides a computer-readable storage medium on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, any of the method steps described above can be implemented.

[0080] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes any of the method steps described above.

[0081] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions and effects do not exceed the scope of the technical solution of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A sketch image retrieval method based on style adaptive residual network, characterized by: A fine-grained sketch dataset is obtained, and feature extraction is performed by constructing and training a style-adaptive residual network to obtain fine-grained retrieval results. In the style-adaptive residual network, a style enhancement gate module is used to enhance features with important semantic information using a channel attention mechanism. The style enhancement module is used to adaptively adjust the style enhancement features through feature relationship matrix and learnable transformation operations.

2. The sketch image retrieval method based on style adaptive residual network according to claim 1, characterized in that: The method comprises the following steps: Step S1: Obtain a fine-grained classification dataset, preprocess the data, expand the dataset using data augmentation technology, and input it into the backbone network to extract mid-level features; Step S2: Dynamically adjust the activation strength of feature channels through the channel attention mechanism to enhance features with important semantic information, so that low-response but critical global or local features can be effectively transmitted; Step S3: Adaptively adjust the style enhancement features through the feature relationship matrix and learnable transformation operations to ensure that the model does not overly rely on local patterns of a specific style, but is able to match based on the overall structural information; Step S4: Perform iterative training according to the specified training parameters, update the model parameters by optimizing the combined loss, continuously save the optimal model according to the verification accuracy, and use the final trained model to obtain the fine-grained retrieval results.

3. The sketch image retrieval method based on style adaptive residual network according to claim 2, characterized in that: Step S1 specifically includes the following steps: Step S11: Using a publicly available fine-grained classification dataset, perform data preprocessing to complete label extraction. Automatically match the photo files with the corresponding sketch files through a regular matching mechanism to prepare for subsequent model training. Step S12: performing data augmentation operations on the collected data set, using random cropping, random horizontal inversion, random vertical flipping, and random rotation data augmentation methods to process the data set to obtain an enhanced data set; Step S13: The mid-level features of the image are extracted through the ResNet-50 model to capture the local structural information in the sketch and photo images to enhance the feature representation ability of the model, and feature normalization is performed to keep the features stable, and the normalized features are returned.

4. The sketch image retrieval method based on style adaptive residual network according to claim 3, characterized in that: The overall process of step S13 is expressed as follows: h spatial Represents the spatial coordinates of the feature map in height, corresponding to the dimension from top to bottom of the image, w spatial Represents the spatial coordinates of the feature map in width, corresponding to the dimension from left to right of the image, F original (h spatial ,w spatial ) represents the spatial position (h spatial ,w spatial ) is extracted by the middle convolutional layer of ResNet-50, F normalized (h spatial ,w spatial ) represents the normalized feature vector at the same position, which is used for subsequent calculations and matching. To prevent the denominator from being zero, a minimum value ∈ is introduced to enhance stability; channel_count represents the total number of channels of the feature map, that is, the dimension of the feature vector at each spatial position; channel_index is the channel index currently traversed, which is used for channel-by-channel processing in the normalized calculation.

5. The sketch image retrieval method based on style adaptive residual network according to claim 4, characterized in that: Step S2 specifically includes the following steps: Step S21: Sketch feature F in S13 original After extraction, channel-level feature enhancement is first performed. The channel dimensions are adjusted through the convolution layer, and the feature distribution is normalized using batch normalization. Subsequently, the channel weights are calculated using the Sigmoid activation function and multiplied element-wise with the original features to achieve dynamic weighting of the feature channels. The weighted features are L2 normalized to maintain amplitude consistency and processed by flattening and dimensionality adjustment. F boosted =σ(BN(Conv attention (F original )))×F original Conv attention The convolution operation used for channel dimension mapping is 1×1 convolution, which is used to generate attention features; σ represents the sigmoid function, which maps the features to the interval [0,1], generates channel-level attention weights, and performs element-wise multiplication to weight the generated channel weights with the original features; Represents the feature map after attention enhancement, which retains the original information while highlighting the key channels; Step S21: Calculate sketch features and enhanced features The guidance vector G between sketch_guidance , the similarity of the input features is calculated through the similarity formula, and the dynamic feature weights are generated through the Softmax transformation; finally, the weight matrix is ​​weighted element by element with the features; G sketch_guidance =Softmax(Sim(F original ,F boosted )) F sketch_refined =G sketch_guidance ⊙F boosted +F boosted Among them, Sim(F original ,F boosted ) represents a sketch feature and enhanced features The similarity between them, ⊙ element-wise multiplication is used to apply the guidance weight to the enhanced features; Represents the attention weight coefficient of each spatial position, emphasizing the area with higher similarity to the enhanced feature; Represents the final fused sketch feature representation, in which important areas are enhanced and background or irrelevant areas are suppressed, as the input of the downstream module; Step S23: Step S22 A linear combination of cross-channel features is achieved through convolutional layers; F channel_mapped_1 =Conv1d stage1 (F sketch_refined ) F channel_mapped_2 =Conv1d stage2 (F channel_mapped_1 +F original ) Conv1d stage1 Represents the first stage 1D convolution operation, performed on the channel dimension, used to linearly integrate cross-channel features, Conv1d stage2 Represents the second stage convolution operation, which further transforms the weighted residual features; is the output feature after the first convolution, F channel_mapped_2 To fuse the final features, nonlinear mapping and residual information are combined as input to subsequent modules; Step S24: Through the residual learning mechanism, the enhanced features are fused with the original features to effectively maintain the integrity and information consistency of the feature space; F residual_enhanced =Conv1d stage1 (Conv1d stage1 (F sketch_refined )+F original )+F sketch_refined The final fusion output features have enhanced semantic expression capabilities and original structural information for subsequent feature matching or discrimination tasks.

6. The sketch image retrieval method based on style adaptive residual network according to claim 5, characterized in that: Step S3 specifically includes the following steps: Step S31: Obtained from S24 The global relationship of local features of the sketch is captured through the self-attention mechanism, and the importance of sketch features is dynamically adjusted; specifically, the sketch features are first It is regarded as a set of (spatial_height×spatial_width)×(spatial_height×spatial_width) local feature points, each of which contains channel information of channel_count dimension. The similarity between feature points is calculated by dot product operation, and the self-attention matrix is ​​obtained by softmax normalization: F residual_enhanced ′=A sketch_attention F residual_enhanced A sketch_attention Reflects the similarity relationship between feature points in different regions of the sketch; through the matrix A sketch_attention Weighted original feature F residual_enhanced , get the enhanced sketch feature Step S32: weighted feature F generated by S31 residual_enhanced ', introduce a feature weight normalization mechanism to ensure that the importance of different regions can be treated consistently during the feature matching process; calculate the weighted sum of each feature point to measure the importance of the feature point, and adjust the feature weight through minimum-maximum normalization: in, Represents the normalized weight of each feature point, ∈ is a very small positive number, with a value of 1×10 -8 , used to prevent the denominator from being zero; the normalized weights can dynamically adjust the importance of the enlarged or reduced areas in the sketch, ensuring that the model will not be biased due to imbalanced area proportions in the subsequent matching process; Step S33: Calculate the Euclidean distance between the sketch local features and the real image features using the optimal matching strategy. Since the deformation of the local structure may cause misalignment, the optimal pooling strategy is used to select the optimal match for each sketch area. Dis local =CDist(F residual_enhanced_sketch' ,F photo_set ) Dis min =min(Dis local ,dim=3) CDist(.) represents the Euclidean distance calculation function across the sample set, which outputs the pairing distance between each sketch position and all positions of each candidate photo; Dis local Represents the local distance matrix of all feature points between the sketch and the photo, Dis min represents the shortest matching distance between each sketch position and each photo, F photo_set The original features of the image extracted by S13; Step S34: Adapt weights through S32 The optimal matching distance is weighted and the final distance aggregation is performed according to different norms, namely L1 norm or L2 norm; the weighted distance matrix is ​​defined as: Dis final Represents the matching distance results between the final sketch and each candidate image, which is used for ranking and retrieval.

7. The sketch image retrieval method based on style adaptive residual network according to claim 6, characterized in that: Step S4 specifically includes the following steps: Step S41: Calculate the overall network loss function Loss, the calculation formula is as follows Let the model learn the matching relationship between sketch features and positive samples, while keeping the feature distance from negative samples; Only when the negative sample distance D(S,P - ) is less than the positive sample distance D(S,P + ) plus the boundary m, the model will be penalized; when D(S,P - )≥D(S,P + )+m, the loss is zero and the model is not updated; where S is the final feature representation of the sketch image, P + Represents the positive sample image features of the sketch semantic matching, P - Negative sample image features that are not related to the sketch, D(.) represents the distance function between any two samples; M represents the number of modalities, including photos and sketches, and E (i) Sketch features of the i-th style; A cross-modal feature transformation network that maps the i-th style to the j-th style; the target term E (j) Semantic target under the j-th style; The total loss function is: Where λ is the weight coefficient; Step S42: Perform iterative training according to the specified parameters, and continuously update the gradient according to the overall loss L calculated in step S41 to perform iterative training; Step S43: During the inference process, the multi-granularity prediction results are combined to fully utilize the complementarity of multi-granularity information to obtain a fine-grained classification result. The calculation formula is as follows: Where sk_label_list[i] is the predicted label of the i-th sample; ph_label_list[indices[i][0]]} represents the list of true labels, and sk_num represents the number of samples; Step S44: During the training process, the model is verified at a predetermined number of iterations according to the verification interval flag, and the optimal model is continuously saved. When the number of iterations reaches the preset maximum iteration threshold, the training process ends and the optimal fine-grained retrieval accuracy of the currently specified data set is returned.

8. A sketch image retrieval system based on style adaptive residual network, characterized by: The method comprises a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps according to any one of claims 1 to 7 can be implemented.

9. A computer-readable storage medium storing computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 7 can be implemented.

10. An electronic device comprising a processor and a memory, wherein: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the method steps according to any one of claims 1 to 7.

Images