A feature construction method based on multi-mode synthesis

Abstract

The application discloses a feature construction method based on multi-mode synthesis, aiming at solving the problems of feature loss, mode matching deviation and up-sampling distortion in the existing multi-mode feature construction technology, and improving the efficiency and quality of feature construction. The method comprises the following steps: performing global average pooling down-sampling on an input image to generate input features; performing segmentation and mode alignment preprocessing on the down-sampled features; constructing a mode warehouse and completing mode matching; introducing a dynamic mode association mechanism to synthesize high-quality features; up-sampling to generate an output image with the same size as the input image; and constructing a training target optimization network. The application improves the feature construction precision and integrity, balances the efficiency and adaptive ability, and can provide reliable support for computer vision downstream tasks.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, digital image processing, and deep learning feature extraction technology. Specifically, it relates to a feature construction technology for image fine feature optimization, multimodal feature fusion, and image resolution reconstruction. In particular, it is a high-precision image feature construction, image feature enhancement, and intelligent image processing optimization method based on multimodal feature splitting, matching, dynamic association, and fusion synthesis under the deep learning network framework. It is applicable to various visual perception, image restoration, and image feature extraction related technology scenarios. Background Technology

[0002] With the rapid development of computer vision and digital image processing technologies, image feature construction, as a core component of deep learning networks, directly determines the accuracy and efficiency of downstream tasks such as image restoration, object recognition, and semantic segmentation. It has wide-ranging applications in fields such as autonomous driving, medical image analysis, and remote sensing image processing. As application scenarios increasingly demand higher image processing accuracy, traditional single-mode feature extraction and construction methods are no longer sufficient to meet the requirements for feature refinement, completeness, and adaptability in practical applications. Multi-mode feature fusion and synthesis are gradually becoming key technologies for improving the quality of feature construction.

[0003] Existing multi-modal feature construction methods generally suffer from several shortcomings: Firstly, most methods employ only simple pooling or convolutional downsampling when downsampling images, easily leading to the loss of detailed image features and affecting the accuracy of subsequent pattern matching and feature synthesis. Secondly, existing pattern matching mechanisms lack dynamic correlation capabilities, often relying on fixed matching rules and failing to adaptively adjust based on the differences between downsampled features and the original patterns, resulting in matching biases and insufficient precision in synthesized features. Furthermore, existing technologies do not fully consider the correlation and complementarity of different pattern features, making it difficult to construct high-quality features with both completeness and precision. Feature distortion and inaccurate resolution restoration are also prone to occur during feature upsampling, ultimately affecting the quality of the output image and the performance of downstream tasks. Simultaneously, some methods lack a robust pattern repository management and retrieval mechanism, resulting in low efficiency in pattern information reading and matching, further restricting the overall efficiency of feature construction.

[0004] To address the shortcomings of existing technologies, this invention aims to overcome problems such as feature loss, inaccurate pattern matching, poor feature synthesis quality, upsampling distortion, and low efficiency during multi-modal feature construction. By optimizing downsampling processing methods, designing a scientific original pattern alignment process, and constructing an efficient pattern warehouse and dynamic pattern association mechanism, it achieves accurate matching and efficient synthesis of multi-modal features. At the same time, it optimizes upsampling strategies to ensure that the output image and input image are of the same size and have complete details, thereby improving the accuracy and efficiency of feature construction and providing high-quality feature support for downstream image processing tasks. Summary of the Invention

[0005] The purpose of this invention is to overcome the defects of existing multi-mode feature construction technology and provide a feature construction method based on multi-mode synthesis. Through systematic process design and technology optimization, it achieves high-precision and high-efficiency image feature construction, solves problems such as feature loss, pattern matching deviation, insufficient synthesis quality and upsampling distortion in existing technologies, and provides reliable feature support for downstream computer vision tasks.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] 1. A feature construction method based on multi-mode synthesis, characterized in that the method includes the following steps:

[0008] S1. The input image is downsampled and processed using global average pooling. The generated features are used as input information for the multi-mode synthesis module.

[0009] S2. Process the downsampling information of the image, including feature segmentation, original pattern alignment, and original pattern feature alignment, to complete the preliminary processing of the downsampling information;

[0010] S3. Build a pattern repository to read pattern information and perform pattern matching. Read the pattern information from the pattern repository and perform pattern matching with the aligned original pattern.

[0011] S4. Multi-mode synthesis of high-quality feature information, including dynamic mode association mechanism, and synthesis of dynamic mode features and original mode features to construct high-quality feature information;

[0012] S5. Upsample high-quality feature information: Upsample the high-quality feature information to generate an output image with the same size as the input image.

[0013] S6. Construct training objectives and train the network.

[0014] 2. The feature construction method based on multi-mode synthesis according to claim 1, characterized in that the specific process of S1 is as follows:

[0015] This step first inputs an input image InputImage with dimensions C×H×W. The input image is then downsampled using a global average pooling (GAP) operation. The input image InputImage is then reduced to the size of GAP_Image1 (C1×H / 2×W / 2), GAP_Image2 (C2×H / 4×W / 4), and GAP_Image3 (C3×H / 8×W / 8) by three consecutive global average pooling (GAP) operations.

[0016]

[0017] The generated feature GAP_Image3 is also denoted as query. The query records the comprehensive features of the input image InputImage, which will be used as input information for the multi-mode synthesis module in the next stage.

[0018] 3. The feature construction method based on multi-mode synthesis according to claim 1, wherein the specific process of S2 is as follows:

[0019] This step uses the input image feature information query generated in the previous stage as input and processes the query in three branches. The first is feature segmentation. The query of size C3×H / 8×W / 8 is divided into T segments according to dimension C2, denoted as Q. Where T=H / 8×W / 8.

[0020]

[0021] The second step is original pattern alignment. The original pattern, denoted as OP, records the original pattern information represented by the input information compression feature query. This original pattern is directly derived from the query through a multilayer perceptron (MLP1). The original pattern OP is used for pattern matching with patterns generated subsequently from the memory repository.

[0022]

[0023] Next is the alignment of original pattern features, denoted as OPF. OPF records the deep feature information of the input information compressed feature query. The original pattern features are transformed by the Multilayer Perceptron (MLP2). The original pattern features OPF are used for aggregation with the dynamic pattern features synthesized by the multi-pattern synthesis module, thereby avoiding the lack of pattern information in the original features in the dynamic pattern features.

[0024]

[0025] After the above operations, the input information compressed feature query generated by downsampling the input image has completed the preliminary preprocessing, which is helpful for subsequent pattern matching and feature fusion.

[0026] 4. The feature construction method based on multi-mode synthesis according to claim 1, wherein the specific process of step S3 is as follows:

[0027] This step constructs K schema repositories (MRs).

[0028]

[0029] For ease of representation, let MRk represent any one of the K pattern repositories MR. Let EM represent the set of embedding vectors of the K pattern repositories.

[0030]

[0031] Each pattern repository MRk stores S embedding vectors k_EM with dimension C2.

[0032]

[0033] The input information, compressed features, query, are segmented into T segments Q. Using... Let Q represent any one of the dividing bars. Using Read the schema information from the schema repository constructed above.

[0034] First, cosine similarity is calculated between the segment Q and the embedding vector k_EM in each pattern repository MRk to obtain a relevance score map MAP of size S×T. Then, a softmax function is applied along the vertical direction of the relevance score map MAP to transform it into a probability distribution k_VPROP of the scores. Let represent any embedding vector stored in MRk.

[0035]

[0036] k_VPROP_t,s is used to represent each embedding vector k_EMs in each pattern repository MRk and each segmentation bar of the input information compressed feature query. The corresponding probability. The weight of each embedding vector k_EMs for each pattern repository MRk is obtained by weighted summing of each embedding vector k_EMs in each pattern repository MRk with its corresponding weight, thus obtaining each segmentation bar. The corresponding pattern read result in the pattern repository MRk .

[0037]

[0038] In each schema repository MRk, each split bar The corresponding result read is T segment bars are read T times to obtain T corresponding results. Finally, these results are aggregated along the channel direction to obtain the corresponding pattern information Pk. K pattern repositories read K different pattern information, and the resulting pattern information set is denoted as P.

[0039]

[0040] After each segmentation bar is read, each embedding vector k_EMs in each pattern repository MRk is updated based on the pattern information represented by the read. A relevance score map MAP of size S×T is calculated in the above stage. In the update stage, the score map MAP is transformed into a score probability distribution k_HPROP along the horizontal direction using a softmax function.

[0041]

[0042] Each segment used to represent the input information compression feature query The probability corresponding to each embedding vector k_EMs in each pattern repository MRk.

[0043] Let k_As represent the set of score probabilities for each embedding vector k_EMs and all segmentation bars. The highest score is denoted as . The lowest score is recorded as Each probability is normalized based on the highest and lowest scores.

[0044]

[0045] yes The probability distribution obtained after normalization is used to update each embedding vector k_EMs in each pattern repository MRk.

[0046]

[0047] Through the above steps, the input information is compressed using the feature query to read K pattern repositories (MR), resulting in K different pattern information P. The original pattern OP generated by Formula 5 is then matched with different patterns P to obtain the corresponding pattern matching scores (SCO).

[0048]

[0049] This represents the pattern matching results obtained by calculating the dot product between the original pattern OP and each pattern Pk. Further, for each pattern matching result... The softmax function is applied to transform the data into a probability distribution Wk, which is also used as the weight for each pattern Pk in subsequent information fusion stages.

[0050] 5. The feature construction method based on multi-mode synthesis according to claim 1, wherein the specific process of S4 is as follows:

[0051] This step involves associating the original pattern OP with the pattern information Pk obtained from each pattern repository MRk by reading the input information compressed feature query, and generating dynamic pattern feature DPFk.

[0052] When k=1, the dynamic pattern features are calculated as follows:

[0053]

[0054] When k>1, the dynamic pattern features are calculated as follows:

[0055]

[0056] Through correlation calculation, a total of K dynamic pattern feature DPFs are generated.

[0057]

[0058] The dynamic pattern feature DPF and the original pattern feature OPF generated by Formula 6 are fused along the channel direction to obtain the fused feature F.

[0059]

[0060] In the above steps, the size of each DPFk and the size of the OPF are both C3×H / 8×W / 8, the size of the dynamic mode feature DPF is K·C3×H / 8×W / 8, and the size of the fused feature F obtained after fusion along the channel direction is (K+1)C3×H / 8×W / 8.

[0061] The fused feature F is processed through a global average pooling (GAP) operation to generate a high-quality multimodal synthetic feature vector mpsv.

[0062]

[0063] The multi-mode synthetic feature vector mpsv has a dimension of C4. This vector integrates information from multiple dynamic modes and the original mode. Multiple hybrid mode features can effectively optimize feature quality.

[0064] The fused feature F is processed through a global average pooling (GAP) operation to generate a high-quality multimodal synthetic feature vector mpsv.

[0065] 6. The feature construction method based on multi-mode synthesis according to claim 1, wherein the specific process of step S5 is as follows:

[0066] In the previous stage, a high-quality multi-mode synthesized feature vector mpsv was generated. In this stage, the multi-mode synthesized feature vector mpsv will be upsampled using the transposed convolution operation ConvTrans. The multi-mode synthesized feature vector mpsv is finally upsampled into the output image OutputImage after four consecutive transposed convolution operations.

[0067]

[0068] The multi-mode synthesized feature vector mpsv is sequentially upsampled into ConvTrans_mpsv1 of size C3×H / 8×W / 8, ConvTrans_mpsv2 of size C2×H / 4×W / 4, ConvTrans_mpsv3 of size C1×H / 2×W / 2, and OutputImage of size C×H×W.

[0069] 7. The feature construction method based on multi-mode synthesis according to claim 1, wherein the specific process of step S6 is as follows:

[0070] In this step, a training objective is constructed to enable learnable training for the image generation task. The image generation task aims to generate images similar to or in the same style as the input image; therefore, a pixel-level loss (Loss) is introduced as the training objective for the image generation task.

[0071]

[0072] The loss is optimized using the L2 distance between the input image and the output image. The L2 distance can blur local parts of the image, thereby increasing the generalization ability of image generation.

[0073] Compared with existing technologies, the beneficial effects of this invention are:

[0074] 1. Effectively solves the problems of feature loss, pattern matching deviation and upsampling distortion in existing technologies. By using global average pooling downsampling, dual original pattern alignment and optimized upsampling strategy, combined with dynamic pattern association mechanism, it significantly improves the accuracy and completeness of feature construction, ensuring that the output image is the same size as the input image and that no details are lost, providing high-quality feature support for downstream image processing tasks.

[0075] 2. Construct an efficient pattern repository and a scientific feature synthesis process to achieve rapid reading and accurate matching of pattern information. At the same time, by dynamically associating and fusing different pattern features, it balances the efficiency and quality of feature construction. Compared with existing fixed matching and simple fusion methods, it significantly improves the adaptive capability and overall efficiency of feature construction, and is suitable for more complex image processing scenarios.

[0076] Figure 1 This is a schematic diagram of a feature construction method based on multi-mode synthesis.

[0077] Figure 2 This is a visualization of the input image reconstructed from a multi-mode synthesized feature vector based on a multi-mode synthesis feature construction method. Detailed Implementation

[0078] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0079] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0080] Figure 1 This is a schematic diagram of a feature construction method based on multi-mode synthesis. (Example) Figure 1 As shown, firstly, an input image InputImage with size C×H×W is input. The input image is downsampled by global average pooling (GAP) operation. The input image InputImage is reduced to the size of GAP_Image1 (C1×H / 2×W / 2), GAP_Image2 (C2×H / 4×W / 4), and GAP_Image3 (C3×H / 8×W / 8) by three consecutive global average pooling (GAP) operations.

[0081]

[0082] The generated feature GAP_Image3 is also denoted as query. The query records the comprehensive features of the input image InputImage, which will be used as input information for the multi-mode synthesis module in the next stage.

[0083] After completion Figure 1 After downsampling the input image, the input image feature information query is obtained. Using this query as input, the query is processed in three branches. First, feature segmentation: the query of size C3×H / 8×W / 8 is segmented into T segments along dimension C2, denoted as Q. Where T = H / 8×W / 8.

[0084]

[0085] The second step is original pattern alignment. The original pattern, denoted as OP, records the original pattern information represented by the input information compression feature query. This original pattern is directly derived from the query through a multilayer perceptron (MLP1). The original pattern OP is used for pattern matching with patterns generated subsequently from the memory repository.

[0086]

[0087] Next is the alignment of original pattern features, denoted as OPF. OPF records the deep feature information of the input information compressed feature query. The original pattern features are transformed by the Multilayer Perceptron (MLP2). The original pattern features OPF are used for aggregation with the dynamic pattern features synthesized by the multi-pattern synthesis module, thereby avoiding the lack of pattern information in the original features in the dynamic pattern features.

[0088]

[0089] After the above operations, the input information compressed feature query generated by downsampling the input image has completed the preliminary preprocessing, which is helpful for subsequent pattern matching and feature fusion.

[0090] like Figure 1 As shown, after preprocessing the query, K schema repositories (MRs) are constructed.

[0091]

[0092] For ease of representation, let MRk represent any one of the K pattern repositories MR. Let EM represent the set of embedding vectors of the K pattern repositories.

[0093]

[0094] Each pattern repository MRk stores S embedding vectors k_EM with dimension C2.

[0095]

[0096] The input information, compressed features, query, are segmented into T segments Q. Using... Let Q represent any one of the dividing bars. Using Read the schema information from the schema repository constructed above.

[0097] First, cosine similarity is calculated between the segment Q and the embedding vector k_EM in each pattern repository MRk to obtain a relevance score map MAP of size S×T. Then, a softmax function is applied along the vertical direction of the relevance score map MAP to transform it into a probability distribution k_VPROP of the scores. Let represent any embedding vector stored in MRk.

[0098]

[0099] k_VPROP_t,s is used to represent each embedding vector k_EMs in each pattern repository MRk and each segmentation bar of the input information compressed feature query. The corresponding probability. The weight of each embedding vector k_EMs for each pattern repository MRk is obtained by weighted summing of each embedding vector k_EMs in each pattern repository MRk with its corresponding weight, thus obtaining each segmentation bar. The corresponding pattern read result in the pattern repository MRk .

[0100]

[0101] In each schema repository MRk, each split bar The corresponding result read is T segment bars are read T times to obtain T corresponding results. Finally, these results are aggregated along the channel direction to obtain the corresponding pattern information Pk. K pattern repositories read K different pattern information, and the resulting pattern information set is denoted as P.

[0102]

[0103] After each segmentation bar is read, each embedding vector k_EMs in each pattern repository MRk is updated based on the pattern information represented by the read. A relevance score map MAP of size S×T is calculated in the above stage. In the update stage, the score map MAP is transformed into a score probability distribution k_HPROP along the horizontal direction using a softmax function.

[0104]

[0105] Each segment used to represent the input information compression feature query The probability corresponding to each embedding vector k_EMs in each pattern repository MRk.

[0106] Let k_As represent the set of score probabilities for each embedding vector k_EMs and all segmentation bars. The highest score is denoted as . The lowest score is recorded as Each probability is normalized based on the highest and lowest scores.

[0107]

[0108] yes The probability distribution obtained after normalization is used to update each embedding vector k_EMs in each pattern repository MRk.

[0109]

[0110] Through the above steps, the input information is compressed using the feature query to read K pattern repositories (MR), resulting in K different pattern information P. The original pattern OP generated by Formula 5 is then matched with different patterns P to obtain the corresponding pattern matching scores (SCO).

[0111]

[0112] This represents the pattern matching results obtained by calculating the dot product between the original pattern OP and each pattern Pk. Further, for each pattern matching result... The softmax function is applied to transform the data into a probability distribution Wk, which is also used as the weight for each pattern Pk in subsequent information fusion stages.

[0113] like Figure 1 As shown in the dynamic pattern association diagram, the original pattern OP and the pattern information Pk obtained by reading each pattern warehouse MRk from the input information compressed feature query are associated and calculated to generate the dynamic pattern feature DPFk.

[0114] When k=1, the dynamic pattern features are calculated as follows:

[0115]

[0116] When k>1, the dynamic pattern features are calculated as follows:

[0117]

[0118] Through correlation calculation, a total of K dynamic pattern feature DPFs are generated.

[0119]

[0120] The dynamic pattern feature DPF and the original pattern feature OPF generated by Formula 6 are fused along the channel direction to obtain the fused feature F.

[0121]

[0122] In the above steps, the size of each DPFk and the size of the OPF are both C3×H / 8×W / 8, the size of the dynamic mode feature DPF is K·C3×H / 8×W / 8, and the size of the fused feature F obtained after fusion along the channel direction is (K+1)C3×H / 8×W / 8.

[0123] The fused feature F is processed through a global average pooling (GAP) operation to generate a high-quality multimodal synthetic feature vector mpsv.

[0124]

[0125] The multi-mode synthetic feature vector mpsv has a dimension of C4. This vector integrates information from multiple dynamic modes and the original mode. Multiple hybrid mode features can effectively optimize feature quality.

[0126] The fused feature F is processed through a global average pooling (GAP) operation to generate a high-quality multimodal synthetic feature vector mpsv.

[0127] like Figure 1 As shown, a high-quality multi-mode synthesized feature vector mpsv was generated in the previous stage. In this stage, the multi-mode synthesized feature vector mpsv will be upsampled using the transposed convolution operation ConvTrans. The multi-mode synthesized feature vector mpsv is finally upsampled into the output image OutputImage after four consecutive transposed convolution operations.

[0128]

[0129] The multi-mode synthesized feature vector mpsv is sequentially upsampled into ConvTrans_mpsv1 of size C3×H / 8×W / 8, ConvTrans_mpsv2 of size C2×H / 4×W / 4, ConvTrans_mpsv3 of size C1×H / 2×W / 2, and OutputImage of size C×H×W.

[0130] Finally, the training objective of the model is determined. In this step, a training objective is constructed to enable learnable training for the image generation task. The image generation task aims to generate images similar to or in the same style as the input image; therefore, a pixel-level loss (Loss) is introduced as the training objective for the image generation task.

[0131]

[0132] The loss is optimized using the L2 distance between the input image and the output image. The L2 distance can blur local parts of the image, thereby increasing the generalization ability of image generation.

[0133] Figure 2This is a visualization of the input image reconstructed using a multi-modal synthesis feature vector reconstruction method. The network was trained on the ImageNet and MS-COCO datasets, demonstrating that this multi-modal synthesis feature reconstruction method accurately generates image location, shape, and even finer texture information, while maintaining a high degree of clarity in the output image compared to the original input image.

[0134] This invention proposes a feature construction method based on multi-modal synthesis, aiming to address the core shortcomings of existing multi-modal feature construction techniques, such as feature loss, pattern matching deviation, and upsampling distortion. It also improves the efficiency, quality, and adaptability of feature construction, providing reliable support for various downstream image processing tasks in computer vision. This method preserves both global and detailed image features by employing global average pooling downsampling, combined with dual original pattern alignment operations and optimized upsampling strategies, along with a dynamic pattern association mechanism. This significantly improves the accuracy and completeness of feature construction, ensuring that the final output image is the same size as the input image and that no details are lost. Furthermore, by constructing an efficient pattern repository and a scientific feature synthesis process, it achieves rapid reading and accurate matching of pattern information. Through dynamic association and fusion of different pattern features, it effectively overcomes the limitations of fixed matching and simple fusion in existing technologies, balancing the efficiency and quality of feature construction and significantly improving its adaptability, enabling it to adapt to more complex image processing scenarios.

[0135] Finally, the details of the above examples of the present invention are merely illustrative of the invention. Any modifications, improvements, and substitutions to the above embodiments by those skilled in the art should be included within the scope of protection of the claims of the present invention.

Claims

1. A feature construction method based on multi-modal synthesis, characterized in that, The method includes the following steps: S1. The input image is downsampled and processed using global average pooling. The generated features are used as input information for the multi-mode synthesis module. S2. Process the downsampling information of the image, including feature segmentation, original pattern alignment, and original pattern feature alignment, to complete the preliminary processing of the downsampling information; S3. Build a pattern repository to read pattern information and perform pattern matching. Read the pattern information from the pattern repository and perform pattern matching with the aligned original pattern. S4. Multi-mode synthesis of high-quality feature information, including dynamic mode association mechanism, and synthesis of dynamic mode features and original mode features to construct high-quality feature information; S5. Upsample high-quality feature information: Upsample the high-quality feature information to generate an output image with the same size as the input image. S6. Construct training objectives and train the network.

2. The feature construction method based on multi-mode synthesis according to claim 1, characterized in that, The specific process of S1 is as follows: This step first inputs an input image InputImage with dimensions C×H×W. The input image is then downsampled using a global average pooling (GAP) operation. The input image InputImage is then reduced to the size of GAP_Image1 (C1×H / 2×W / 2), GAP_Image2 (C2×H / 4×W / 4), and GAP_Image3 (C3×H / 8×W / 8) by three consecutive global average pooling (GAP) operations. The generated feature GAP_Image3 is also denoted as query. The query records the comprehensive features of the input image InputImage, which will be used as input information for the multi-mode synthesis module in the next stage.

3. The method of claim 1, wherein the feature construction is based on a multi-modal synthesis. The specific process of S2 is as follows: This step uses the input image feature information query generated in the previous stage as input and processes the query in three branches. The first is feature segmentation. The query of size C3×H / 8×W / 8 is divided into T segments according to dimension C2, denoted as Q. Where T=H / 8×W / 8. The second step is original pattern alignment. The original pattern, denoted as OP, records the original pattern information represented by the input information compression feature query. This original pattern is directly derived from the query through a multilayer perceptron (MLP1). The original pattern OP is used for pattern matching with patterns generated subsequently from the memory repository. Next is the alignment of original pattern features, denoted as OPF. OPF records the deep feature information of the input information compressed feature query. The original pattern features are transformed by the Multilayer Perceptron (MLP2). The original pattern features OPF are used for aggregation with the dynamic pattern features synthesized by the multi-pattern synthesis module, thereby avoiding the lack of pattern information in the original features in the dynamic pattern features. After the above operations, the input information compressed feature query generated by downsampling the input image has completed the preliminary preprocessing, which is helpful for subsequent pattern matching and feature fusion.

4. The method of claim 1, wherein the feature construction is based on multi-modal synthesis. The specific process of S3 is as follows: This step constructs K schema repositories (MRs). For ease of representation, let MRk represent any one of the K pattern repositories MR. Let EM represent the set of embedding vectors of the K pattern repositories. Each pattern repository MRk stores S embedding vectors k_EM with dimension C2. The input information compression feature query is divided into T segments Q. Any one of the segments Q is represented by The segment Q is divided into T segments Q. Any one of the segments Q is represented by Pattern information reading is performed on the pattern warehouse constructed above. First, cosine similarity is calculated between the segment Q and the embedding vector k_EM in each pattern repository MRk to obtain a relevance score map MAP of size S×T. Then, a softmax function is applied along the vertical direction of the relevance score map MAP to transform it into a probability distribution k_VPROP of the scores. Let represent any embedding vector stored in MRk. k_VPROP_t,s is used to represent each embedding vector k_EMs in each pattern repository MRk and each segmentation bar of the input information compressed feature query. The corresponding probability. The weight of each embedding vector k_EMs for each pattern repository MRk is obtained by weighted summing of each embedding vector k_EMs in each pattern repository MRk with its corresponding weight, thus obtaining each segmentation bar. The corresponding pattern read result in the pattern repository MRk . In each schema repository MRk, each split bar The corresponding result read is T segment bars are read T times to obtain T corresponding results. Finally, these results are aggregated along the channel direction to obtain the corresponding pattern information Pk. K pattern repositories read K different pattern information, and the resulting pattern information set is denoted as P. After each segmentation bar is read, each embedding vector k_EMs in each pattern repository MRk is updated based on the pattern information represented by the read. A relevance score map MAP of size S×T is calculated in the above stage. In the update stage, the score map MAP is transformed into a score probability distribution k_HPROP along the horizontal direction using a softmax function. Each segment used to represent the input information compression feature query The probability corresponding to each embedding vector k_EMs in each pattern repository MRk. Let k_As represent the set of score probabilities for each embedding vector k_EMs and all segmentation bars. The highest score is denoted as . The lowest score is recorded as Each probability is normalized based on the highest and lowest scores. yes The probability distribution obtained after normalization is used to update each embedding vector k_EMs in each pattern repository MRk. Through the above steps, the input information is compressed using the feature query to read K pattern repositories (MR), resulting in K different pattern information P. The original pattern OP generated by Formula 5 is then matched with different patterns P to obtain the corresponding pattern matching scores (SCO). This represents the pattern matching results obtained by calculating the dot product between the original pattern OP and each pattern Pk. Further, for each pattern matching result... The softmax function is applied to transform the data into a probability distribution Wk, which is also used as the weight for each pattern Pk in subsequent information fusion stages.

5. The feature construction method based on multi-mode synthesis according to claim 1, characterized in that, The specific process of S4 is as follows: This step involves associating the original pattern OP with the pattern information Pk obtained from each pattern repository MRk by reading the input information compressed feature query, and generating dynamic pattern feature DPFk. When k=1, the dynamic pattern features are calculated as follows: When k>1, the dynamic pattern features are calculated as follows: Through correlation calculation, a total of K dynamic pattern feature DPFs are generated. The dynamic pattern feature DPF and the original pattern feature OPF generated by Formula 6 are fused along the channel direction to obtain the fused feature F. In the above steps, the size of each DPFk and the size of the OPF are both C3×H / 8×W / 8, the size of the dynamic mode feature DPF is K·C3×H / 8×W / 8, and the size of the fused feature F obtained after fusion along the channel direction is (K+1)C3×H / 8×W / 8. The fused feature F is processed through a global average pooling (GAP) operation to generate a high-quality multimodal synthetic feature vector mpsv. The multi-mode synthetic feature vector mpsv has a dimension of C4. This vector integrates information from multiple dynamic modes and the original mode. Multiple hybrid mode features can effectively optimize feature quality. The fused feature F is processed through a global average pooling (GAP) operation to generate a high-quality multimodal synthetic feature vector mpsv.

6. The feature construction method based on multi-mode synthesis according to claim 1, characterized in that, The specific process of S5 is as follows: In the previous stage, a high-quality multi-mode synthesized feature vector mpsv was generated. In this stage, the multi-mode synthesized feature vector mpsv will be upsampled using the transposed convolution operation ConvTrans. The multi-mode synthesized feature vector mpsv is finally upsampled into the output image OutputImage after four consecutive transposed convolution operations. The multi-mode synthesized feature vector mpsv is sequentially upsampled into ConvTrans_mpsv1 of size C3×H / 8×W / 8, ConvTrans_mpsv2 of size C2×H / 4×W / 4, ConvTrans_mpsv3 of size C1×H / 2×W / 2, and OutputImage of size C×H×W.

7. The feature construction method based on multi-mode synthesis according to claim 1, characterized in that, The specific process of S6 is as follows: In this step, a training objective is constructed to enable learnable training for the image generation task. The image generation task aims to generate images similar to or in the same style as the input image; therefore, a pixel-level loss (Loss) is introduced as the training objective for the image generation task. The loss is optimized using the L2 distance between the input image and the output image. The L2 distance can blur local parts of the image, thereby increasing the generalization ability of image generation.

Images