Gynecological cervical image enhancement processing method

Abstract

The invention provides a method for enhancing a gynecological cervical image, and belongs to the field of image enhancement. The method comprises a plurality of key modules: a cervical image acquisition module acquires images through a medical color camera and an infrared camera, and aligns lesion areas in paired cervical images through image registration; the image preprocessing module performs feature coding on the image to ensure that feature extraction is effective; the double-layer image preprocessing module further extracts multi-level features and captures details and deep structures; the double-layer image fusion module deeply fuses the features of the color image and the infrared image to generate a final enhanced feature; the feature weight is adjusted through the first-level loss function module, and the consistency of image details and vision is optimized. Through cooperative work of the modules, the quality of the cervical image is improved, the accuracy and definition of the image in a lesion area are enhanced, and the method is suitable for early screening and diagnosis of cervical diseases.

Description

Technical Field

[0001] The invention belongs to the field of image enhancement, and in particular relates to a gynecological cervical image enhancement processing method. Background Art

[0002] Gynecological cervical lesion detection is an important task in the field of gynecological medicine and is widely used in the screening and diagnosis of cervical cancer and its lesions. Conventional cervical image acquisition includes color imaging and infrared imaging. These two modal images can provide different information about cervical tissue. Among them, color images can clearly show the color characteristics of the lesion area, while infrared images can penetrate the surface tissue and provide deep structural information. However, due to the incompleteness of single-modality image information and the inevitable noise, low contrast and other problems in the acquisition process, direct reliance on a single-modality image may lead to a decrease in diagnostic accuracy.

[0003] With the rapid development of deep learning technology, multimodal image processing has become an important means to solve the above problems; by utilizing the complementarity of multimodal features, deep fusion of color images and infrared images can significantly improve the detail clarity and contrast of the lesion area, providing more reliable imaging support for disease screening and diagnosis; however, existing multimodal fusion methods often lack optimization strategies for cervical lesion images, and it is difficult to take into account both feature extraction accuracy and enhancement effect at the same time; to address this problem, developing an efficient, accurate and robust cervical image enhancement processing method has important research value and application prospects. Summary of the invention

[0004] The present invention provides a gynecological cervical image enhancement processing method, which aims to generate high-quality cervical images through feature extraction, fusion and optimization of color images and infrared images, improve the detail expression of the lesion area, and provide support for the early screening and diagnosis of gynecological diseases.

[0005] The present invention aims to propose a gynecological cervical image enhancement processing model and provide a gynecological cervical image enhancement processing method, which includes the following steps.

[0006] S1. Gynecological cervical image acquisition: Gynecological cervical sample images are acquired using a medical color camera and an infrared camera, and the lesion areas in the paired cervical images are aligned.

[0007] S2. Construct a gynecological cervical image preprocessing module to encode the features of the cervical color image and the cervical infrared image respectively.

[0008] S3. Construct a double-layer image preprocessing module and use the double-layer image preprocessing module to extract features of gynecological cervical color images and gynecological cervical infrared images.

[0009] S4. Construct a double-layer image fusion module, and use the double-layer image fusion module to fuse the features of the gynecological cervical color image and the gynecological cervical infrared image to form the final feature.

[0010] S5. Construct a first-level loss function and adjust the weights to make the enhanced image have more details and visual consistency.

[0011] S6. Gynecological cervical image processing, obtaining a gynecological cervical image that needs to be enhanced, inputting it into a gynecological cervical image enhancement model, and outputting a high-resolution gynecological cervical image.

[0012] Preferably, in step S2, for the gynecological cervical image preprocessing module, the acquired cervical color image and infrared image are paired to ensure that the two images are aligned, and the lesion areas in the gynecological cervical color image and the gynecological cervical infrared image are aligned by registering the images.

[0013] Preferably, in step S2, for the gynecological cervical image preprocessing module, by registering the images, the consistency of the contents of the color image and the infrared image in spatial position is ensured, so that the same lesion area can be accurately corresponded in the two modalities, reducing the deviation in subsequent feature fusion.

[0014] Preferably, in step S3, for the double-layer image preprocessing module, the cervical color image is input Size , where H and W represent the height and width of the image, 3 represents the three channels of the cervical color image, and the cervical color image Divide into image blocks with a length of p and a width of p. Each image block is flattened into a one-dimensional data vector. All data vectors form a data matrix. ,in represents the number of image blocks, and C is the feature dimension. The feature matrix is ​​input into the Transformer encoder to obtain the enhanced color image feature data. , where L represents the dimension of the projected data; at the same time, the input cervical infrared image Size , where H and W represent the height and width of the image, 1 represents the single channel of the cervical infrared image, and Divide into image blocks with a length of p and a width of p. Each image block is flattened into a one-dimensional data vector. All data vectors form a data matrix. ,in Represents the number of image blocks, and C is the feature dimension. The feature matrix is ​​input into the Transformer encoder to obtain the enhanced infrared image feature data. , where L represents the dimension of the data after projection.

[0015] Preferably, in step S3, for the two-layer image preprocessing module, by dividing the color image and the infrared image into small blocks of fixed size, the computational complexity is reduced and the local information is captured conveniently; the operation of flattening each image block into a one-dimensional vector facilitates the subsequent linear projection, ensuring that the image block data can be uniformly entered into the model processing flow; the flattened vectors of all image blocks are combined into a feature matrix for comprehensively describing the local features of the entire image; the feature matrix is ​​mapped to a unified dimension through linear projection to achieve compression and standardized representation of the original features.

[0016] Preferably, in step S4, for the double-layer image fusion module, Figure 2 As shown, in the first branch, the color image feature data of the cervical color image is used As a query , using infrared image feature data of cervical infrared images As key and value, generate and In the second branch, the infrared image feature data of the cervical infrared image is used As a query , using the color image feature data of the cervical color image As key and value, generate and ,in , d is the dimension of feature alignment between image data; then the first branch cross-image attention weight calculation is performed ,in Indicates the degree of attention of the first branch, is the normalization function, and then the second branch cross-image attention weight calculation is performed ,in Indicates the attention level of the second branch; using a multi-head attention mechanism, each head independently calculates the attention output , , where n represents each computing head, and then the spatial extreme value selection layer is used to integrate the attention levels of the two branches to obtain the final cross-image feature ,in represents the spatial extreme value selection process, Represents dimension expansion concatenation.

[0017] Preferably, in step S4, for the two-layer image fusion module, a cross-modal attention mechanism is used to establish a connection between the two modalities, find the complementary relationship between the color image features and the infrared image features, enhance the information flow between the modalities, and provide a more accurate association for the fusion features; the dot product attention weight is calculated to highlight the relationship between the specific area in the color image and the related area in the infrared image, and the influence of non-important areas is reduced through normalization operations, so that the attention mechanism focuses on key areas; the dot product attention weight is used for weighted summation to ensure that the first branch features after fusion contain both the query characteristics of the color image and the deep information of the infrared image, improve the detail expression ability of the color image in the lesion area, and introduce the penetration characteristics of the infrared modality; the features of the two branches are spliced ​​in the channel direction to form the final fusion features, realize cross-modal deep fusion, and improve the comprehensive effect of image enhancement.

[0018] Preferably, in step S5, for the first-level loss function, a first-level loss function is constructed ,in and They are the single image loss of color images and the single image loss of infrared images, is the cross-image loss function, Controls the weight of each loss.

[0019] Preferably, in step S5, for the first-level loss function, ensure that the features of the color image and the infrared image remain complementary and consistent during the fusion process, enhance the overall clarity of the final image, and make it contain both surface texture and deep tissue information; by adjusting the weights, achieve balanced optimization of the two modalities, ensure that both the clarity of the color image is maintained and the deep information of the infrared image is fully utilized, and provide a comprehensive optimization goal for the final image of cross-modal fusion; improve the network's performance in multimodal feature extraction, fusion and enhancement tasks, and improve the visual consistency of the final fused image and the expressiveness of the lesion area.

[0020] Compared with the prior art, the present invention has the following technical effects: The present invention combines the characteristics of color images and infrared images, and effectively retains the surface texture characteristics and deep tissue information of the cervical lesion area through the feature extraction module, and captures the key details under multi-modality, providing high-quality feature input for subsequent fusion, and enhancing the diagnostic information content of the image; by designing a feature fusion module based on the cross-modal attention mechanism, the present invention can effectively integrate the information of color images and infrared images, and improve the detail expression of the image and the visualization effect of the lesion area on a global scale; the present invention improves the enhancement quality of gynecological cervical images through the collaborative work of modules, and the generated images have the characteristics of high resolution, clear details and prominent lesion areas, which have significant clinical value in the early screening and accurate diagnosis of gynecological diseases, and can improve the accuracy and efficiency of lesion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of a method for gynecological cervical image enhancement processing provided by the present invention.

[0022] Figure 2 It is a structural diagram of a double-layer image preprocessing module provided by the present invention.

[0023] Figure 3 It is a structural diagram of the double-layer image fusion module provided by the present invention.

[0024] Figure 4 It is a low-resolution gynecological cervix image provided by the present invention.

[0025] Figure 5 This is an effect diagram of the low-resolution gynecological cervical image provided by the present invention after high-resolution processing. DETAILED DESCRIPTION

[0026] The present invention aims to propose a method for gynecological cervical image enhancement processing, and realizes efficient image enhancement by designing a gynecological cervical image enhancement model including multiple key modules; the model includes a cervical image preprocessing module, a double-layer image preprocessing module, a double-layer image fusion module and a loss function optimization module, and each module cooperates with each other to jointly improve the quality and diagnostic value of the image; firstly, the cervical image preprocessing module is used to extract and align the color image and the infrared image to ensure the consistency of the information between the modalities; the double-layer image preprocessing module is used to extract the multi-level features of the two modal images to capture the details and deep structural information of the lesion area; the double-layer image fusion module integrates the surface texture information of the color image and the deep structural information of the infrared image through a cross-modal attention mechanism to generate multi-modal enhancement features; finally, the loss function optimization module is used to dynamically adjust the weights of the single-modal and multi-modal features to further optimize the detail performance and visual consistency of the enhanced image; through the collaborative work of these modules, the present invention improves the enhancement effect of gynecological cervical images and provides high-quality image support for the early screening and diagnosis of cervical diseases.

[0027] See also Figure 1 As shown, a method for gynecological cervical image enhancement processing in an embodiment of the present application has the following process.

[0028] S1. Gynecological cervical image acquisition: Gynecological cervical sample images were collected using a medical color camera and an infrared camera, and 4,000 images were obtained. The collected cervical sample images were manually annotated, and the lesion areas in the paired cervical images were aligned. The obtained images constituted the cervical sample image dataset, and the dataset was divided into a training set and a validation set in a ratio of 7:3.

[0029] S2. Construct a gynecological cervical image preprocessing module to encode the features of the cervical color image and the cervical infrared image respectively.

[0030] Furthermore, in step S2, for the gynecological cervical image preprocessing module, the acquired cervical color image and infrared image are paired to ensure that the two images are aligned, and the lesion areas in the gynecological cervical color image and the gynecological cervical infrared image are aligned by image registration to reduce the deviation in subsequent feature fusion.

[0031] S3. Construct a double-layer image preprocessing module and use the double-layer image preprocessing module to extract features of gynecological cervical color images and gynecological cervical infrared images.

[0032] Further, in step S3, for the double-layer image preprocessing module, see FIG. 2 , the cervical color image is input Size , where H and W represent the height and width of the image, 3 represents the three channels of the cervical color image, and the cervical color image Divide into image blocks with a length of p and a width of p. Each image block is flattened into a one-dimensional data vector. All data vectors form a data matrix. ,in represents the number of image blocks, and C is the feature dimension. The feature matrix is ​​input into the Transformer encoder to obtain the enhanced color image feature data. , where L represents the dimension of the projected data; at the same time, the input cervical infrared image Size , where H and W represent the height and width of the image, 1 represents the single channel of the cervical infrared image, and Divide into image blocks with a length of p and a width of p. Each image block is flattened into a one-dimensional data vector. All data vectors form a data matrix. ,in Represents the number of image blocks, and C is the feature dimension. The feature matrix is ​​input into the Transformer encoder to obtain the enhanced infrared image feature data. , where L represents the dimension of the data after projection.

[0033] Furthermore, in step S3, the code for the double-layer image preprocessing module is as follows: class DualImagePreprocessingModule(nn.Module): def __init__(self, embed_dim): super(DualImagePreprocessingModule, self).__init__() self.patch_embed_cs = nn.Conv2d(3, embed_dim, kernel_size=16, stride=16) self.patch_embed_hw = nn.Conv2d(1, embed_dim, kernel_size=16, stride=16) self.transformer_cs = nn.TransformerEncoder( nn.TransformerEncoderLayer(d_model=embed_dim, nhead=8), num_layers=4 ) self.transformer_hw = nn.TransformerEncoder( nn.TransformerEncoderLayer(d_model=embed_dim, nhead=8), num_layers=4 ) def forward(self, cs_image, hw_image): patches_cs = self.patch_embed_cs(cs_image) patches_cs = patches_cs.flatten(2).transpose(1, 2) # [B, N, C] patches_hw = self.patch_embed_hw(hw_image) patches_hw = patches_hw.flatten(2).transpose(1, 2) # [B, N, C] features_cs = self.transformer_cs(patches_cs) features_hw = self.transformer_hw(patches_hw) return features_cs, features_hw.

[0034] S4. Construct a double-layer image fusion module, and use the double-layer image fusion module to fuse the features of the gynecological cervical color image and the gynecological cervical infrared image to form the final feature.

[0035] Further, in step S4, for the double-layer image fusion module, see FIG3 , in the first branch, the color image feature data of the cervical color image is used. As a query , using infrared image feature data of cervical infrared images As key and value, generate and In the second branch, the infrared image feature data of the cervical infrared image is used As a query , using the color image feature data of the cervical color image As key and value, generate and ,in , d is the dimension of feature alignment between image data; then the first branch cross-image attention weight calculation is performed ,in Indicates the degree of attention of the first branch, is the normalization function, and then the second branch cross-image attention weight calculation is performed ,in Indicates the attention level of the second branch; using a multi-head attention mechanism, each head independently calculates the attention output , , where n represents each computing head, and then the spatial extreme value selection layer is used to integrate the attention levels of the two branches to obtain the final cross-image feature ,in represents the spatial extreme value selection process, Represents dimension expansion concatenation.

[0036] Furthermore, in step S4, the code for the double-layer image fusion module is as follows: class DualImageFusionModule(nn.Module): def __init__(self, embed_dim): super(DualImageFusionModule, self).__init__() self.query_cs = nn.Linear(embed_dim, embed_dim) self.key_hw = nn.Linear(embed_dim, embed_dim) self.value_hw = nn.Linear(embed_dim, embed_dim) self.query_hw = nn.Linear(embed_dim, embed_dim) self.key_cs = nn.Linear(embed_dim, embed_dim) self.value_cs = nn.Linear(embed_dim, embed_dim) self.softmax = nn.Softmax(dim=-1) def forward(self, cs_features, hw_features): # CS as Query, HW as Key and Value q_cs = self.query_cs(cs_features) k_hw = self.key_hw(hw_features) v_hw = self.value_hw(hw_features) attention_cs = self.softmax(torch.matmul(q_cs, k_hw.transpose(-2,-1)) / (cs_features.size(-1) / 2)) fused_cs = torch.matmul(attention_cs, v_hw) q_hw = self.query_hw(hw_features) k_cs = self.key_cs(cs_features) v_cs = self.value_cs(cs_features) attention_hw = self.softmax(torch.matmul(q_hw, k_cs.transpose(-2,-1)) / (hw_features.size(-1) / 2)) fused_hw = torch.matmul(attention_hw, v_cs) fused_features = torch.cat((fused_cs, fused_hw), dim=-1) return fused_features.

[0037] S5. Construct a first-level loss function and adjust the weights to make the enhanced image have more details and visual consistency.

[0038] Further, in step S5, for the first-level loss function, a first-level loss function is constructed ,in and They are the single image loss of color images and the single image loss of infrared images, is the cross-image loss function, Controls the weight of each loss.

[0039] Further, in step S5, the gynecological cervical image enhancement processing model is implemented in code in the Pycharm application and Python language, and the Pytorch framework is used for learning and model training. During the training process, the model input is a low-resolution gynecological cervical image with a resolution of 600×600×3, and the model is trained from scratch for a total of 300 epochs. A complete method of evaluation and evaluation is carried out during the training process to ensure ultra-high accuracy and complete detailed description of the enhanced image, so as to improve the ability to detect and identify cervical diseases.

[0040] S6. Gynecological cervical image processing, obtaining a gynecological cervical image that needs to be enhanced, inputting it into a gynecological cervical image enhancement model, and outputting a high-resolution gynecological cervical image.

[0041] Further, in step S6, if Figure 4 and Figure 5 As shown, Figure 4 The left half shows a low-resolution color image of the cervix. Figure 4 The right half shows a low-resolution infrared image of the cervix. Figure 5 The high-resolution gynecological cervical image processed by the gynecological cervical image enhancement model is displayed. It can be seen that the image clarity and detail performance have been improved, and the details are presented more clearly, making subsequent disease identification more accurate.

[0042] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A gynecological cervical image enhancement processing method, characterized in that: The following steps are involved: S1. Gynecological cervical image acquisition: using a medical color camera and an infrared camera to acquire gynecological cervical sample images, and aligning the lesion areas in the paired cervical images; S2, constructing a gynecological cervical image preprocessing module to encode the features of the cervical color image and the cervical infrared image respectively; S3, constructing a double-layer image preprocessing module, and using the double-layer image preprocessing module to extract features of gynecological cervical color image and gynecological cervical infrared image; S4, constructing a double-layer image fusion module, using the double-layer image fusion module to fuse the features of the gynecological cervix color image and the gynecological cervix infrared image to form a final feature; S5. Construct a first-level loss function to make the enhanced image have more details and visual consistency by adjusting the weights; S6. Gynecological cervical image processing, obtaining a gynecological cervical image that needs to be enhanced, inputting it into a gynecological cervical image enhancement model, and outputting a high-resolution gynecological cervical image.

2. A gynecological cervical image enhancement processing method according to claim 1, characterized in that: The specific method for constructing the gynecological cervical image preprocessing module in S2 is: pairing the acquired cervical color image and infrared image to ensure that the two images are aligned, and aligning the lesion areas in the gynecological cervical color image and the gynecological cervical infrared image by registering the images.

3. A gynecological cervical image enhancement processing method according to claim 1, characterized in that: The specific method for constructing the double-layer image preprocessing module in S3 is as follows: input the cervical color image Size , where H and W represent the height and width of the image, 3 represents the three channels of the cervical color image, and the cervical color image Divide into image blocks with a length of p and a width of p. Each image block is flattened into a one-dimensional data vector. All data vectors form a data matrix. ,in represents the number of image blocks, and C is the feature dimension; Input the feature matrix into the Transformer encoder to obtain the enhanced color image feature data , where L represents the dimension of the projected data; at the same time, the input cervical infrared image Size , where H and W represent the height and width of the image, 1 represents a single channel of the cervical infrared image, and the cervical infrared image Divide into image blocks with a length of p and a width of p. Each image block is flattened into a one-dimensional data vector. All data vectors form a data matrix. ,in represents the number of image blocks, and C is the feature dimension; Input the feature matrix into the Transformer encoder to obtain the enhanced infrared image feature data , where L represents the dimension of the data after projection.

4. A gynecological cervical image enhancement processing method according to claim 1, characterized in that: The specific method for constructing the double-layer image fusion module in S4 is as follows: in the first branch, the color image feature data of the cervical color image is used As a query , using infrared image feature data of cervical infrared images As key and value, generate and In the second branch, the infrared image feature data of the cervical infrared image is used As a query , using the color image feature data of the cervical color image As key and value, generate and ,in , d It is the dimension of feature alignment between image data; Then perform the first branch cross-image attention weight calculation ,in Indicates the degree of attention of the first branch, is the normalization function, and then the second branch cross-image attention weight calculation is performed ,in Indicates the attention level of the second branch; using a multi-head attention mechanism, each head independently calculates the attention output , , where n represents each computing head, and then the spatial extreme value selection layer is used to integrate the attention levels of the two branches to obtain the final cross-image feature ,in represents the spatial extreme value selection process, Represents dimension expansion concatenation.

5. The method for enhancing gynecological cervical images according to claim 1, characterized in that: The specific method for constructing the first-level loss function in S5 is: constructing a first-level loss function ,in and They are the single image loss of color images and the single image loss of infrared images, is the cross-image loss function, Controls the weight of each loss.

Images