Image feature extraction method, device and storage medium

Abstract

This application discloses an image feature extraction method, device, and storage medium. The image feature extraction method includes: extracting frequency domain information of an image to be processed; dividing the image to be processed into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions; removing interfering image information from the other frequency domain image regions to obtain frequency domain image templates corresponding to the other frequency domain image regions; performing attention calculation on the other frequency domain image regions and the low-frequency image regions based on the frequency domain image templates to obtain attention features corresponding to the other frequency domain image regions; and fusing the attention features and the image features corresponding to the low-frequency image regions to obtain the image feature extraction result. A morphological attention mechanism can be embedded into a neural network model. When the frequency domain features change, the morphological attention will also adapt accordingly, enabling joint optimization of model parameters and morphological attention processing to accurately extract image features.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image feature extraction method, device and storage medium. Background Technology

[0002] Existing image processing methods mostly use spatial domain images as the direct processing object. Recently, there has been a gradual increase in the number of methods that use frequency domain transformation to convert images to the frequency domain before processing. Methods based on frequency domain features often treat each frequency domain component as an independent object and then fuse them.

[0003] However, it is difficult to extract meaningful image features from individual frequency domain components, which may reduce the actual performance of the algorithm. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides at least one image feature extraction method, apparatus, and storage medium.

[0005] The first aspect of this application provides an image feature extraction method, which includes: extracting frequency domain information of the image to be processed; dividing the image to be processed into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions; wherein, the low-frequency image regions are image regions corresponding to low-frequency components, and the other frequency domain image regions are image regions of other frequency domain components besides low-frequency components; removing interference image information in the other frequency domain image regions to obtain frequency domain image templates corresponding to the other frequency domain image regions; performing attention calculation on the other frequency domain image regions and the low-frequency image regions based on the frequency domain image templates to obtain attention features corresponding to the other frequency domain image regions; and fusing the attention features and the image features corresponding to the low-frequency image regions to obtain the image feature extraction result.

[0006] In one embodiment, the image to be processed is divided into regions in the spatial domain based on frequency domain information to obtain low-frequency image regions and other frequency domain image regions. This includes: dividing the frequency domain information into low-frequency components and other frequency domain components according to the frequency domain information level; and converting the low-frequency components and other frequency domain components from the frequency domain to the spatial domain to obtain low-frequency image regions and other frequency domain image regions.

[0007] In one embodiment, the image to be processed is divided into regions in the spatial domain based on frequency domain information to obtain low-frequency image regions and other frequency domain image regions. This includes: dividing the image to be processed into low-frequency image regions, mid-frequency image regions and high-frequency image regions in the spatial domain based on frequency domain information, wherein the mid-frequency image regions are the image regions corresponding to the mid-frequency components and the high-frequency image regions are the image regions corresponding to the high-frequency components.

[0008] In one embodiment, removing interfering image information from other frequency domain image regions to obtain frequency domain image templates corresponding to other frequency domain image regions includes: removing isolated noise from intermediate frequency image regions to obtain intermediate frequency image templates corresponding to intermediate frequency image regions; and removing large-area connected regions from high frequency image regions to obtain high frequency image templates corresponding to high frequency image regions.

[0009] In one embodiment, removing isolated noise in the intermediate frequency image region to obtain an intermediate frequency image template corresponding to the intermediate frequency image region includes: binarizing the intermediate frequency image region to obtain an initial intermediate frequency template; performing image erosion processing on the initial intermediate frequency template to obtain a connected intermediate frequency texture; and performing image dilation processing on the connected intermediate frequency texture to obtain an intermediate frequency image template corresponding to the intermediate frequency image region.

[0010] In one embodiment, removing large connected regions in a high-frequency image region to obtain a high-frequency image template corresponding to the high-frequency image region includes: binarizing the high-frequency image region to obtain an initial high-frequency template; performing image erosion processing on the initial high-frequency template and inverting it to obtain an isolated high-frequency texture; and performing image dilation processing on the isolated high-frequency texture to obtain a high-frequency image template corresponding to the high-frequency image region.

[0011] In one embodiment, attention calculation is performed on other frequency domain image regions and low-frequency image regions based on a frequency domain image template to obtain attention features corresponding to other frequency domain image regions. This includes: performing attention calculation on low-frequency image regions and mid-frequency image regions based on a mid-frequency image template to obtain mid-frequency attention features; and performing attention calculation on low-frequency image regions and high-frequency image regions based on a high-frequency image template to obtain high-frequency attention features.

[0012] In one embodiment, an image feature extraction network is pre-trained, which consists of multiple attention modules. The method includes: acquiring attention features obtained by the previous layer attention module through attention calculation, and acquiring fusion features obtained by the previous layer attention module by fusing attention features and low-frequency image regions; inputting the frequency domain image template, the attention features output by the previous layer attention module, and the fusion features into the current layer attention module, acquiring attention features obtained by the current layer attention module through attention calculation, and acquiring fusion features obtained by the current layer attention module by fusing attention features and low-frequency image regions; until the fusion features output by the last layer attention module are acquired, and using the fusion features as the image feature extraction result.

[0013] A second aspect of this application provides an image feature extraction apparatus, comprising: a region division module for extracting frequency domain information of an image to be processed, and dividing the image to be processed into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions; wherein, the low-frequency image regions are image regions corresponding to low-frequency components, and the other frequency domain image regions are image regions of other frequency domain components besides the low-frequency components; a template generation module for removing interfering image information in the other frequency domain image regions to obtain frequency domain image templates corresponding to the other frequency domain image regions; an attention calculation module for performing attention calculation on the other frequency domain image regions and the low-frequency image regions based on the frequency domain image templates to obtain attention features corresponding to the other frequency domain image regions; and a feature extraction module for fusing the attention features and the image features corresponding to the low-frequency image regions to obtain image feature extraction results.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the above-described image feature extraction method.

[0015] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described image feature extraction method.

[0016] The above scheme extracts the frequency domain information of the image to be processed, divides the image into regions in the spatial domain based on the frequency domain information, and obtains low-frequency image regions and other frequency domain image regions; removes interfering image information from the other frequency domain image regions to obtain frequency domain image templates corresponding to the other frequency domain image regions; performs attention calculation on the other frequency domain image regions and the low-frequency image regions based on the frequency domain image templates to obtain attention features corresponding to the other frequency domain image regions; and fuses the attention features and the image features corresponding to the low-frequency image regions to obtain the image feature extraction results. This scheme embeds the morphological attention mechanism into the neural network model, so that when the frequency domain features change, the morphological attention also changes adaptively, enabling joint optimization of model parameters and morphological attention processing, accurately extracting image features, and facilitating subsequent image processing tasks.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a schematic diagram illustrating the implementation environment of the solution in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating an exemplary embodiment of the image feature extraction method of this application; Figure 3 This is a schematic diagram of an image feature extraction network illustrated in an exemplary embodiment of this application; Figure 4 This is a block diagram illustrating an image feature extraction apparatus according to an exemplary embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application; Figure 6 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0021] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0022] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0023] The image feature extraction method provided in the embodiments of this application will be described below.

[0024] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating an implementation environment of the scheme according to an exemplary embodiment of this application. The implementation environment may include a terminal 110 and a server 120, and the terminal 110 and the server 120 are interconnected.

[0025] The number of terminals 110 can be one or more. Terminals 110 can be cameras, smartphones, tablets, laptops, desktop computers, smartwatches, etc., but are not limited to these.

[0026] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0027] In one example, server 120 can perform image feature extraction processing on the image to be processed obtained from terminal 110 to obtain the image feature extraction result. Of course, server 120 can store the image feature extraction result locally, send it back to terminal 110, or transmit it to other terminals.

[0028] In one example, a client application with a target application is installed and running on terminal 110. This target application could be an application that provides image feature extraction functionality. Based on this target application, image feature extraction processing is performed on the image to be processed, yielding the image feature extraction result. Server 120 can be a backend server for this target application, used to provide backend services to the client of this target application.

[0029] The image feature extraction method provided in this application embodiment can be executed by a terminal 110, such as a client of the target application installed and running on the terminal 110, or by a server 120, or by the interaction and cooperation between the terminal 110 and the server 120, that is, some steps of the method are executed by the terminal 110 and other steps are executed by the server 120.

[0030] The image feature extraction method provided in this application can be applied to various image processing tasks, such as image quality assessment, image enhancement, target detection, and image coding. That is, image quality assessment, image enhancement, target detection, and image coding can be achieved based on the image feature extraction results. This application does not limit the scope of the application.

[0031] Please see Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of the image feature extraction method of this application, as shown below. Figure 2 As shown, the image feature extraction method includes at least steps S210 to S240, which are described in detail below: Step S210: Extract the frequency domain information of the image to be processed, and divide the image to be processed into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions; wherein, the low-frequency image region is the image region corresponding to the low-frequency component, and the other frequency domain image regions are the image regions of other frequency domain components other than the low-frequency component.

[0032] The image to be processed can be converted to the frequency domain using algorithms such as Fast Fourier Transform (FFT), wavelet transform (WT), or Discrete Cosine Transform (DCT) to extract the frequency domain information of the image to be processed. The specific conversion method can be flexibly selected according to the actual application scenario, and this application does not limit it.

[0033] This can involve converting the image to be processed into a grayscale image and extracting frequency domain information from the grayscale image, or it can involve extracting frequency domain information from each channel of the image to be processed separately, such as extracting frequency domain information from the R channel, G channel, and B channel separately. This application does not limit this approach.

[0034] Then, based on the frequency domain information of the image to be processed, the image to be processed is divided into regions in the spatial domain to obtain the image regions corresponding to different frequency domain components.

[0035] Specifically, the image regions corresponding to the low-frequency components are divided and used as low-frequency image regions, and the image regions of other frequency domain components besides the low-frequency components are divided and used as other frequency domain image regions.

[0036] The number of frequency domain components other than the low-frequency component can be one or more. For example, other frequency domain components may include high-frequency components and / or mid-frequency components, etc. Alternatively, the high-frequency component can be further divided into ultra-high-frequency components, mid-high-frequency components, low-high-frequency components, etc., and the image region corresponding to each other frequency domain component can be divided to obtain one or more other frequency domain image regions. The number of frequency domain component levels and the number of image regions can be flexibly selected according to the actual application scenario, and this application does not limit them.

[0037] Here is an example illustrating the specific implementation method of dividing the image to be processed into regions in the spatial domain: For example, one could first divide the frequency domain information of the image to be processed into frequency domain components in the frequency domain, and then inversely transform the obtained frequency domain components to the spatial domain to obtain the image regions corresponding to different frequency domain components.

[0038] For example, the image to be processed can be divided into blocks first, such as dividing the image into non-overlapping blocks, extracting frequency domain information for each block to obtain the frequency domain features corresponding to each block, grouping blocks with similar frequency domain features into one category, and each category corresponds to an image region with similar frequency domain features, thus obtaining the image region corresponding to different frequency domain components.

[0039] For example, the texture complexity of different image regions can be evaluated based on the frequency domain information of the image to be processed, and the regions can be divided according to the texture complexity. For example, regions with low texture complexity can be divided into low-frequency image regions, and regions with high texture complexity can be divided into high-frequency image regions.

[0040] The specific method of regional division can be flexibly selected according to the actual application scenario, and this application does not limit it.

[0041] Step S220: Remove interference image information from other frequency domain image regions to obtain frequency domain image templates corresponding to other frequency domain image regions.

[0042] It should be noted that the interference image information is different for image regions with different frequency domain components and for different image processing tasks. The interference image information in other frequency domain image regions needs to be determined based on the image processing task and / or the intensity of the corresponding frequency domain components.

[0043] For example, in object detection tasks, the interference image information corresponding to the mid-frequency image region includes isolated noise, while the interference image information corresponding to the high-frequency image region includes non-isolated noise; in image noise estimation tasks, the interference image information corresponding to both the mid-frequency and high-frequency image regions includes non-isolated noise.

[0044] By removing interference image information from other frequency domain image regions, interference-free frequency domain image templates are obtained for each of the other frequency domain image regions.

[0045] Step S230: Based on the frequency domain image template, perform attention calculation on other frequency domain image regions and low-frequency image regions to obtain the attention features corresponding to other frequency domain image regions.

[0046] Attention computation is a core mechanism in deep learning, used to enable neural network models to pay attention to different parts of the input when processing sequential data. Its core idea is to dynamically allocate attention to different input elements by calculating the weight distribution.

[0047] Based on the frequency domain image template, attention is calculated for other frequency domain image regions and low-frequency image regions to obtain the attention features corresponding to other frequency domain image regions.

[0048] It should be noted that if there are multiple other frequency domain image regions, attention calculation needs to be performed separately for each other frequency domain image region to obtain multiple attention features.

[0049] The neural network model used for attention calculation can be built based on Convolutional Neural Network (CNN) and / or Transformer, etc. The neural network model is trained according to the specific image processing task to obtain specific model parameters. This application does not limit the specific neural network model architecture or training method.

[0050] Step S240: Fuse attention features and image features corresponding to low-frequency image regions to obtain image feature extraction results.

[0051] After obtaining the attention features, the attention features and the image features corresponding to the low-frequency image regions are fused to obtain the image feature extraction results.

[0052] The methods of fusion include, but are not limited to, splicing, and / or weighted summation, etc., and this application does not limit them.

[0053] This application extracts frequency domain information from the image to be processed, and divides the image into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions. Interference image information in the other frequency domain image regions is removed to obtain frequency domain image templates corresponding to those regions. Attention calculation is performed on the other frequency domain image regions and the low-frequency image regions based on the frequency domain image templates to obtain attention features corresponding to the other frequency domain image regions. The attention features and the image features corresponding to the low-frequency image regions are fused to obtain the image feature extraction result. A morphological attention mechanism can be embedded into the neural network model, and the morphological attention will adapt adaptively when the frequency domain features change, enabling joint optimization of model parameters and morphological attention processing to accurately extract image features, facilitating subsequent image processing tasks.

[0054] The following describes some embodiments of this application in detail.

[0055] In some implementations, step S210 involves dividing the image to be processed in the spatial domain based on frequency domain information to obtain low-frequency image regions and other frequency domain image regions, including the following steps S211 to S212.

[0056] Step S211: Divide the frequency domain information into low-frequency components and other frequency domain components according to the frequency domain information level.

[0057] For example, the Fourier transform can be used to convert an image from the spatial domain to the frequency domain. The Fourier transform can decompose an image into a superposition of sine waves of different frequencies, dividing the frequency domain information into low-frequency components and high-frequency components. The low-frequency components correspond to the overall contour and structural information of the image, while the high-frequency components contain detailed information such as edges and textures.

[0058] The threshold for dividing the frequency domain information can be preset based on experience, or it can be flexibly set according to the image content, image type, image processing task requirements, etc. This application does not limit it in this regard.

[0059] Step S212: Convert the low-frequency component and other frequency domain components from the frequency domain to the spatial domain to obtain the low-frequency image region and other frequency domain image regions.

[0060] Then, the low-frequency components and other frequency domain components are transformed from the frequency domain to the spatial domain. For example, the inverse Fourier transform is used to transform the low-frequency components and other frequency domain components from the frequency domain to the spatial domain to obtain the low-frequency image region and other frequency domain image regions.

[0061] In some implementations, based on frequency domain information, the image to be processed is divided into a low-frequency image region, a mid-frequency image region, and a high-frequency image region in the spatial domain. The mid-frequency image region is the image region corresponding to the mid-frequency component, and the high-frequency image region is the image region corresponding to the high-frequency component.

[0062] Specifically, low-frequency image regions correspond to smooth areas in the image, containing the main image information; mid-frequency image regions correspond to the image's contours and / or textures; and high-frequency image regions correspond to the image's details and / or noise.

[0063] For example, the low-frequency component, mid-frequency component, and high-frequency component can be extracted using the following formula 1: (Formula 1) Where I represents the image to be processed. This represents the frequency domain transformation operation of the image. This indicates that the frequency domain information is divided into three parts: low, medium, and high. These represent the low-frequency component, mid-frequency component, and high-frequency component, respectively.

[0064] Then, the low-frequency image region, mid-frequency image region, and high-frequency image region are obtained by using the following formulas 2 to 4 respectively: (Formula 2) (Formula 3) (Formula 4) in, This indicates a conversion from the frequency domain to the spatial domain. These represent the low-frequency image region, the mid-frequency image region, and the high-frequency image region, respectively.

[0065] Taking low-frequency, mid-frequency, and high-frequency image regions as examples, the steps for generating frequency domain image templates and extracting attention features are illustrated.

[0066] In some implementations, step S220 removes interference image information from other frequency domain image regions to obtain frequency domain image templates corresponding to other frequency domain image regions, including the following steps S221 to S222.

[0067] Step S221: Remove isolated noise in the intermediate frequency image region to obtain the intermediate frequency image template corresponding to the intermediate frequency image region.

[0068] Isolated noise can be removed using morphological filtering methods. For example, to remove isolated noise in the intermediate frequency (IF) image region and obtain the IF image template corresponding to the IF image region, the following steps are taken: binarizing the IF image region to obtain the initial IF template; performing image erosion processing on the initial IF template to obtain the connected IF texture; and performing image dilation processing on the connected IF texture to obtain the IF image template corresponding to the IF image region.

[0069] Image erosion can remove pixel locations containing isolated noise, retaining only large connected texture regions.

[0070] Image dilation is used to restore connected mid-frequency textures to their original size.

[0071] Specifically, the intermediate frequency image template is extracted using the following formula 5: (Formula 5) in, , These represent image erosion and image dilation processes, respectively; E is the structuring element used in the operation. Indicates the mid-frequency image region Binarization is performed; t represents the binarization threshold; The intermediate frequency image template is represented by a binary matrix with a value of 0 or 1.

[0072] Of course, other methods can also be used to remove isolated noise. For example, the connected components in the intermediate frequency image region can be identified, the area of ​​each connected component can be counted, and connected components smaller than a preset area threshold can be filtered to remove isolated noise and obtain the intermediate frequency image template corresponding to the intermediate frequency image region; or, a preset filter can be used to filter the intermediate frequency image region to obtain the intermediate frequency image template corresponding to the intermediate frequency image region. This application does not limit the specific method of removing isolated noise.

[0073] Step S222: Remove large connected regions in the high-frequency image region to obtain the high-frequency image template corresponding to the high-frequency image region.

[0074] Similarly, large connected regions can be removed using morphological filtering methods. For example, large connected regions in high-frequency image regions can be removed to obtain high-frequency image templates corresponding to the high-frequency image regions. This includes: binarizing the high-frequency image regions to obtain initial high-frequency templates; performing image erosion processing on the initial high-frequency templates and inverting them to obtain isolated high-frequency textures; and performing image dilation processing on the isolated high-frequency textures to obtain high-frequency image templates corresponding to the high-frequency image regions.

[0075] Image erosion and inversion can produce isolated high-frequency textures containing only isolated noise and / or details.

[0076] Image dilation is used to restore isolated high-frequency textures to their original size.

[0077] Specifically, the high-frequency image template is extracted using the following formula 6: (Formula 6) in, , These represent image erosion and image dilation processes, respectively; E is the structuring element used in the operation. Indicates high-frequency image regions Binarization is performed; t represents the binarization threshold; Represents a high-frequency image template, which is a binary matrix with a value of 0 or 1.

[0078] Of course, other methods can also be used to remove large connected regions. For example, the connected regions in the mid-frequency image region can be identified, the area of ​​each connected region can be counted, and the connected regions with an area greater than a preset area threshold can be filtered to remove large connected regions and obtain the high-frequency image template corresponding to the high-frequency image region; or, a preset filter can be used to filter the high-frequency image region to obtain the high-frequency image template corresponding to the high-frequency image region. This application does not limit the specific method of removing large connected regions.

[0079] It should be noted that steps S221 and S222 above are not executed sequentially.

[0080] By removing interfering image information from each frequency domain image region, the accuracy of the frequency domain image template is improved, which facilitates subsequent attention calculations on different frequency domain information.

[0081] In some implementations, dilatational erosion algorithms, connected component area statistics algorithms, or preset filters are used to remove isolated noise in the intermediate frequency (IF) image region to obtain an initial IF template, and to remove large connected regions in the high-frequency (HF) image region to obtain an initial HF template. The difference between the HF template and the initial HF image region is calculated to obtain high-frequency connected regions. These high-frequency connected regions are added to the IF image region to obtain an IF supplementary image. Isolated noise is then removed from the IF supplementary image to obtain the final IF image template. Similarly, the difference between the IF image template and the initial IF image region is calculated to obtain IF isolated noise. This IF isolated noise is added to the HF image region to obtain a HF supplementary image. Large connected regions are then removed from the HF supplementary image to obtain the final HF image template.

[0082] Since the texture, details, and noise of an image may overlap in the frequency domain, the information such as texture, details, and noise may be distributed in multiple different frequency domain components. Therefore, by adding the information filtered initially to the image region corresponding to another frequency domain component through the above method, interference information filtering is performed to ensure the accuracy of the extraction of information such as texture, details, and noise and to avoid omissions in information extraction.

[0083] Then, attention calculations are performed based on the mid-frequency image template and the high-frequency image template respectively.

[0084] In some implementations, step S230 involves performing attention calculations on other frequency domain image regions and low-frequency image regions based on the frequency domain image template to obtain attention features corresponding to other frequency domain image regions, including the following steps S231 to S232.

[0085] Step S231: Perform attention calculation on the low-frequency image region and the mid-frequency image region based on the mid-frequency image template to obtain the mid-frequency attention features.

[0086] For example, the process of attention calculation is represented by the following formula 7: (Formula 7) in, The attention network layer corresponding to the mid-frequency image region is composed of networks such as convolutional neural networks and / or Transformers. Indicates mid-frequency attention characteristics; Indicates the mid-frequency image region; Represents an intermediate frequency image template; This indicates the low-frequency image region.

[0087] Step S232: Perform attention calculation on low-frequency and high-frequency image regions based on high-frequency image templates to obtain high-frequency attention features.

[0088] For example, the process of attention calculation is represented by the following formula 8: (Formula 8) in, The attention network layer representing the high-frequency image region is composed of networks such as convolutional neural networks and / or Transformers. Indicates high-frequency attention characteristics; Indicates high-frequency image regions; Represents a high-frequency image template; This indicates the low-frequency image region.

[0089] It should be noted that steps S231 and S232 are not executed sequentially.

[0090] The above embodiments improve the accuracy of image information analysis and feature extraction by dividing the image into features in three frequency domains: low, medium, and high, and designing different morphological attention methods for the medium-frequency and high-frequency components according to the different frequency domains.

[0091] Of course, if there are more or fewer other frequency domain image regions, then a similar processing method as that used for the mid-frequency image region and high-frequency image region above is adopted to generate a frequency domain image template and extract attention features.

[0092] Then, the attention features and the image features corresponding to the low-frequency image regions are fused to obtain the image feature extraction results.

[0093] For example, the image features corresponding to low-frequency image regions are fused with mid-frequency attention features and high-frequency attention features in sequence to obtain the image feature extraction results.

[0094] The image features corresponding to low-frequency image regions can be extracted using the following formula 9: (Formula 9) in, Represents low-frequency image regions The corresponding image embedding network consists of networks such as convolutional neural networks and / or Transformers; This represents the image features corresponding to low-frequency image regions.

[0095] Image feature fusion is performed using the following formula 10: (Formula 10) in, , The feature fusion network is composed of networks such as convolutional neural networks and / or Transformers. This represents the result of image feature fusion.

[0096] You can directly As a result of image feature extraction, it can also be further processed... The same feature extraction process is repeated multiple times to obtain the final image feature extraction result.

[0097] For example, an image feature extraction network is pre-trained, which consists of multiple attention modules. The method includes: obtaining the attention features obtained by the previous layer attention module through attention calculation, and obtaining the fusion features obtained by the previous layer attention module by fusing the attention features and the low-frequency image region; inputting the frequency domain image template, the attention features output by the previous layer attention module, and the fusion features into the current layer attention module, obtaining the attention features obtained by the current layer attention module through attention calculation, and obtaining the fusion features obtained by the current layer attention module by fusing the attention features and the low-frequency image region; until the fusion features output by the last layer attention module are obtained, and using the fusion features as the image feature extraction result.

[0098] Using mid-frequency attention features and high-frequency attention features as examples, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of an image feature extraction network illustrated in an exemplary embodiment of this application, such as... Figure 3 As shown, the image feature extraction network consists of multiple stacked attention modules, where the i-th attention module is represented as... The i-th attention module receives the fused features from the output of the previous layer attention module. Mid-frequency attention characteristics High-frequency attention characteristics and output the fusion features of the current layer. Mid-frequency attention characteristics High-frequency attention characteristics The relevant formula is expressed as Formula 11 below: (Formula 11) The three output features are used as the next attention module for further feature extraction.

[0099] Until the last layer of attention module is obtained The output fusion features are expressed by the following formula 12: (Formula 12) in, The result is the image feature extraction.

[0100] The above embodiments further improve the accuracy of feature extraction by stacking multiple attention modules.

[0101] Figure 4This is a block diagram illustrating an image feature extraction apparatus according to an exemplary embodiment of this application. Figure 4 As shown, the exemplary image feature extraction device 400 includes: The region division module 410 is used to extract the frequency domain information of the image to be processed, and to divide the image to be processed into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions; wherein, the low-frequency image region is the image region corresponding to the low-frequency component, and the other frequency domain image regions are the image regions of other frequency domain components other than the low-frequency component. The template generation module 420 is used to remove interference image information in other frequency domain image regions and obtain frequency domain image templates corresponding to other frequency domain image regions. The attention calculation module 430 is used to perform attention calculation on other frequency domain image regions and low-frequency image regions based on the frequency domain image template, so as to obtain the attention features corresponding to other frequency domain image regions. The feature extraction module 440 is used to fuse attention features and image features corresponding to low-frequency image regions to obtain image feature extraction results.

[0102] It should be noted that the image feature extraction device and the image feature extraction method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the image feature extraction device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0103] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment of this application. The electronic device 500 includes a memory 510 and a processor 520. The processor 520 executes program instructions stored in the memory 510 to implement the steps in any of the above-described image feature extraction method embodiments. In a specific implementation scenario, the electronic device 500 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 500 may also include mobile devices such as laptops and tablets, without limitation.

[0104] Specifically, processor 520 controls itself and memory 510 to implement the steps in any of the above-described image feature extraction method embodiments. Processor 520 may also be referred to as a Central Processing Unit (CPU). Processor 520 may be an integrated circuit chip with signal processing capabilities. Processor 520 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 520 may be implemented using integrated circuit chips.

[0105] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the structure of a computer-readable storage medium in an exemplary embodiment of this application. The computer-readable storage medium 600 stores program instructions 610 that can be executed by a processor. The program instructions 610 are used to implement the steps in any of the above-described image feature extraction method embodiments.

[0106] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0107] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0109] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image feature extraction method, characterized in that, The method includes: Extract the frequency domain information of the image to be processed, and divide the image into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions; wherein, the low-frequency image regions are the image regions corresponding to the low-frequency components, and the other frequency domain image regions are the image regions of other frequency domain components besides the low-frequency components. Remove the interfering image information from the other frequency domain image regions to obtain the frequency domain image templates corresponding to the other frequency domain image regions; Based on the frequency domain image template, attention calculation is performed on the other frequency domain image regions and the low-frequency image regions to obtain the attention features corresponding to the other frequency domain image regions. The image feature extraction result is obtained by fusing the attention features and the image features corresponding to the low-frequency image regions.

2. The method according to claim 1, characterized in that, The step of dividing the image to be processed into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions includes: The frequency domain information is divided into low-frequency components and other frequency domain components according to their relative levels. The low-frequency component and the other frequency domain components are converted from the frequency domain to the spatial domain to obtain the low-frequency image region and the other frequency domain image regions.

3. The method according to claim 1, characterized in that, The step of dividing the image to be processed into regions in the spatial domain based on the frequency domain information to obtain low-frequency image regions and other frequency domain image regions includes: Based on the frequency domain information, the image to be processed is divided into a low-frequency image region, a mid-frequency image region, and a high-frequency image region in the spatial domain. The mid-frequency image region is the image region corresponding to the mid-frequency component, and the high-frequency image region is the image region corresponding to the high-frequency component.

4. The method according to claim 3, characterized in that, The step of removing interference image information from the other frequency domain image regions to obtain the frequency domain image templates corresponding to the other frequency domain image regions includes: Remove isolated noise from the intermediate frequency image region to obtain the intermediate frequency image template corresponding to the intermediate frequency image region; Remove large connected regions from the high-frequency image region to obtain the high-frequency image template corresponding to the high-frequency image region.

5. The method according to claim 4, characterized in that, The step of removing isolated noise from the intermediate frequency image region to obtain the intermediate frequency image template corresponding to the intermediate frequency image region includes: The intermediate frequency image region is binarized to obtain the initial intermediate frequency template; The initial mid-frequency template is subjected to image erosion processing to obtain a connected mid-frequency texture; The connected intermediate frequency texture is subjected to image dilation processing to obtain the intermediate frequency image template corresponding to the intermediate frequency image region.

6. The method according to claim 4, characterized in that, The step of removing large connected regions from the high-frequency image region to obtain the high-frequency image template corresponding to the high-frequency image region includes: The high-frequency image regions are binarized to obtain high-frequency initial templates; The high-frequency initial template is subjected to image erosion processing and inverted to obtain isolated high-frequency textures; The isolated high-frequency texture is subjected to image dilation processing to obtain the high-frequency image template corresponding to the high-frequency image region.

7. The method according to claim 4, characterized in that, The step of performing attention calculations on the other frequency domain image regions and the low-frequency image regions based on the frequency domain image template to obtain attention features corresponding to the other frequency domain image regions includes: Attention is calculated on the low-frequency image region and the mid-frequency image region based on the mid-frequency image template to obtain mid-frequency attention features; Attention is calculated on the low-frequency image region and the high-frequency image region based on the high-frequency image template to obtain high-frequency attention features.

8. The method according to claim 1, characterized in that, The method includes a pre-trained image feature extraction network, which consists of multiple attention modules, and the method further includes: Obtain the attention features obtained by the attention module of the previous layer through attention calculation, and obtain the fusion features obtained by the previous layer attention module by fusing the attention features and the low-frequency image region; The frequency domain image template, the attention features output by the previous layer attention module, and the fusion features are input into the current layer attention module to obtain the attention features obtained by the current layer attention module through attention calculation, and the fusion features obtained by the current layer attention module through fusing the attention features and the low-frequency image region. The process continues until the fusion features output by the last attention module are obtained, and these fusion features are used as the result of image feature extraction.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-8.

Images