Image feature processing method, device, electronic device and storage medium

Abstract

The present invention provides an image feature processing method, device, electronic device and storage medium. The image feature processing method includes: obtaining initial image features; processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; fusing each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fusing each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; processing the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and processing the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; and fusing the first frequency image feature and the second frequency image feature to obtain a target image feature. The method can improve the image feature processing effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image feature processing method, device, electronic device and storage medium. Background Art

[0002] With the development of image processing technology, the requirements for image feature processing are getting higher and higher, and the image feature processing effect plays a vital role in the image processing effect. Image feature processing is widely used in image recognition, object detection and image segmentation.

[0003] In the prior art, a convolution kernel of a single scale is often used to process image features, which results in poor image feature processing effect. Summary of the invention

[0004] The present invention provides an image feature processing method, device, electronic device and storage medium, which are used to solve the defect of poor image feature processing effect in the prior art and achieve the purpose of improving the image feature processing effect.

[0005] The present invention provides an image feature processing method, comprising: obtaining initial image features; processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; fusing each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fusing each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; processing the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and processing the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; and fusing the first frequency image feature and the second frequency image feature to obtain a target image feature.

[0006] According to an image feature processing method provided by the present invention, the hybrid convolution kernel includes the point-by-point convolution kernel and the multi-scale convolution kernel, and the use of the hybrid convolution kernel to process the second fused image feature to obtain the second frequency image feature includes: using the point-by-point convolution kernel to process the second fused image feature to obtain a first frequency sub-image feature; using the multi-scale convolution kernel to process the second fused image feature to obtain a second frequency sub-image feature; and obtaining the second frequency image feature based on the first frequency sub-image feature and the second frequency sub-image feature.

[0007] According to an image feature processing method provided by the present invention, the step of obtaining the second frequency image feature based on the first frequency sub-image feature and the second frequency sub-image feature comprises: performing feature difference processing on the first frequency sub-image feature and the second frequency sub-image feature to obtain a difference image feature; and performing sampling adjustment on the difference image feature according to an embedded object to obtain the second frequency image feature; wherein the embedded object is a model or network for training the difference image feature.

[0008] According to an image feature processing method provided by the present invention, the process of obtaining the multi-scale convolution kernel includes: obtaining a reference convolution kernel; and adjusting the reference convolution kernel using a multi-scale convolution coefficient to obtain the multi-scale convolution kernel.

[0009] According to an image feature processing method provided by the present invention, the use of the multi-scale convolution kernel to process the second fused image feature to obtain a second frequency sub-image feature includes: using a candidate scale convolution kernel in the multi-scale convolution kernel to process the second fused image feature to obtain an intermediate frequency sub-image feature; splicing the intermediate frequency sub-image features corresponding to each of the candidate scale convolution kernels to obtain the second frequency sub-image feature.

[0010] According to an image feature processing method provided by the present invention, the processing of the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set includes: based on a first adaptive coefficient, processing the initial image features to obtain a first processed image feature; dividing the first processed image features according to the number of division layers to obtain the first intermediate image feature set; and based on a second adaptive coefficient, processing the initial image features to obtain a second processed image feature; dividing the second processed image features according to the number of division layers to obtain the second intermediate image feature set; wherein the first adaptive coefficient is different from the second adaptive coefficient.

[0011] The present invention also provides an image feature processing device, including: a first acquisition module, used to acquire initial image features; a second acquisition module, used to process the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; a third acquisition module, used to fuse each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and, to fuse each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; a fourth acquisition module, used to process the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and, to process the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; the first acquisition module, used to fuse the first frequency image feature and the second frequency image feature to obtain a target image feature.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned image feature processing methods are implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image feature processing methods are implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any one of the above-mentioned image feature processing methods are implemented.

[0015] The image feature processing method, device, electronic device and storage medium provided by the present invention obtain initial image features; process the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; fuse each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fuse each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; process the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and process the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; and fuse the first frequency image feature and the second frequency image feature to obtain a target image feature. By fusing the first frequency image feature and the second frequency image feature, a target image feature with a more prominent frequency feature can be obtained, thereby improving the image feature processing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flow charts of the image feature processing method provided by the present invention;

[0018] Figure 2 This is the second flow chart of the image feature processing method provided by the present invention;

[0019] Figure 3 This is the third flow chart of the image feature processing method provided by the present invention;

[0020] Figure 4 This is the fourth flow chart of the image feature processing method provided by the present invention;

[0021] Figure 5 This is the fifth flow chart of the image feature processing method provided by the present invention;

[0022] Figure 6 This is one of the schematic diagrams of the architecture of the image feature processing method provided by the present invention;

[0023] Figure 7 This is the second schematic diagram of the architecture of the image feature processing method provided by the present invention;

[0024] Figure 8 It is a structural schematic diagram of the image feature processing device provided by the present invention;

[0025] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Combine the following Figure 1-Figure 5 The image feature processing method of the present invention is described.

[0028] In one embodiment, Figure 1As shown, a method for processing image features is provided, which is described by taking the method applied to a server as an example, and includes the following steps:

[0029] Step 102: Acquire initial image features.

[0030] Among them, image features refer to the characteristics of an image that are different from other images, including color, texture or shape.

[0031] Specifically, the server may extract features from the image using an image feature extraction method to obtain initial image features.

[0032] In one embodiment, the server may extract features from the image using a feature extraction method such as SIFT (Scale-invariant feature transform) or HOG (Histogram of Oriented Gradient) to obtain initial image features.

[0033] Step 104: Process the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set.

[0034] Specifically, after acquiring the initial image features, the server processes the initial image features using different weights to obtain different sets of intermediate image features.

[0035] In one embodiment, the weights may be adaptive coefficients obtained using an attention mechanism to divide the initial image features into two feature channels. In each feature channel, the initial image features are converted into image features of a size of 1*1 and the same dimension as the initial image features. The first intermediate image feature set and the second intermediate image feature set are obtained by multiplying the adaptive coefficients in each channel by the 1*1 image features, respectively.

[0036] In one embodiment, after obtaining the product of the adaptive coefficient in each channel and the 1*1 image feature, the obtained product image feature is divided into N parts, and the obtained image feature division is the same size as the initial image feature, and the dimension is 1 / N of the initial image feature. The first intermediate image feature set, similarly, can be obtained.

[0037] Step 106, fusing each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fusing each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature.

[0038] Specifically, after obtaining the first intermediate image feature set and the second intermediate image feature set, the server can obtain the first fused image feature by adding and fusing each first intermediate image feature in the first intermediate image feature set; and can obtain the second fused image feature by adding and fusing each second intermediate image feature in the second intermediate image feature set.

[0039] Step 108, using a point-by-point convolution kernel to process the first fused image feature to obtain a first frequency image feature; and using a mixed convolution kernel to process the second fused image feature to obtain a second frequency image feature.

[0040] Specifically, after obtaining the first fused image feature and the second fused image feature, the server uses a point-by-point convolution kernel to extract low-frequency features in the first fused image feature, and uses the low-frequency features as the first frequency image feature. The server uses a mixed convolution kernel to extract high-frequency features in the second fused image feature, and uses the high-frequency features as the second frequency image feature.

[0041] Step 110: Fusing the first frequency image feature and the second frequency image feature to obtain a target image feature.

[0042] Specifically, after the server obtains the first frequency image feature and the second frequency image feature, the server adds and fuses the first frequency image feature and the second frequency image feature to obtain the target image feature.

[0043] In the above-mentioned image feature processing method, the initial image feature is obtained; the initial image feature is processed to obtain a first intermediate image feature set and a second intermediate image feature set; each first intermediate image feature in the above-mentioned first intermediate image feature set is fused to obtain a first fused image feature; and each second intermediate image feature in the above-mentioned second intermediate image feature set is fused to obtain a second fused image feature; the first fused image feature is processed using a point-by-point convolution kernel to obtain a first frequency image feature; and the second fused image feature is processed using a mixed convolution kernel to obtain a second frequency image feature; the first frequency image feature and the second frequency image feature are feature-fused to obtain a target image feature. By fusing the first frequency image feature and the second frequency image feature, a target image feature with a more prominent frequency feature can be obtained, thereby improving the image feature processing effect.

[0044] In one embodiment, Figure 2 As shown, the hybrid convolution kernel includes a point-by-point convolution kernel and a multi-scale convolution kernel. The hybrid convolution kernel is used to process the second fused image features to obtain the second frequency image features including:

[0045] Step 202: Process the second fused image feature using a point-by-point convolution kernel to obtain a first frequency sub-image feature.

[0046] Specifically, after obtaining the second fused image feature, the server processes the second fused image feature using a point-by-point convolution kernel to obtain the first frequency sub-image feature.

[0047] Step 204: Process the second fused image features using a multi-scale convolution kernel to obtain second frequency sub-image features.

[0048] Among them, the multi-scale convolution kernel refers to a convolution kernel including multiple scales, for example, including a 3*3 convolution kernel, a 5*5 convolution kernel, or a 7*7 convolution kernel.

[0049] Specifically, the server processes the second fused image features respectively through each scale convolution kernel in the multi-scale convolution kernel, and splices the processed image features to obtain the second frequency sub-image features.

[0050] Step 206: Obtain a second frequency image feature based on the first frequency sub-image feature and the second frequency sub-image feature.

[0051] Specifically, after obtaining the first frequency sub-image feature and the second frequency sub-image feature, the server may obtain the second frequency sub-image feature by performing correlation calculation on the first frequency sub-image feature and the second frequency sub-image feature.

[0052] In this embodiment, the first frequency sub-image feature is obtained by processing the second fused image feature using a point-by-point convolution kernel, and the second frequency sub-image feature is obtained by processing the second fused image feature using a multi-scale convolution kernel. Based on the first frequency sub-image feature and the second frequency sub-image feature, the second frequency image feature is obtained, which can achieve the purpose of accurately obtaining the second frequency image feature.

[0053] In one embodiment, Figure 3 As shown, based on the first frequency sub-image feature and the second frequency sub-image feature, obtaining the second frequency image feature includes:

[0054] Step 302: Perform feature difference processing on the first frequency sub-image feature and the second frequency sub-image feature to obtain a difference image feature.

[0055] Specifically, after obtaining the first frequency sub-image feature and the second frequency sub-image feature, the server may perform feature difference processing on the first frequency sub-image feature and the second frequency sub-image feature to obtain a difference image feature.

[0056] Step 304: According to the embedded object, the difference image feature is sampled and adjusted to obtain a second frequency image feature.

[0057] The embedded object refers to a model or network trained on the difference image feature. For example, if the embedded object of the difference image feature is a convolutional neural network, the difference image feature can be sampled and adjusted according to the training or application requirements of the convolutional neural network. The sampling adjustment can be downsampling or upsampling.

[0058] Specifically, after obtaining the difference image feature, if the embedded object where the difference image feature is located needs to downsample the difference image feature, the server samples the difference image feature in a downsampling manner to obtain the second frequency image feature.

[0059] In this embodiment, the difference image feature is obtained by performing feature difference processing on the first frequency sub-image feature and the second frequency sub-image feature, and the difference image feature is sampled and adjusted according to the embedded object to obtain the second frequency image feature, wherein the embedded object is a model or network for training the difference image feature. The purpose of improving the portability and multi-adaptability of the second frequency image feature processing method can be achieved.

[0060] In one embodiment, Figure 4 As shown in FIG. 1 , the process of obtaining the above multi-scale convolution kernel includes:

[0061] Step 402, obtaining a reference convolution kernel.

[0062] Among them, the reference convolution kernel refers to a convolution kernel that can be used as a reference, through which the other scale convolution kernels in the multi-scale convolution kernel can be obtained.

[0063] Specifically, the reference convolution kernel can be obtained through empirical values ​​or through the wide application of the convolution kernel. For example, a convolution kernel of 3*3 scale can be used as the reference convolution kernel.

[0064] Step 404: Use the multi-scale convolution coefficient to adjust the reference convolution kernel to obtain a multi-scale convolution kernel.

[0065] Among them, the multi-scale convolution coefficient refers to the coefficient that can adjust each scale convolution kernel in the multi-scale convolution kernel. For example, when the multi-scale convolution is an expanded convolution, the expansion rate can be used as the multi-scale convolution coefficient, or, when the multi-scale convolution is a group convolution, the group convolution coefficient can be used as the multi-scale convolution coefficient.

[0066] Specifically, the server obtains a reference convolution kernel and may use a multi-scale convolution coefficient to obtain a multi-scale convolution kernel, where the multi-scale convolution kernel includes convolution kernels of multiple scales.

[0067] In one embodiment, the multi-scale convolution kernel may be an expanded convolution kernel. Assuming that the expansion rate of the expanded convolution kernel is represented by r, the reference convolution kernel is a 3*3 convolution kernel, and the size of the expanded convolution kernel is K, the expanded convolution kernel K is expressed as a formula:

[0068] K=3+2(r-1) (1)

[0069] It can be understood that if the expanded convolution kernel is further expanded, the expansion coefficient can be increased to obtain a larger convolution kernel.

[0070] In one embodiment, the multi-scale convolution kernel can be a group convolution. Assuming that the group convolution coefficient of the group convolution is represented by g2, the reference convolution kernel is a 3*3 convolution kernel, the current convolution kernel is represented by K1, the reference convolution kernel 3*3 convolution kernel can be used as the current convolution kernel, and the convolution kernel of a larger size is represented by K2. Then the convolution kernel K2 of a larger size can be expressed as a formula:

[0071]

[0072] Among them, g1 is 1 and K1 is 3.

[0073] It can be understood that, through the above formula (2), a larger convolution kernel K2 can be obtained according to the setting of the group convolution coefficient of the group convolution.

[0074] In this embodiment, by obtaining a reference convolution kernel and using a multi-scale convolution coefficient to adjust the reference convolution kernel to obtain a multi-scale convolution kernel, a multi-scale convolution kernel including multiple-scale convolution kernels can be accurately obtained.

[0075] In one embodiment, Figure 5 As shown, the second fused image features are processed using a multi-scale convolution kernel to obtain the second frequency sub-image features including:

[0076] Step 502: Process the second fused image features using the candidate scale convolution kernel in the multi-scale convolution kernel to obtain the intermediate frequency sub-image features.

[0077] Specifically, after the server obtains the multi-scale convolution kernel, the multi-scale convolution kernel includes at least one scale convolution kernel, for example, it may include a 3*3 convolution kernel, a 5*5 convolution kernel, or a convolution kernel of a larger size, etc. The intermediate frequency sub-image feature is obtained by processing the above-mentioned second fused image feature using each candidate convolution kernel in the multi-scale convolution kernel. For example, the second fused image feature is processed using a 3*3 convolution kernel to obtain the intermediate frequency sub-image feature corresponding to the 3*3 convolution kernel processing; similarly, the second fused image feature is processed using a 5*5 convolution kernel to obtain the intermediate frequency sub-image feature corresponding to the 5*5 convolution kernel processing.

[0078] Step 504: concatenate the intermediate frequency sub-image features corresponding to the candidate scale convolution kernels to obtain the second frequency sub-image features.

[0079] Specifically, after obtaining the intermediate frequency sub-image feature, the server may concatenate the intermediate frequency sub-image features corresponding to each candidate scale convolution kernel to obtain the second frequency sub-image feature. For example, the second fused image feature is processed using a 3*3 convolution kernel to obtain an intermediate frequency sub-image feature T1, and the second fused image feature is processed using a 5*5 convolution kernel to obtain an intermediate frequency sub-image feature T2. Then, the intermediate frequency sub-image feature T1 and the intermediate frequency sub-image feature T2 are concatenated to obtain the second frequency sub-image feature.

[0080] In this embodiment, the second fused image feature is processed by using the candidate scale convolution kernel in the multi-scale convolution kernel to obtain the intermediate frequency sub-image feature, and the intermediate frequency sub-image features corresponding to each candidate scale convolution kernel are spliced ​​to obtain the second frequency sub-image feature, which can avoid the loss of key information and enhance the feature representation capability.

[0081] In one embodiment, processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set includes: processing the initial image features based on a first adaptive coefficient to obtain a first processed image feature; dividing the first processed image feature according to the number of division layers to obtain a first intermediate image feature set; and, processing the initial image features based on a second adaptive coefficient to obtain a second processed image feature; dividing the second processed image feature according to the number of division layers to obtain a second intermediate image feature set; wherein the first adaptive coefficient is different from the second adaptive coefficient.

[0082] The adaptive coefficient refers to the coefficient of the channel for processing the initial image features. The coefficient can be the same or different in different channels. The number of division layers refers to the number of layers obtained by dividing the features of each channel.

[0083] Specifically, the server may process the initial image features based on the first adaptive coefficient to obtain first processed image features, and divide the first processed image features to obtain a first intermediate image feature set. Similarly, the server may process the initial image features based on the second adaptive coefficient to obtain second processed image features, and divide the second processed image features to obtain a second intermediate image feature set.

[0084] In one embodiment, after receiving the original image feature, the server converts the feature into a unit image feature of the same dimension and scale of 1*1, and after global pooling, multiplies it with the channel coefficient, i.e., the first adaptive coefficient, to obtain the first processed image feature, and divides the first processed image feature into N parts. For example, assuming that the first adaptive coefficient is represented as α, and the first processed image feature is represented as X α , then the first intermediate image feature set is {X α1 , ..., X α2}; Similarly, assuming that the second adaptive coefficient is denoted as β, and the second processed image feature is denoted as X α , then the second intermediate image feature set is {X β1 ,...,X β2}.

[0085] In this embodiment, the initial image features are processed based on the first adaptive coefficient to obtain the first processed image features; the first processed image features are divided according to the number of division layers to obtain the first intermediate image feature set; and the initial image features are processed based on the second adaptive coefficient to obtain the second processed image features; the second processed image features are divided according to the number of division layers to obtain the second intermediate image feature set, thereby achieving the purpose of strengthening channel information.

[0086] In one embodiment, Figure 6 As shown in the figure, the server splits the acquired initial image features into high-frequency channels and low-frequency channels, and sets the number of branch layers of each channel to 1 / N of the initial image features, obtaining a high-frequency sub-feature set {X α1 , ..., X α2} and the low-frequency sub-feature set {X β1 ,...,X β2}, after fusing each high-frequency sub-feature in the high-frequency sub-feature set, we get the high-frequency feature X top ; After fusing each low-frequency sub-feature in the low-frequency sub-feature set, the low-frequency feature X is obtained bottom ; And use point-by-point convolution to process the low-frequency features, and the processed low-frequency features X low . And use the exponential filter to extract high-frequency features and obtain the processed low-frequency features X high , the processed low-frequency feature X low And the processed high-frequency features X high Add point by point to get the integrated image feature X out It should be noted that C in Represents the dimension of the initial image features, C outrepresents the dimension of the integrated image feature, H or H' represents the height of the image feature, W or W' represents the width of the image feature, S represents the image feature obtained by processing the initial image feature with a 1*1 convolution kernel, and the image feature S is processed by the adaptive coefficient of the channel, and then processed by global pooling and segmented according to the number of layers to obtain the above high-frequency sub-feature set {X α1 , ..., X α2} and the low-frequency sub-feature set {X β1 ,...,X β2}.

[0087] The method in this embodiment as a whole can be set to integrate the convolution module (FreConv), which can replace a 3*3 convolution network, such as ResNet, VGG or DenseNet. It can be understood that ResNet, VGG or DenseNet are all backbone networks of deep learning networks. Effectively extracting and integrating high-frequency and low-frequency features in images, the convolution module FreConv can adaptively utilize the frequency information of input data in a more reasonable way, enhance the feature representation capability, and reduce the computing cost and memory cost of the server.

[0088] In one embodiment, Figure 6 As shown, the above-mentioned quasi-exponential filter includes a subtracted exponential filter and a reducing exponential filter, wherein the subtracted exponential filter is used to perform point-by-point convolution processing on high-frequency features; the reducing exponential filter is used to perform multi-scale convolution on high-frequency features; by subtracting the reducing exponential filter from the subtracted exponential filter, a quasi-exponential filter architecture is constructed, and the quasi-exponential filter architecture is initialized using the parameters of the exponential filter.

[0089] In one embodiment, Figure 7 As shown, the multi-scale convolution processing of high-frequency features includes multi-scale convolution processing of high-frequency features using dilated convolution or group convolution. Taking 3*3 convolution as a reference, the 3*3 convolution is expanded using the dilation rate in the dilated convolution or the group convolution coefficient of the group convolution, and the high-frequency features are processed through each dilated convolution to obtain image features after each dilated convolution processing, and the image features are spliced ​​to obtain spliced ​​image features.

[0090] The image feature processing device provided by the present invention is described below. The image feature processing device described below and the image feature processing method described above can be referred to each other.

[0091] In one embodiment, Figure 8As shown, an image feature processing device is provided, including: a first acquisition module 802, a second acquisition module 804, a third acquisition module 806, a fourth acquisition module 808 and a first acquisition module 810, wherein: the first acquisition module 802 is used to acquire initial image features; the second acquisition module 804 is used to process the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; the third acquisition module 806 is used to fuse each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and, fuse each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; the fourth acquisition module 808 is used to process the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and, process the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; the first acquisition module 810 is used to fuse the first frequency image feature and the second frequency image feature to obtain a target image feature.

[0092] In one embodiment, the fourth acquisition module 808 includes: a first obtaining unit, used to process the second fused image feature using a point-by-point convolution kernel to obtain a first frequency sub-image feature; a second obtaining unit, used to process the second fused image feature using a multi-scale convolution kernel to obtain a second frequency sub-image feature; and a second obtaining unit, used to obtain a second frequency image feature based on the first frequency sub-image feature and the second frequency sub-image feature.

[0093] In one embodiment, the fourth acquisition module 808 is used to perform feature difference processing on the first frequency sub-image feature and the second frequency sub-image feature to obtain a difference image feature; and sample and adjust the difference image feature according to the embedded object to obtain a second frequency image feature; wherein the embedded object is a model or network for training the difference image feature.

[0094] In one embodiment, the fourth acquisition module 808 is used to acquire a reference convolution kernel; and adjust the reference convolution kernel using a multi-scale convolution coefficient to obtain a multi-scale convolution kernel.

[0095] In one embodiment, the fourth acquisition module 808 is used to process the second fused image feature using the candidate scale convolution kernel in the multi-scale convolution kernel to obtain the intermediate frequency sub-image feature; and splice the intermediate frequency sub-image features corresponding to each candidate scale convolution kernel to obtain the second frequency sub-image feature.

[0096] In one embodiment, the second acquisition module 804 is used to process the initial image features based on the first adaptive coefficient to obtain the first processed image features; divide the first processed image features according to the number of division layers to obtain the first intermediate image feature set; and, based on the second adaptive coefficient, process the initial image features to obtain the second processed image features; divide the second processed image features according to the number of division layers to obtain the second intermediate image feature set; wherein the first adaptive coefficient is different from the second adaptive coefficient.

[0097] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the image feature processing method, which includes: obtaining initial image features; processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; fusing each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fusing each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; processing the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and processing the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; and fusing the first frequency image feature and the second frequency image feature to obtain a target image feature.

[0098] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image feature processing method provided by the above-mentioned methods, which includes: obtaining initial image features; processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; fusing each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fusing each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; processing the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and processing the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; and fusing the first frequency image feature and the second frequency image feature to obtain a target image feature.

[0100] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the image feature processing method provided by the above-mentioned methods, the method comprising: obtaining initial image features; processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; fusing each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fusing each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; processing the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and processing the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; and fusing the first frequency image feature and the second frequency image feature to obtain a target image feature.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing image features, characterized in that: include: Obtaining initial image features; Processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; fusing each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and fusing each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; Processing the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and processing the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; The first frequency image feature and the second frequency image feature are fused to obtain a target image feature.

2. The image feature processing method according to claim 1, characterized in that: The hybrid convolution kernel includes the point-by-point convolution kernel and the multi-scale convolution kernel, and the second fused image feature is processed by the hybrid convolution kernel to obtain the second frequency image feature, including: Processing the second fused image feature by using the point-by-point convolution kernel to obtain a first frequency sub-image feature; Processing the second fused image feature by using the multi-scale convolution kernel to obtain a second frequency sub-image feature; The second frequency image feature is obtained based on the first frequency sub-image feature and the second frequency sub-image feature.

3. The image feature processing method according to claim 2, characterized in that: The obtaining the second frequency image feature based on the first frequency sub-image feature and the second frequency sub-image feature comprises: Performing feature difference processing on the first frequency sub-image feature and the second frequency sub-image feature to obtain a difference image feature; According to the embedded object, the difference image feature is sampled and adjusted to obtain the second frequency image feature; wherein the embedded object is a model or network for training the difference image feature.

4. The image feature processing method according to claim 2, characterized in that: The process of obtaining the multi-scale convolution kernel includes: Get the reference convolution kernel; The reference convolution kernel is adjusted using the multi-scale convolution coefficient to obtain the multi-scale convolution kernel.

5. The image feature processing method according to claim 4, characterized in that: The step of processing the second fused image feature by using the multi-scale convolution kernel to obtain the second frequency sub-image feature comprises: Using the candidate scale convolution kernel in the multi-scale convolution kernel, processing the second fused image feature to obtain an intermediate frequency sub-image feature; The intermediate frequency sub-image features corresponding to the candidate scale convolution kernels are concatenated to obtain the second frequency sub-image features.

6. The image feature processing method according to any one of claims 2 to 5, characterized in that: The processing of the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set comprises: Based on the first adaptive coefficient, the initial image feature is processed to obtain a first processed image feature; Dividing the first processed image features according to the number of division layers to obtain the first intermediate image feature set; as well as, Based on the second adaptive coefficient, the initial image feature is processed to obtain a second processed image feature; Dividing the second processed image features according to the number of division levels to obtain a second intermediate image feature set; The first adaptive coefficient is different from the second adaptive coefficient.

7. An image feature processing device, characterized in that: include: A first acquisition module, used to acquire initial image features; A second acquisition module, used for processing the initial image features to obtain a first intermediate image feature set and a second intermediate image feature set; a third acquisition module, configured to fuse each first intermediate image feature in the first intermediate image feature set to obtain a first fused image feature; and to fuse each second intermediate image feature in the second intermediate image feature set to obtain a second fused image feature; a fourth acquisition module, configured to process the first fused image feature using a point-by-point convolution kernel to obtain a first frequency image feature; and to process the second fused image feature using a mixed convolution kernel to obtain a second frequency image feature; The first obtaining module is used to perform feature fusion on the first frequency image feature and the second frequency image feature to obtain a target image feature.

8. The image feature processing device according to claim 7, characterized in that: The hybrid convolution kernel includes the point-by-point convolution kernel and the multi-scale convolution kernel, and the fourth acquisition module includes: a first obtaining unit, configured to process the second fused image feature by using the point-by-point convolution kernel to obtain a first frequency sub-image feature; a second obtaining unit, configured to process the second fused image feature by using the multi-scale convolution kernel to obtain a second frequency sub-image feature; The second obtaining unit is configured to obtain the second frequency image feature based on the first frequency sub-image feature and the second frequency sub-image feature.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the image feature processing method according to any one of claims 1 to 6 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image feature processing method according to any one of claims 1 to 6 are implemented.

Images