Image feature processing method and related apparatus
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU SHIYUAN ELECTRONICS CO LTD
- Filing Date
- 2021-11-23
- Publication Date
- 2026-05-26
AI Technical Summary
Existing convolutional neural network models suffer from overfitting and weak generalization performance when processing images due to feature redundancy.
A spectral normalization layer is used to perform dimensionality transformation, singular value decomposition, and dimensionality transformation on image features in a convolutional neural network. The initial image features are decomposed by the spectral normalization layer to identify singular features, and deconvolution is performed to remove redundant features.
The generalization performance of the model was enhanced by removing redundant image features, thereby improving the model's generalization ability.
Smart Images
Figure CN116168260B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, specifically to an image feature processing method and related apparatus, which includes an image feature processing device, a computer device, and a computer-readable storage medium. Background Technology
[0002] Current Convolutional Neural Networks (CNNs) consist of Convolutional (V) layers, Batch Normalization (BN) layers, and ReLU (Activation Function). BN layers are typically embedded between Convolutional (V) layers and ReLU layers. BN layers are used to normalize feature values and offer the following advantages: 1) faster training and convergence of the neural network; 2) control gradient explosion to prevent gradient vanishing; and 3) prevention of overfitting. BN layers are generally used after non-linear units such as linear and convolutional layers to eliminate variance bias.
[0003] However, when existing convolutional neural network models process images, the presence of feature redundancy can lead to overfitting if this redundancy is not suppressed, resulting in weak generalization performance. Summary of the Invention
[0004] This application provides an image feature processing method and related apparatus, which includes an image feature processing device, a computer device, and a computer-readable storage medium, and can enhance the generalization performance of the model.
[0005] An image feature processing method, comprising:
[0006] Obtain the initial image features generated by the convolutional layers of the convolutional neural network from the image to be processed;
[0007] The initial image features are transformed by using the spectral normalization layer in a convolutional neural network to obtain the initial dimensionality image features;
[0008] A spectral normalization layer is used to perform singular value decomposition on the initial dimension image features in order to determine the singular features corresponding to the initial dimension image features.
[0009] A spectral normalization layer is used to perform dimensional transformation on the singular features to obtain the target dimension image features.
[0010] Accordingly, embodiments of this application provide an image feature processing apparatus, including:
[0011] The acquisition unit can be used to acquire the initial image features generated by the convolutional layers in a convolutional neural network through convolution of the image to be processed.
[0012] The first transformation unit can be used to perform dimensionality transformation processing on the initial image features using the spectral normalization layer in the convolutional neural network to obtain the initial dimensionality image features.
[0013] The decomposition unit can be used to perform singular value decomposition on the initial dimension image features using a spectral normalization layer in order to determine the singular features corresponding to the initial dimension image features.
[0014] The second transformation unit can be used to perform dimensional transformation processing on singular features using a spectral normalization layer to obtain the target dimension image features.
[0015] In some embodiments, the first transformation unit may be specifically used to convolve the initial image features using a spectral normalization layer to obtain convolved image features; and to perform dimensionality transformation processing on the convolved image features to obtain initial dimensional image features.
[0016] In some embodiments, the second transformation unit may be specifically used to deconvolve the singular features using a spectral normalization layer to obtain the deconvolved image features; and to perform dimensionality transformation processing on the deconvolved image features to obtain the target dimension image features.
[0017] In some embodiments, the decomposition unit may be used to perform exponential iteration processing on the initial dimension image features using a spectral normalization layer to determine the singular features corresponding to the initial dimension image features.
[0018] In some embodiments, the decomposition unit may be used to perform power-law iteration on the initial dimension image features using a spectral normalization layer; when the number of power-law iterations reaches a preset threshold, the singular features corresponding to the initial dimension image features are determined.
[0019] In some embodiments, the decomposition unit can be specifically used to perform power-law iteration on the initial dimension image features using a spectral normalization layer to obtain initial singular features; when the initial singular features meet a preset threshold, the initial singular features are determined to be the singular features corresponding to the initial dimension image features.
[0020] In some embodiments, the decomposition unit can be specifically used to perform power-law iteration on the initial dimension image features using a spectral normalization layer to obtain the left singular vector and the right singular vector corresponding to the target singular value of the initial dimension image features; and to fuse the left singular vector and the right singular vector corresponding to the target singular value to obtain the singular features corresponding to the initial dimension image features.
[0021] Furthermore, embodiments of this application also provide a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform operations in any of the image feature processing methods provided in embodiments of this application.
[0022] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program adapted for loading by a processor to execute any of the image feature processing methods provided in embodiments of this application.
[0023] This application embodiment can obtain the initial image features generated by the convolutional layers of a convolutional neural network (CNN) from the image to be processed; perform dimensionality transformation on the initial image features using a spectral normalization layer in the CNN to obtain initial dimensional image features; perform singular value decomposition (SVD) on the initial dimensional image features using a SVD layer to determine the singular features corresponding to the initial dimensional image features; and perform dimensionality transformation on the singular features using a SVD layer to obtain the target dimensional image features. Because this application embodiment can perform SVD on the initial dimensional image features using a spectral normalization layer to determine the singular features corresponding to the initial dimensional image features, redundancy removal of the initial image features is achieved, thereby enhancing the generalization performance of the model. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a schematic diagram of a scene illustrating the image feature processing method provided in an embodiment of this application;
[0026] Figure 2 This is a schematic flowchart of the image feature processing method provided in the embodiments of this application;
[0027] Figure 3 This is a schematic diagram of the convolutional neural network provided in an embodiment of this application;
[0028] Figure 4 This is a schematic diagram of the image feature processing device provided in the embodiments of this application;
[0029] Figure 5 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation
[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0031] This application provides an image feature processing method and related apparatus. The related apparatus includes an image feature processing device, a computer device, and a computer-readable storage medium. The image feature processing device can be integrated into the computer device, which can be a server or a terminal, etc.
[0032] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.
[0033] For example, see Figure 1 Taking the integration of an image feature processing device into a computer device as an example, the computer device acquires the initial image features generated by the convolutional layer of the convolutional neural network through convolution of the image to be processed; the initial image features are transformed by the spectral normalization layer in the convolutional neural network to obtain the initial dimensional image features; the initial dimensional image features are decomposed by the spectral normalization layer to determine the singular features corresponding to the initial dimensional image features; and the target dimensional image features are transformed by the spectral normalization layer.
[0034] The initial image features can be represented by a four-dimensional matrix or a three-dimensional matrix, depending on the specific requirements.
[0035] The initial dimension image features are represented by matrices, and the matrix dimension of the initial dimension image features can be smaller than the matrix dimension of the initial image features.
[0036] The target dimension image features are represented by matrices, and the matrix dimension of the target dimension image features can be the same as the matrix dimension of the initial image features.
[0037] The singular features can be represented by the left singular vector corresponding to the singular values of the initial dimension image features, or by the right singular vector corresponding to the singular values of the initial dimension image features, or by a matrix representation obtained by fusing the left singular vector and the right singular vector of the singular values of the initial dimension image features.
[0038] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.
[0039] This embodiment will be described from the perspective of an image feature processing device, which can be integrated into a computer device, such as a server or a terminal. The terminal can include tablet computers, laptops, personal computers (PCs), wearable devices, virtual reality devices, or other smart devices that can acquire data.
[0040] like Figure 2 As shown, the specific process of this image feature processing method is as follows:
[0041] S101. Obtain the initial image features generated by the convolutional layers in the convolutional neural network from the image to be processed.
[0042] The initial image features can be represented by a four-dimensional matrix or a three-dimensional matrix, depending on the specific requirements.
[0043] The image to be processed can be a medical image or a face image.
[0044] In this application, the convolutional neural network performs convolution processing on the image to be processed through convolutional layers to obtain the initial image features of the image to be processed. The initial image features are represented by a four-dimensional matrix, for example, the initial image features are a matrix of size B×C×H×W. Where B represents the batch size, C represents the number of channels of the initial image features, H represents the width of the initial image features, and W represents the height of the initial image features.
[0045] S102. The initial image features are transformed by the spectral normalization layer in the convolutional neural network to obtain the initial dimensional image features.
[0046] The initial dimensional image features are represented by matrices, and the matrix dimension of the initial dimensional image features can be smaller than the matrix dimension of the initial image features. The initial image features can be four-dimensional matrices.
[0047] The embodiments of this application can achieve dimensional transformation of the initial image features through matrix recombination, that is, transforming the matrix dimension of the initial image features from four dimensions to two dimensions, thereby obtaining the initial dimensional image features.
[0048] The embodiments of this application can reduce the dimensionality of the initial image features by convolving them to obtain the initial dimensionality image features.
[0049] Specifically, the computer device uses a spectral normalization layer to convolve the initial image features to obtain convolved image features; the convolved image features are then subjected to dimensionality transformation to obtain the initial dimensional image features.
[0050] In this process, the computer equipment uses a spectral normalization layer to convolve the initial image features. The resulting convolved image features can be achieved by reducing the number of channels in the initial image features. Specifically, a two-dimensional convolution can reduce the C channels of the initial image features to one channel. During this process, padding is set to 1 and stride is set to 1. The resulting convolved image features have a size of F. B×1×H×W The matrix. The transformation formula between the initial image features and the convolutional image features is as shown in formula (1):
[0051] F B×1×H×W =Conv2D(F B×C×H×W )Formula (1)
[0052] Among them, F B×1×H×W F is the size of the feature matrix of the image after convolution. B×C×H×W It is the size of the matrix of initial image features.
[0053] In this process, the computer equipment performs dimensionality transformation on the convolutional image features to obtain the initial dimensionality image features, which can be understood as reorganizing the convolutional image features.
[0054] For example, dimensionality reduction of the convolved image features can be achieved through convolution to obtain the initial dimension image features, that is, the matrix F of the convolved image features. B×1×H×W Convert to W M×N Where M = B × 1, N = H × W.
[0055] S103. Use a spectral normalization layer to perform singular value decomposition on the initial dimension image features in order to determine the singular features corresponding to the initial dimension image features.
[0056] The singular features can be represented by the left singular vector corresponding to the singular values of the initial dimension image features, or by the right singular vector corresponding to the singular values of the initial dimension image features, or by a matrix representation obtained by fusing the left singular vector and the right singular vector of the singular values of the initial dimension image features.
[0057] This application embodiment utilizes singular value decomposition (SVD) to remove redundancy from the initial dimensional image features. This application embodiment can use the SVD method to perform singular value decomposition on the initial dimensional image features to determine singular features.
[0058] Specifically, the computer device uses a spectral normalization layer to perform exponential iteration on the initial dimension image features in order to determine the singular features corresponding to the initial dimension image features.
[0059] Specifically, the computer device uses a spectral normalization layer to perform exponential iteration on the initial dimensional image features; when the number of exponential iterations reaches a preset threshold, the singular features corresponding to the initial dimensional image features are determined.
[0060] In this embodiment, the singular features are obtained by fusing the left singular vector and the right singular vector with the maximum singular value.
[0061] The preset threshold for the number of iterations is generally 10. That is, when the number of power iterations reaches 10, the left and right singular vectors corresponding to the maximum singular value of the initial dimensional image features can be determined. Multiplying the left and right singular vectors corresponding to the maximum singular value yields the singular features. The matrix of singular features is the matrix after redundancy removal.
[0062] In this embodiment of the application, the singular features corresponding to the initial dimension image features can be determined by low-rank decomposition based on the SVD method. The low-rank decomposition formula is shown in formula (2):
[0063]
[0064] Where W represents the matrix of initial-dimensional image features, r represents the rank of the initial-dimensional image feature matrix, and λ represents the singular values. λ1 represents the maximum singular value, u1 represents the left singular vector corresponding to the maximum singular value, and v1 represents the right singular vector corresponding to the maximum singular value. The maximum singular value is equal to the spectral norm. However, low-rank decomposition is computationally intensive and costly. Therefore, this embodiment uses power iteration processing instead of low-rank decomposition.
[0065] The calculation process for the exponentiation iteration can be as follows:
[0066] a n ←W·b n-1 Formula (3)
[0067] b n ←W·a n Formula (4)
[0068]
[0069] Where n refers to the power iteration number, σn This refers to singular values, W1 refers to the matrix of initial-dimensional image features, and b n It refers to the right singular vector, a n It refers to the left singular vector. When n equals 0, v0 can be a random vector, which can be obtained by using a Gaussian distribution.
[0070] In this embodiment of the application, the computer device uses a spectral normalization layer to perform power-law iteration on the initial dimension image features to obtain the left singular vector and the right singular vector corresponding to the target singular value of the initial dimension image features; the left singular vector and the right singular vector corresponding to the target singular value are fused to obtain the singular features corresponding to the initial dimension image features.
[0071] The target singular value can be the largest singular value; the larger the singular value, the greater its contribution to the matrix. The left and right singular vectors corresponding to the target singular values can be fused by multiplication, thus obtaining singular features. These singular features are defined by their magnitude... The matrix representation of .
[0072] In this embodiment, when the size of the singular feature matrix is close to the size of the matrix of the initial dimension image features, the singular feature matrix is a better matrix after redundancy removal. This can be understood as follows: the computer device uses a spectral normalization layer to perform exponential iteration processing on the initial dimension image features to obtain the initial singular features; when the initial singular features meet a preset threshold, the initial singular features are determined to be the singular features corresponding to the initial dimension image features.
[0073] The preset threshold can be a threshold range, where all values within the threshold range are less than the matrix of the initial dimension image features.
[0074] S104. Use a spectral normalization layer to perform dimensional transformation on the singular features to obtain the target dimension image features.
[0075] The target dimension image features are represented by matrices, and the matrix dimension of the target dimension image features can be the same as the matrix dimension of the initial image features.
[0076] In this embodiment, singular features can be matrix reorganized. This embodiment can achieve dimensional transformation of singular features through matrix reorganization, that is, the matrix dimension of singular features is transformed from two-dimensional to four-dimensional, thereby obtaining the target dimension image features.
[0077] The embodiments of this application can obtain the target dimension image features by deconvolving the singular features.
[0078] Specifically, the computer device uses a spectral normalization layer to deconvolve the singular features to obtain the deconvolutioned image features; the deconvolutioned image features are then subjected to dimensionality transformation processing to obtain the target dimension image features.
[0079] In this process, the computer equipment uses a spectral normalization layer to deconvolve the singular features, resulting in deconvolutioned image features. This can be understood as a reorganization of the singular features, that is, a matrix of the singular features. Reorganizing from two dimensions into a four-dimensional matrix Four-dimensional matrix This is the matrix of image features after deconvolution.
[0080] The process of transforming the dimensions of the deconvolutioned image features to obtain the target dimension image features can be understood as expanding the number of channels in the deconvolutioned image features. In this embodiment, two-dimensional deconvolution can be used to expand the number of channels from 1 to C. During this process, Padding is set to 1 and Stride is set to 1. The transformation formula between the deconvolutioned image features and the target dimension image features is shown in formula (6):
[0081]
[0082] in, It is the size of the image feature matrix after deconvolution. It is the size of the matrix of image features in the target dimension.
[0083] In this embodiment of the application, the computer device also uses a spectral normalization layer to calculate the variance and mean of the target dimension image features, and performs normalization processing on the target dimension image features based on the variance and mean to obtain the normalized target dimension image features.
[0084] In this embodiment, a standard layer can also be used to quantize and offset the standardized target dimension image features, as shown in the following formula:
[0085]
[0086] Where x represents any value in the matrix of target dimension image features, y represents the output value of the spectral normalization layer, E(x) represents the mean, Var(x) represents the variance, γ is the scale parameter, which is used to quantize the normalized target dimension image features, β is the shift parameter, which is used to shift the normalized target dimension image features, and ε is a constant that tends to 0.
[0087] In this embodiment, redundancy removal of initial image features is performed simultaneously on the initial image features of all images to be processed in a batch, rather than on the initial image features of a single image to be processed. This is because the redundant features of images in a batch should be similar. Experiments have shown that redundancy removal of redundant features on a batch of images to be processed is more effective than redundancy removal of the initial image features of a single image to be processed.
[0088] like Figure 3 As shown, the embodiments of this application are applied to convolutional neural networks.
[0089] Conv3-32, Conv3-64, Conv3-128, Conv3-256, and Conv3-512 all represent 3×3 filter convolutional layers, where 32, 64, 128, 256, and 512 represent the number of channels in the convolutional layer; MaxPool represents the max pooling layer; FC Layer represents the fully connected layer with a length of 512; and BSN represents the spectral normalization layer.
[0090] This application embodiment can obtain the initial image features generated by the convolutional layers of a convolutional neural network (CNN) from the image to be processed; perform dimensionality transformation on the initial image features using a spectral normalization layer in the CNN to obtain initial dimensional image features; perform singular value decomposition (SVD) on the initial dimensional image features using a SVD layer to determine the singular features corresponding to the initial dimensional image features; and perform dimensionality transformation on the singular features using a SVD layer to obtain the target dimensional image features. Because this application embodiment can perform SVD on the initial dimensional image features using a spectral normalization layer to determine the singular features corresponding to the initial dimensional image features, redundancy removal of the initial image features is achieved, thereby enhancing the generalization performance of the model.
[0091] To better implement the above methods, this application also provides an image feature processing device, which can be integrated into a computer device, such as a server or terminal. The terminal may include a tablet computer, a laptop computer, and / or a personal computer.
[0092] For example, such as Figure 4 As shown, the image feature processing device may include an acquisition unit 301, a first conversion unit 302, a decomposition unit 303, and a second conversion unit 304, as follows:
[0093] (1) Obtain unit 301;
[0094] The acquisition unit 301 can be used to acquire the initial image features generated by the convolutional layers in the convolutional neural network through convolution of the image to be processed.
[0095] (2) First conversion unit 302;
[0096] The first transformation unit 302 can be used to perform dimensional transformation processing on the initial image features using the spectral normalization layer in the convolutional neural network to obtain the initial dimensional image features.
[0097] In some embodiments, the first transformation unit 302 can be used to convolve the initial image features using a spectral normalization layer to obtain convolved image features; and to perform dimensional transformation processing on the convolved image features to obtain initial dimensional image features.
[0098] (3) Decomposition unit 303;
[0099] The decomposition unit 303 can be used to perform singular value decomposition on the initial dimension image features using a spectral normalization layer to determine the singular features corresponding to the initial dimension image features.
[0100] In some embodiments, the decomposition unit 303 can be used to perform power-law iteration on the initial dimension image features using a spectral normalization layer to determine the singular features corresponding to the initial dimension image features.
[0101] In some embodiments, the decomposition unit 303 can be used to perform power-law iteration on the initial dimension image features using a spectral normalization layer; when the number of power-law iterations reaches a preset threshold, the singular features corresponding to the initial dimension image features are determined.
[0102] In some embodiments, the decomposition unit 303 can be used to perform power-law iteration on the initial dimension image features using a spectral normalization layer to obtain initial singular features; when the initial singular features meet a preset threshold, the initial singular features are determined to be the singular features corresponding to the initial dimension image features.
[0103] In some embodiments, the decomposition unit 303 can be used to perform power-law iteration on the initial dimension image features using a spectral normalization layer to obtain the left singular vector and the right singular vector corresponding to the target singular value of the initial dimension image features; and to fuse the left singular vector and the right singular vector corresponding to the target singular value to obtain the singular features corresponding to the initial dimension image features.
[0104] (4) Second conversion unit 304;
[0105] The second transformation unit 304 can be used to perform dimensional transformation processing on singular features using a spectral normalization layer to obtain the target dimension image features.
[0106] In some embodiments, the second transformation unit 304 can be used to deconvolve the singular features using a spectral normalization layer to obtain the deconvolved image features; and to perform dimensionality transformation processing on the deconvolved image features to obtain the target dimension image features.
[0107] As can be seen from the above, the acquisition unit 301 of this application embodiment can acquire the initial image features generated by the convolutional layer of the convolutional neural network from the image to be processed; the first transformation unit 302 can use the spectral normalization layer in the convolutional neural network to perform dimensional transformation processing on the initial image features to obtain the initial dimensional image features; the decomposition unit 303 can use the spectral normalization layer to perform singular value decomposition processing on the initial dimensional image features to determine the singular features corresponding to the initial dimensional image features; the second transformation unit 304 can use the spectral normalization layer to perform dimensional transformation processing on the singular features to obtain the target dimensional image features. Since this application embodiment can use the spectral normalization layer to perform singular value decomposition processing on the initial dimensional image features to determine the singular features corresponding to the initial dimensional image features, redundancy removal of the initial image features is achieved, thereby enhancing the generalization performance of the model.
[0108] This application also provides a computer device, such as... Figure 5 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:
[0109] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 5 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0110] The processor 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby providing overall monitoring of the computer device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and computer programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.
[0111] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
[0112] The computer device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0113] The computer device may also include an input unit 404, which can be used to receive input digital or character information communication, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
[0114] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 runs the computer programs stored in the memory 402 to realize various functions, as follows:
[0115] The process involves obtaining the initial image features generated by the convolutional layers of a convolutional neural network (CNN) from the image to be processed; performing dimensionality transformation on the initial image features using the spectral normalization layer of the CNN to obtain the initial dimensional image features; performing singular value decomposition on the initial dimensional image features using the spectral normalization layer to determine the singular features corresponding to the initial dimensional image features; and performing dimensionality transformation on the singular features using the spectral normalization layer to obtain the target dimensional image features.
[0116] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0117] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0118] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute any of the image feature processing methods provided in embodiments of this application.
[0119] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0120] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0121] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the image feature processing methods provided in the embodiments of this application, the beneficial effects that any of the image feature processing methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0122] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.
[0123] The foregoing has provided a detailed description of an image feature processing method and related apparatus provided in the embodiments of this application. The related apparatus includes an image feature processing device, a computer device, and a computer-readable storage medium. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. An image feature processing method, characterized in that, include: Obtain the initial image features generated by the convolutional layers of the convolutional neural network from the image to be processed; The initial image features are convolved using a spectral normalization layer to reduce the channel data of the initial image features, resulting in convolved image features. The convolutional image features are subjected to dimensionality transformation processing to perform matrix recombination on the convolutional image features to obtain initial dimensional image features, wherein the matrix dimension of the initial dimensional image features is smaller than the matrix dimension of the initial image features. The spectral normalization layer is used to perform singular value decomposition on the initial dimension image features to determine the singular features corresponding to the initial dimension image features. The singular features are deconvolved using the spectral normalization layer to perform matrix recombination on the singular features, resulting in deconvolved image features. The deconvolutioned image features are subjected to dimensionality transformation processing to expand the number of channels of the deconvolutioned image features, resulting in target dimension image features. The matrix dimension of the target dimension image features is the same as the matrix dimension of the initial image features. A spectral normalization layer is used to calculate the variance and mean of the target dimension image features, and the target dimension image features are normalized based on the variance and mean to obtain the normalized target dimension image features.
2. The image feature processing method according to claim 1, characterized in that, The step of performing singular value decomposition on the initial dimension image features using the spectral normalization layer to determine the singular features corresponding to the initial dimension image features includes: The spectral normalization layer is used to perform exponential iteration on the initial dimension image features to determine the singular features corresponding to the initial dimension image features.
3. The image feature processing method according to claim 2, characterized in that, The step of using the spectral normalization layer to perform power-law iteration on the initial dimension image features to determine the singular features corresponding to the initial dimension image features includes: The spectral normalization layer is used to perform exponential iteration processing on the initial dimension image features; When the number of power iterations reaches a preset threshold, the singular features corresponding to the initial dimension image features are determined.
4. The image feature processing method according to claim 2, characterized in that, The step of using the spectral normalization layer to perform power-law iteration on the initial dimension image features to determine the singular features corresponding to the initial dimension image features includes: The initial singular features are obtained by performing power-law iteration on the initial dimensional image features using the spectral normalization layer. When the initial singular feature meets a preset threshold, the initial singular feature is determined to be the singular feature corresponding to the initial dimension image feature.
5. The image feature processing method according to claim 2, characterized in that, The step of using the spectral normalization layer to perform power-law iteration on the initial dimension image features to determine the singular features corresponding to the initial dimension image features includes: The spectral normalization layer is used to perform power-law iteration on the initial dimension image features to obtain the left singular vector and the right singular vector corresponding to the target singular value of the initial dimension image features; The left singular vector corresponding to the target singular value and the right singular vector corresponding to the target singular value are fused to obtain the singular features corresponding to the initial dimension image features.
6. An image feature processing apparatus, characterized in that, include: The acquisition unit is used to acquire the initial image features generated by the convolutional layers in the convolutional neural network through convolution of the image to be processed. The first transformation unit is used to convolve the initial image features using a spectral normalization layer to reduce the channel data of the initial image features and obtain convolved image features. The convolved image features are then subjected to dimensionality transformation processing to perform matrix recombination on the convolved image features and obtain initial dimensional image features. The matrix dimension of the initial dimensional image features is smaller than the matrix dimension of the initial image features. The decomposition unit is used to perform singular value decomposition on the initial dimension image features using the spectral normalization layer to determine the singular features corresponding to the initial dimension image features. The second transformation unit is used to deconvolve the singular features using the spectral normalization layer to perform matrix recombination on the singular features to obtain deconvoluted image features, and to perform dimensional transformation processing on the deconvoluted image features to expand the number of channels of the deconvoluted image features to obtain target dimension image features, wherein the matrix dimension of the target dimension image features is the same as the matrix dimension of the initial image features. The image feature processing device is also used to calculate the variance and mean of the target dimension image features using a spectral normalization layer, and to perform normalization processing on the target dimension image features based on the variance and mean to obtain normalized target dimension image features.
7. A computer device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the image feature processing method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the image feature processing method according to any one of claims 1 to 5.