Image processing apparatus and electronic device

Abstract

Embodiments of the present application provide an image processing device and electronic equipment, wherein the device comprises: an image compression network, the image compression network comprising a plurality of convolution layers, each convolution layer comprising: a complexity determination module configured to obtain an image feature vector and determine a complexity coefficient of the image feature vector, determine a target network compression module according to the complexity coefficient, in a case where the complexity coefficient is greater than a preset threshold, the target network compression module being a compression module of a complex network, otherwise, the target network compression module being a compression module of a simple network; the compression module of the complex network or the compression module of the simple network being configured to calculate a received image feature vector to obtain a compressed image feature vector. Through the present application, the problem of image compression without loss of information is solved, and the effect of adaptively selecting whether to use a compression module of a complex network or a compression module of a simple network to compress an image is achieved.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to an image processing apparatus and an electronic device. Background Technology

[0002] Model design plays a crucial role in many computer vision tasks. The growing demand for high-quality networks in mobile / embedded devices has driven research into efficient network design. For example, GoogleNet increases network depth with lower complexity compared to simply stacking convolutional layers; SqueezeNet uses a bottleneck approach to design a very small network; and MobileNet, for instance, uses separable convolutions in the depth direction to reduce computation and model size. MobileNetV3's design is based on a combination of complementary search techniques.

[0003] Another trend in obtaining smaller networks is model compression. Factorization-based methods attempt to approximate the original convolution operation through tensor decomposition, thereby accelerating the convolution operation. Knowledge extraction-based methods learn a small network to mimic a larger teacher network. Pruning-based methods attempt to reduce computation by pruning redundant connections or convolution channels.

[0004] In computer vision and natural language processing (NLP) tasks, the generation of dynamic convolutional kernels is essential. In the field of computer vision, Klein in 2015 generated convolutional kernels directly through linear layers based on the feature maps of the first few layers.

[0005] Despite these efforts, we found redundancy still exists between convolutional kernels, leading to redundant computations. Dynamic convolution can reduce redundant computations, thus complementing those efficient networks. However, in convolutional networks, correlation kernels are used to extract features independent of noise. Therefore, traditional convolutional networks struggle to be compressed without loss of information. Summary of the Invention

[0006] This application provides an image processing apparatus and an electronic device to at least solve the problem in the related art that convolutional networks are difficult to compress without losing information.

[0007] According to one embodiment of this application, an image processing apparatus is provided, the apparatus comprising:

[0008] An image compression network is provided, comprising multiple convolutional layers. Each convolutional layer includes a complexity determination module, a complex network compression module, and a simple network compression module. The complexity determination module is used to acquire an image feature vector and determine its complexity coefficient. Based on the complexity coefficient, it determines a target network compression module. If the complexity coefficient is greater than a preset threshold, the target network compression module is a complex network compression module; otherwise, it is a simple network compression module. The complex network compression module or the simple network compression module receives the image feature vector input from the complexity determination module and calculates the image feature vector to obtain a compressed image feature vector.

[0009] In an exemplary embodiment, the complexity determination module includes: a fully connected submodule, used to obtain the image feature vector and map the image feature vector onto two elements using [x1, x2] = W × input to obtain a one-dimensional vector, and use the one-dimensional vector as the initial complexity coefficient, where x1 and x2 represent two elements, W represents the parameter matrix of the fully connected submodule, and input represents the image feature vector.

[0010] In an exemplary embodiment, the complexity determination module includes a complexity coefficient determination submodule, which is configured to: select one of two elements in the initial complexity coefficient and compare it with the preset threshold; if the first element in the initial complexity coefficient is greater than the preset threshold, use the first element of the initial complexity coefficient as the complexity coefficient; if the first element in the initial complexity coefficient is less than or equal to the preset threshold, use the second element of the initial complexity coefficient as the complexity coefficient.

[0011] In an exemplary embodiment, the complexity determination module further includes a network compression selection submodule, which is configured to: compare the complexity coefficient with the preset threshold; if the complexity coefficient is greater than the preset threshold, select the compression module of the complex network as the target network compression module, otherwise select the compression module of the simple network as the target network compression module.

[0012] In one exemplary embodiment, the complexity determination module further includes an image gradient update submodule, which is used to obtain the gradient of the image feature vector and transmit the gradient to the corresponding target network compression module.

[0013] In an exemplary embodiment, the target network compression module is used to calculate an updated gradient based on the gradient and the complexity coefficient, wherein the updated gradient is used to determine the compressed image feature vector.

[0014] In one exemplary embodiment, the image processing apparatus further includes: a classification prediction network, configured to receive the compressed image feature vector and perform classification prediction based on the compressed image feature vector.

[0015] In an exemplary embodiment, the complexity determination module further includes: a perception layer output submodule, configured to receive the compressed image feature vector and output it to the next convolutional layer or the classification prediction network. Specifically, when the perception layer output submodule is located in any of the multiple convolutional layers except the last one, after receiving the compressed image feature vector, it transmits it to the next convolutional layer. When the perception layer output submodule is located in the last layer of the multiple convolutional layers, after receiving the compressed image feature vector, it transmits it to the classification prediction network.

[0016] In one exemplary embodiment, the apparatus further includes: an image input module, configured to receive the input image feature vector and input the image feature vector into the image compression network.

[0017] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0018] Through this application, since the image compression network includes multiple convolutional layers, each convolutional layer includes a compression module for receiving the image feature vector input by the complexity determination module, calculating the image feature vector, and obtaining the compressed image feature vector, and also includes a complexity determination module for obtaining the image feature vector, determining the complexity coefficient of the image feature vector, and determining whether to use the complex network compression module or the simple network compression module to compress the image feature vector based on the complexity coefficient. Therefore, the problem of image compression without loss of information can be solved, achieving the effect of adaptively selecting whether to use the complex network compression module or the simple network compression module to compress the image. Attached Figure Description

[0019] Figure 1 This is a structural block diagram of an image compression network according to an embodiment of this application;

[0020] Figure 2This is a structural block diagram of another image compression network according to an embodiment of this application;

[0021] Figure 3 This is a structural diagram of the complexity determination module according to an embodiment of this application;

[0022] Figure 4 This is a flowchart of an execution scheme for an image processing apparatus according to Embodiment 1 of this application;

[0023] Figure 5 This is a flowchart of an execution scheme for an image processing apparatus according to Embodiment 2 of this application;

[0024] Figure 6 This is a structural block diagram of an image processing apparatus according to Embodiment 3 of this application;

[0025] Figure 7 This is a flowchart of an execution scheme for an image processing apparatus according to Embodiment 3 of this application;

[0026] Figure 8 This is a flowchart of an execution scheme of an image processing apparatus according to Embodiment 4 of this application;

[0027] Figure 9 This is a hardware structure block diagram of a mobile terminal according to an embodiment of the present application of an image processing method. Detailed Implementation

[0028] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0030] This embodiment also provides an image processing apparatus, where the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0031] This embodiment provides an image processing device. Figure 1 This is a structural block diagram of an image processing apparatus according to an embodiment of this application, such as... Figure 1 As shown, the device includes:

[0032] The image compression network comprises multiple convolutional layers. Each convolutional layer includes a complexity determination module, a complex network compression module, and a simple network compression module. The complexity determination module acquires image feature vectors and determines their complexity coefficients. Based on these coefficients, it identifies the target network compression module. If the complexity coefficient exceeds a preset threshold, the target network compression module is a complex network compression module; otherwise, it is a simple network compression module. The complex network compression module or the simple network compression module receives the image feature vectors input from the complexity determination module, calculates the image feature vectors, and obtains the compressed image feature vectors.

[0033] Specifically, the image compression network described above includes multiple convolutional layers, which are used to process complex image features. A robust deep learning method with multiple convolutional layers is used to reduce the number of trainable parameters in the network by mining spatial correlations in the data, thereby improving the efficiency of the backpropagation algorithm of the forward propagation network. Different compression modules of the processing network can be selected for different image sizes based on the complexity coefficient, solving the problem that existing networks compress all input images, regardless of size, into a single small model, resulting in information loss. By using the compression modules of the aforementioned different networks, the problem of no information loss in image processing is ensured.

[0034] In an exemplary embodiment, the complexity determination module includes a fully connected submodule for acquiring the image feature vector and mapping the image feature vector to two elements using [x1, x2] = W × input to obtain a one-dimensional vector, which is used as the initial complexity coefficient, where x1 and x2 represent two elements, W represents the parameter matrix of the fully connected submodule, and input represents the image feature vector.

[0035] Specifically, the aforementioned complexity determination module is a module that predicts complexity coefficients based on the image feature vector output by the previous layer network. It includes a fully connected layer, which is a fully connected sub-module. The fully connected layer treats the image feature vector output by the previous layer as a one-dimensional vector. Through the fully connected network, it maps the one-dimensional image feature vector to a one-dimensional vector [x1, x2] with two elements, and uses [x1, x2] as the initial complexity coefficient. The specific mapping algorithm is: [x1, x2] = fc_layer(input) = W × input, where fc_layer() represents the fully connected layer, and input represents the image feature vector. By mapping the image feature vector to a one-dimensional vector with two elements as the elements of the initial complexity coefficient, it is convenient to determine the complexity coefficient and select the corresponding compression module of the network.

[0036] In an exemplary embodiment, the complexity determination module includes a complexity coefficient determination submodule, which is used to: select one of the two elements in the initial complexity coefficient and compare it with a preset threshold; if the first element in the initial complexity coefficient is greater than the preset threshold, use the first element of the initial complexity coefficient as the complexity coefficient; if the first element in the initial complexity coefficient is less than or equal to the preset threshold, use the second element of the initial complexity coefficient as the complexity coefficient.

[0037] Specifically, the complexity coefficient determination submodule determines the complexity coefficient based on the initial complexity coefficient, where the initial complexity coefficient is [x1, x2]. The complexity coefficient is determined by selecting the top 1 coefficient using the top 1 selection operator; the operation of the top 1 selection operator is as follows:

[0038] if (x1>n):

[0039] w1= x1

[0040] w2 = 0

[0041] else:

[0042] w1 = 0

[0043] w2 = x2;

[0044] Where n is the set complexity coefficient condition, [w 1， [w2] represents the two elements of the complexity coefficient, such as Figure 2 As shown, the two elements of the complexity coefficient correspond to the complexity network compression module and the simple network compression module, respectively. By setting the two elements of the complexity coefficient to zero, the corresponding processing network can be selected, simplifying the processing method and reducing computation. The execution flow of the complexity coefficient determination submodule is as follows: Figure 3 As shown.

[0045] In an exemplary embodiment, the complexity determination module further includes a network compression selection submodule, which is used to: compare the complexity coefficient with a preset threshold; if the complexity coefficient is greater than the preset threshold, select the compression module of the complex network as the target network compression module, otherwise select the compression module of the simple network as the target network compression module.

[0046] Specifically, the complexity coefficient mentioned above is a one-dimensional vector of two elements. Therefore, when comparing the complexity coefficient with a preset threshold, one of the elements is selected as the basis. That is, the first element of the complexity coefficient corresponds to the compression module of the complex network, and the second element corresponds to the compression module of the simple network. The top1 selection operator is used to select either the compression module of the complex network or the compression module of the simple network. Here, the two elements of the complexity coefficient are used to select the compression modules of the two networks respectively, w1 is used to select the compression module of the complex network, and w2 is used to select the compression module of the simple network. The operation of the top1 selection operator is as follows:

[0047] if (x1>n):

[0048] w1= x1

[0049] w2 = 0

[0050] else:

[0051] w1 = 0

[0052] w2 = x2;

[0053] Where n is a set complexity coefficient condition, such as Figure 3 As shown, when w1 = x1, w2 is set to zero, indicating that the compression module of the complex network is selected to compress the image. When w1 = 0, w2 = x2 indicates that the compression module of the simple network is selected to compress the image. When the model is used on mobile devices, w1 can be forced to be set to 0, that is, only the compression module of the simple network is used to process the image, which significantly reduces the amount of computation to adapt to the limitations of low computing power on mobile devices.

[0054] In an exemplary embodiment, the complexity determination module further includes an image gradient update submodule, which is used to obtain the gradient of the image feature vector and transmit the gradient to the corresponding target network compression module. The gradient is calculated through backpropagation to perform complex image processing on the complex network compression module, which has a fast processing speed and reduces the amount of computation.

[0055] In an exemplary embodiment, the target network compression module described above is used to update the gradient based on the gradient and the complexity coefficient, wherein updating the gradient is used to determine the compressed image feature vector.

[0056] Specifically, the aforementioned target network compression module includes compression modules for simple networks and compression modules for complex networks, such as... Figure 1As shown, when the compression module for complex networks is selected, input1 = input × w1 is transmitted to the compression module for complex networks and used for gradient calculation. When the compression module for simple networks is selected, input2 = input × w2 is transmitted to the compression module for complex networks and used for gradient calculation. The specific gradient calculation adopts existing technical means, such as gradient backpropagation. The specific calculation process adopts existing technology and will not be described in detail here.

[0057] In an exemplary embodiment, the complexity determination module further includes a perception layer output submodule, which receives the compressed image feature vector and outputs it to the next convolutional layer or the classification prediction network. When the perception layer output submodule is located in any layer of the multi-layer convolutional layers except the last layer, it transmits the compressed image feature vector to the next convolutional layer. When the perception layer output submodule is located in the last layer of the multi-layer convolutional layers, it transmits the compressed image feature vector to the classification prediction network.

[0058] Specifically, the aforementioned perception layer output submodule includes a multilayer perceptron (MLP); such as Figure 2 As shown, if the compression module of the selected complex network is used for compression, then output = MLP(output1×w1+outpu2×w2), where w1 is a non-zero coefficient and w2 is a zero-coefficient; if the compression module of the selected simple network is used for compression, then output = MLP(output1×w1+outpu2×w2), where w1 is a zero-coefficient and w2 is a non-zero coefficient; then the output result is output to the complexity determination module of the next convolutional layer.

[0059] In one exemplary embodiment, the image processing apparatus further includes an image input module for receiving an input image feature vector and inputting the image feature vector into an image compression network.

[0060] Specifically, the image input module described above is used to transmit the feature vector of the image acquired by the mobile terminal to a multi-layer convolutional layer for image compression processing. The image input module uses the existing matrix-to-vector method to convert the input image matrix into image feature vectors for subsequent image compression.

[0061] In one exemplary embodiment, the image processing apparatus further includes a classification prediction network for receiving compressed image feature vectors and performing classification prediction based on the compressed image feature vectors.

[0062] Specifically, the classification prediction network mentioned above uses any one of VGG Net, ResNet, ResNeXt, or SE-Net to classify the compressed image. This is a common technique in image processing and will not be described in detail here.

[0063] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0064] This embodiment provides a training algorithm for an image processing device, including training the proposed dynamic convolution. When the coefficients in the complexity prediction module are 0, the corresponding part of the network does not participate in forward computation and gradient backpropagation, thus reducing the computational load of the network. The coefficients in the neural network are learned and adjusted during training through the backpropagation mechanism of the neural network. Each element in the W matrix used for forward computation in the complexity prediction module is randomly initialized before training begins, ensuring that the predicted coefficients of x1 and x2 for any input in the initial training stage have no obvious bias and the prediction probabilities are not significantly different. Since the complex convolution network is only selected when the predicted coefficients are greater than 0.9... Therefore, during model training, a simple convolutional network is initially considered for output prediction. When the accuracy of prediction using a simple convolutional network is low, the model continuously updates its parameters. When the parameter update leads to the selection of a complex convolutional network for prediction, the accuracy of the network's prediction is improved, and the model continuously strengthens the parameter updates in that direction. Thus, in simple tasks, the model, after training, will learn prediction coefficients that tend to use a simple convolutional network, while in complex input tasks, the model can make prediction coefficients tend to use a complex convolutional network. Through this mechanism, the model can automatically learn to use a complex convolutional neural network for complex tasks, resulting in prediction coefficients W1 exceeding a set threshold; while for simple tasks, a simple convolutional neural network is used, i.e., prediction coefficients W1 are below the set threshold. Example 1

[0065] This application also provides an execution scheme for an image processing apparatus, such as... Figure 4 As shown, it includes the following steps:

[0066] Step S101: Obtain the image feature vector. The image feature vector is a multi-dimensional feature vector. The multi-dimensional vector is transformed into a one-dimensional vector using the reshape algorithm in Python. The one-dimensional vector includes two elements.

[0067] Step S102: Use the two-element one-dimensional vector as the complexity coefficient for selecting a complex network compression module and the complexity coefficient for selecting a simple network compression module.

[0068] Step S103: Set preset thresholds for the complexity coefficients of the compression modules of complex networks and simple networks. Compare the complexity coefficients of each module with the preset thresholds. If the complexity coefficient exceeds the threshold, the compression module of the complex network is selected to compress the image. If the complexity coefficient does not exceed the preset threshold, the compression module of the simple network is selected to compress the image. Example 2

[0069] This application also provides an execution scheme for an image processing apparatus, such as... Figure 5 As shown, it includes the following steps:

[0070] Step S201: Obtain the image feature vector. The image feature vector is a multi-dimensional feature vector. The multi-dimensional vector is transformed into a one-dimensional vector using the reshape algorithm in Python. The one-dimensional vector includes two elements.

[0071] Step S202: Use the sum of the two elements as the complexity coefficient;

[0072] Step S203: Set a preset threshold for selecting the compression module of the complex network and the compression module of the simple network. By comparing the complexity coefficient with the preset threshold, if the complexity coefficient exceeds the threshold, the compression module of the complex network is selected to compress the image; if it does not exceed the preset threshold, the compression module of the simple network is selected to compress the image. Example 3

[0073] This application also provides an execution scheme for an image processing apparatus, such as... Figure 6 As shown, the specific execution scheme of this architecture is as follows: Figure 7 As shown, it includes the following steps:

[0074] Step S301: The image feature vector is obtained through the image input module and transmitted to the image compression network, which includes multiple convolutional layers. The complexity determination module of the first convolutional layer of the image compression network uses [x1, x2] = W × input to map the image feature vector onto two elements to obtain a one-dimensional vector. This one-dimensional vector is used as the initial complexity coefficient, where x1 and x2 represent two elements, W represents the parameter matrix of the fully connected submodule, and input represents the image feature vector. One of the two elements in the initial complexity coefficient is selected and compared with a preset threshold. If the first element in the initial complexity coefficient is greater than the preset threshold, the first element of the initial complexity coefficient is used as the complexity coefficient. If the first element in the initial complexity coefficient is less than or equal to the preset threshold, the second element of the initial complexity coefficient is used as the complexity coefficient.

[0075] Step S302: The complexity determination module of each convolutional layer of the image compression network determines the target network compression module based on the above complexity coefficients. This includes: using the top-1 selection operator to select either a compression module for a complex network or a compression module for a simple network. Specifically, two elements of the complexity coefficient are used to select compression modules for the two networks as the target network compression module, w1 is used to select the compression module for the complex network as the target network compression module, and w2 is used to select the compression module for the simple network as the target network compression module. The operation of the top-1 selection operator is as follows:

[0076] if (x1>n):

[0077] w1= x1

[0078] w2 = 0

[0079] else:

[0080] w1 = 0

[0081] w2 = x2;

[0082] Here, n represents the set complexity coefficient. When w1 = x1, w2 is set to zero, indicating that the compression module of the complex network is selected as the target network compression module for image compression. When w1 = 0, w2 = x2, indicating that the compression module of the simple network is selected as the target network compression module for image compression. When using the model on mobile devices, w1 can be forced to be set to 0, that is, only the compression module of the simple network is used to process the image.

[0083] Step S303: The target network compression module of each convolutional layer calculates the feature vector of the received image to obtain the compressed image feature vector of each convolutional layer.

[0084] In step S304, the compressed image feature vector of each convolutional layer is used as the input receiver of the next layer by the output of the compression module of the complex network (output1) or the output of the compression module of the simple network (output2) of the convolutional layer. This process continues until the last convolutional layer is reached, at which point the final compressed image feature vector is obtained. The final image feature vector is then transmitted to the classification prediction network to classify the compressed image.

[0085] Specifically, in step S303 above, the target network compression module of each convolutional layer calculates the received image feature vector using existing image compression calculation methods, which will not be described in detail here. The target network compression module includes a simple network compression module and a complex network compression module. The simple network compression module uses a CNN convolutional network, consisting of three layers: a convolutional layer, a pooling layer, and a fully connected layer. This is existing technology, and the specific convolutional operations will not be described in detail here. The complex network compression module uses a CNN convolutional network, such as the LeNet5 network, which includes three or more layers: an input layer, multiple convolutional layers, multiple pooling layers, a fully connected layer, and an output layer. Figure 5 As shown, the LeNet5 network consists of a total of 8 layers, including the input layer: input layer (INPUT), convolutional layer (C1), pooling layer (S2), convolutional layer (C3), pooling layer (S4), convolutional layer (C5), fully connected layer (F6), and output layer (radial base layer). The specific principles are existing technologies and will not be described in detail here. Example 4

[0086] This application also provides an execution scheme for an image processing apparatus, such as... Figure 8 As shown, it includes the following steps:

[0087] Step S401: Obtain the image feature vector from the previous layer. The complexity determination module maps the image feature vector to two elements using [x1, x2] = W × input, resulting in a one-dimensional vector. This one-dimensional vector is used as the initial complexity coefficient, where x1 and x2 represent two elements, W represents the parameter matrix of the fully connected submodule, and input represents the image feature vector. One element of the initial complexity coefficient is compared with a preset threshold. If the first element of the initial complexity coefficient is greater than the preset threshold, the first element of the initial complexity coefficient is used as the complexity coefficient. If the first element of the initial complexity coefficient is less than or equal to the preset threshold, the second element of the initial complexity coefficient is used as the complexity coefficient.

[0088] Step S402, the complexity determination module determines the target network compression module based on the above complexity coefficients, including:

[0089] The top-1 selection operator is used to select either a compression module for a complex network or a compression module for a simple network. Two elements of the complexity coefficient are used to select the compression module for each network as the target network compression module. w1 is used to select the compression module for the complex network as the target network compression module, and w2 is used to select the compression module for the simple network as the target network compression module. The operation of the top-1 selection operator is as follows:

[0090] if (x1>n):

[0091] w1= x1

[0092] w2 = 0

[0093] else:

[0094] w1 = 0

[0095] w2 = x2;

[0096] Where n is the set complexity coefficient condition. When w1 = x1, w2 is set to zero, which means that the compression module of the complex network is selected as the target network compression module to compress the image. When w1 = 0, w2 = x2 means that the compression module of the simple network is selected as the target network compression module to compress the image.

[0097] Step S403: Calculate the received image feature vector using the selected complex network compression module or simple network compression module to obtain the compressed image feature vector.

[0098] In step S404, the compressed image feature vector is output to the perceptual layer output submodule of the complexity determination module through the output terminals of the compression module of the complex network (output1) and the simple network (output2). The input terminal of the perceptual layer output submodule is a fully connected layer, which receives the complexity coefficients of the compression module of the complex network and the compression module of the simple network and sums them. The sum is then output to the next convolutional layer through the output terminal of the perceptual layer output submodule. Here, output = MLP(output1*w1 + output2*w2), and MLP represents the fully connected layer of the perceptual layer output submodule.

[0099] In step 3 of the two embodiments above, the compression module of the complex network or the compression module of the simple network calculates the image feature vector using existing image compression calculation methods, which will not be described in detail here. The compression module of the simple network uses a CNN convolutional network, which includes three layers: a convolutional layer, a pooling layer, and a fully connected layer. This is existing technology, and the specific convolutional operations will not be described in detail here. The compression module of the complex network uses a CNN convolutional network, such as the LeNet5 network, which includes three or more layers: an input layer, multiple convolutional layers, multiple pooling layers, a fully connected layer, and an output layer. The LeNet5 network includes a total of 8 layers, including the input layer: an input layer (INPUT), a convolutional layer (C1), a pooling layer (S2), a convolutional layer (C3), a pooling layer (S4), a convolutional layer (C5), a fully connected layer (F6), and an output layer (radial base layer). The specific principles are existing technology and will not be described in detail here.

[0100] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 9 This is a hardware structure block diagram of a mobile terminal for an image processing method according to an embodiment of this application. For example... Figure 9 As shown, a mobile terminal may include one or more ( Figure 9 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 9 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown.

[0101] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to an image processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0102] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0103] Through the above embodiments, the problem of compressing images without losing information can be solved by adaptively selecting whether to use the complex network compression module or the simple network compression module, thereby improving the effect and speed of image compression.

[0104] The entities that perform the above steps can be servers, terminals, etc., but are not limited to these.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0106] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when it is run.

[0107] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0108] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0109] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0110] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0111] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0112] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. An image processing apparatus, characterized in that, This includes an image compression network, which comprises multiple convolutional layers. Each convolutional layer includes: a complexity determination module, a compression module for complex networks, and a compression module for simple networks. The complexity determination module is used to acquire image feature vectors and determine the complexity coefficient of the image feature vectors, and determine the target network compression module based on the complexity coefficient. If the complexity coefficient is greater than a preset threshold, the target network compression module is a compression module for a complex network; otherwise, the target network compression module is a compression module for a simple network. The compression module of the complex network or the compression module of the simple network is used to receive the image feature vector input by the complexity determination module, and calculate the image feature vector to obtain the compressed image feature vector. The fully connected layer is a fully connected submodule. It treats the image feature vector output from the previous layer as a one-dimensional vector and maps it to a two-element one-dimensional vector [x1, x2] using the fully connected network. [x1, x2] is used as the initial complexity coefficient. The specific mapping algorithm is: [x1, x2] = fc_layer(input) = W × input, where fc_layer() represents the fully connected layer, and input represents the image feature vector. The image feature vector is mapped to a two-element one-dimensional vector, which is then used as the element of the initial complexity coefficient. The complexity determination module includes a complexity coefficient determination submodule. This submodule is used to: select one of the two elements in the initial complexity coefficient and compare it with a preset threshold; if the first element of the initial complexity coefficient is greater than the preset threshold, the first element of the initial complexity coefficient is used as the complexity coefficient; if the first element of the initial complexity coefficient is less than or equal to the preset threshold, the second element of the initial complexity coefficient is used as the complexity coefficient.

2. The apparatus according to claim 1, characterized in that, The complexity determination module further includes a network compression selection submodule, which is used for: The complexity coefficient is compared with the preset threshold; If the complexity coefficient is greater than the preset threshold, then the compression module of the complex network is selected as the target network compression module; otherwise, the compression module of the simple network is selected as the target network compression module.

3. The apparatus according to claim 1, characterized in that, The complexity determination module further includes an image gradient update submodule, which is used to obtain the gradient of the image feature vector and transmit the gradient to the corresponding target network compression module.

4. The apparatus according to claim 3, characterized in that, The target network compression module is used to calculate and update the gradient based on the gradient and the complexity coefficient, wherein the updated gradient is used to determine the compressed image feature vector.

5. The apparatus according to claim 1, characterized in that, The image processing apparatus further includes a classification prediction network, used to receive the compressed image feature vector and perform classification prediction based on the compressed image feature vector.

6. The apparatus according to claim 5, characterized in that, The complexity determination module also includes: The perceptual layer output submodule is used to receive the compressed image feature vector and output it to the next convolutional layer or the classification prediction network. Specifically, when the perceptual layer output submodule is located in any of the multiple convolutional layers except the last one, it transmits the compressed image feature vector to the next convolutional layer after receiving it. When the perceptual layer output submodule is located in the last layer of the multiple convolutional layers, it transmits the compressed image feature vector to the classification prediction network after receiving it.

7. The apparatus according to any one of claims 1 to 6, characterized in that, Also includes: An image input module is used to receive the input image feature vector and input the image feature vector into the image compression network.

8. An electronic device, characterized in that, Includes the apparatus described in any one of claims 1 to 7.

Images