Image processing method, device, equipment and medium

Abstract

The embodiments of the present invention provide an image processing method, apparatus, device and medium, aiming to improve the accuracy of image recognition, the method comprising: obtaining an image to be processed; inputting the image to be processed into a target network to obtain a processing result output by the target network; wherein the activation function of at least one layer in the target network is a differentiable activation function, and the differentiable activation function is obtained by training an initial differentiable activation function; the processing result is a classification result of the image to be processed or a recognition result of an object contained in the image to be processed.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, equipment and medium. Background Art

[0002] With the development of artificial intelligence, the use of neural network models to recognize objects has become more and more widely used. Generally speaking, there are many neural network models used for object recognition. For example, there are ResNet models and mobileNet models in related technologies. These neural network models can achieve better object recognition results.

[0003] However, with the in-depth practice of using these neural network models for object recognition, the applicant found that the activation functions used by these neural network models will have bottlenecks in the training process. For example, in the process of training the neural network model, some parameters of the neural network model stop updating, resulting in insufficient training of the neural network model. If the trained neural network model is used for object recognition, the recognition accuracy is not high. Summary of the invention

[0004] In view of the above problems, an image processing method, apparatus, device and medium according to an embodiment of the present invention are proposed to overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problem, the first aspect of the present invention discloses an image processing method, the method comprising:

[0006] Obtaining an image to be processed;

[0007] Inputting the image to be processed into a target network to obtain a processing result output by the target network;

[0008] Wherein, the activation function of at least one layer in the target network is a differentiable activation function, and the differentiable activation function is obtained by training the initial differentiable activation function;

[0009] The processing result is a classification result of the image to be processed or a recognition result of an object contained in the image to be processed.

[0010] Optionally, the initial activation function includes learnable parameters.

[0011] Optionally, the training steps of the target network are as follows:

[0012] Inputting a sample image into a convolutional neural network to obtain a processing result output by the convolutional neural network, wherein an activation function of at least one layer in the convolutional neural network is the initial differentiable activation function, and parameters of the convolutional neural network include learnable parameters;

[0013] According to the residual between the processing result output by the convolutional neural network and the standard processing result, the parameters of the convolutional neural network are updated until the preset training end condition is met to obtain the target network, and the standard processing result is the classification label pre-annotated for the sample image or the label of the object contained in the sample image.

[0014] Optionally, the learnable parameters in the initial differentiable activation function include a first parameter that makes the upper and lower limits of the first-order derivative of the initial differentiable activation function adjustable, and / or a second parameter for adjusting the degree of smoothness and differentiability of the differentiable activation function.

[0015] Optionally, the second parameter is at a pixel level, and the values ​​of the second parameter corresponding to different pixels are independent of each other; or

[0016] The second parameter is at a channel level, and each pixel point in the same channel shares the same value of the second parameter; or

[0017] The second parameter is hierarchical, and each pixel point of each channel of the same layer shares the same value of the second parameter.

[0018] Optionally, the initial differentiable activation function is:

[0019] S β (p 1 x,p 2 x)=(p 1 -p 2 )x·σ[β(p 1 -p 2 )x]+p 2 x;

[0020] Among them, p 1 is a parameter that makes the upper limit of the first-order derivative of the initial differentiable activation function adjustable, p 2 is a parameter that makes the lower limit of the first-order derivative of the initial differentiable activation function adjustable, β is a parameter used to adjust the smoothness and differentiability of the initial differentiable activation function, σ is the Sigmoid function, and x represents the input.

[0021] Optionally, the convolutional neural network is constructed by the following steps:

[0022] Determine at least one original activation function layer from the original neural network;

[0023] The at least one original activation function layer is replaced with a differentiable activation function layer to obtain the convolutional neural network, wherein the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0024] Optionally, the convolutional neural network is constructed by the following steps:

[0025] Determine a plurality of original convolutional layers to be processed from the original neural network, wherein an original activation function layer is configured between every two adjacent original convolutional layers to be processed in the plurality of original convolutional layers to be processed;

[0026] The original activation function layer configured between one or more groups of two adjacent original convolutional layers to be processed is replaced with a differentiable activation function layer to obtain the convolutional neural network, wherein the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0027] Optionally, the convolutional neural network is constructed by the following steps:

[0028] Determine a feature processing layer to be processed from the original neural network, wherein the output end of the feature processing layer is connected to the input end of the original activation function layer, and the feature processing layer is a residual layer or a feature concatenation layer;

[0029] The original activation function layer connected to the output end of the feature processing layer is replaced with the differentiable activation function layer to obtain the convolutional neural network, and the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0030] According to a second aspect of the embodiments of the present invention, there is provided an image processing device, the device comprising:

[0031] An image acquisition module, used for acquiring an image to be processed;

[0032] An image input module, used to input the image to be processed into a target network to obtain a processing result output by the target network;

[0033] Wherein, the activation function of at least one layer in the target network is a differentiable activation function, and the differentiable activation function is obtained by training the initial differentiable activation function;

[0034] The processing result is a classification result of the image to be processed or a recognition result of an object contained in the image to be processed.

[0035] According to a third aspect of the embodiments of the present invention, an electronic device is disclosed, including:

[0036] one or more processors; and

[0037] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the device to perform the image processing method as described in the embodiment of the first aspect of the present invention.

[0038] According to a fourth aspect of the embodiments of the present invention, a computer-readable storage medium is disclosed, wherein a computer program stored in the medium enables a processor to execute the image processing method as described in the embodiment of the first aspect of the present invention.

[0039] In an embodiment of the present invention, an image to be processed can be obtained, and the image to be processed can be input into a target network to obtain a processing result output by the target network; wherein the activation function of at least one layer in the target network is a differentiable activation function; and the processing result is a classification result of the image to be processed or a recognition result of an object contained in the image to be processed.

[0040] Compared with the prior art, the embodiments of the present invention have at least the following advantages:

[0041] Some points in the existing activation functions are not differentiable (for example, in activation functions such as relu, the intersection of y=x and y=0 is not differentiable). The gradients (i.e., derivatives) of these non-differentiable points cannot be returned during the back propagation process, and only approximate gradients can be taken. However, the activation function of the embodiment of the present invention is differentiable, that is, each point of the activation function is differentiable, so the exact value of the gradient of each point can be returned during the back propagation process. In this way, the training effect can be improved, so that the target network has a higher accuracy rate when performing object recognition, thereby significantly improving the accuracy of the neural network model when performing object recognition, and achieving a higher accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0043] Figure 1 In the embodiment of the present invention, S β Nonlinear curve graphs of (a, b);

[0044] Figure 2 is a nonlinear curve graph of a differentiable activation function in an embodiment of the present invention;

[0045] Figure 3A It is a schematic diagram of a construction method of a convolutional neural network in the implementation of the present invention;

[0046] Figure 3B It is a schematic diagram of another method for constructing a convolutional neural network in the implementation of the present invention;

[0047] Figure 3C It is a schematic diagram of another method of constructing a convolutional neural network in the implementation of the present invention;

[0048] Figure 4 is a flowchart of steps of an image processing method in the implementation of the present invention;

[0049] Figure 5 It is a statistical graph comparing the recognition error rates of the original neural network and the target network in the implementation of the present invention;

[0050] Figure 6 It is a structural block diagram of an image processing device in the implementation of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] In the related art, when using the existing neural network model for object recognition, it generally adopts activation functions such as ReLU (Rectified Linear Unit, linear rectifier function). However, these activation functions are not differentiable everywhere, which causes some parameters of the neural network model to stop updating during the training process of the neural network model using these activation functions, resulting in insufficient training degree, which leads to low accuracy of object recognition by the obtained neural network model. In order to improve the recognition accuracy of the neural network model, the applicant has proposed the following technical concept: using a target network with a differentiable activation function for object recognition. In practice, the image to be processed can be input into the target network, and then the processing result output by the target network is obtained. In this way, since the activation function in the target network is a differentiable activation function, the parameters of the network will be continuously updated during the training process of the target network, and the training degree will be strengthened, so that the object recognition function of the obtained target network is enhanced, and the recognition accuracy is greatly improved.

[0053] In this embodiment, the target network is obtained by training a convolutional neural network through training samples. In this way, in the process of constructing the target network, it is necessary to first obtain the convolutional neural network, and then obtain the target network by training the convolutional neural network.

[0054] Among them, the convolutional neural network has an initial differentiable activation function. To facilitate understanding of the present invention, the initial differentiable activation function of the present application is first introduced. The initial differentiable activation function can be shown as the following formula:

[0055] Sβ (p 1 x,p 2 x)=(p 1 -p 2 )x·σ[β(p 1 -p 2 )x]+p 2 x;

[0056] Among them, p 1 is a parameter that makes the upper limit of the first-order derivative of the initial differentiable activation function adjustable, p 2 is a parameter that makes the lower limit of the first-order derivative of the initial differentiable activation function adjustable, β is a parameter used to adjust the smoothness and differentiability of the initial differentiable activation function, σ is the Sigmoid function, and x represents the input.

[0057] Among them, p 1 It can be understood as controlling the upper limit of the gradient of the convolutional neural network during training. 2 It can be understood as controlling the lower limit of the gradient of the convolutional neural network during training. Both can change and update as the training progresses, so that when the target network is obtained, the trained differentiable activation function is also obtained. In this way, the trained differentiable activation function is also differentiable. As a result, the application can effectively perform gradient backpropagation in the process of training the target network, and the upper and lower limits of the gradient can be dynamically adjusted.

[0058] Among them, the process of obtaining the above initial differentiable activation function can be explained as follows:

[0059] First, we can use a common α-softmax formula to obtain the smoothed form S of the max(a,b) function: β (a,b)), which is derived as follows:

[0060]

[0061] Among them, β is used to control the degree of smoothness. When β approaches positive infinity, S β (a,b) is max(a,b). The rest are smooth max, and the resulting nonlinear curve is Figure 1 shown.

[0062] Next, let a = p 1 x,b=p 2 x, then we can get the above differentiable activation function:

[0063] S β (p 1 x,p 2 x)=(p 1 -p 2)x·σ[β(p 1 -p 2 )x]+p 2 x.

[0064] After differentiating the differentiable activation function, the resulting nonlinear distribution is as follows Figure 2 As shown, it can be seen that the upper and lower limits of the first-order derivative are adjustable and smoother through the initial differentiable activation function. In this way, in the process of training the target network, the learnable parameters in the initial differentiable activation function are fixed. After fixation, β is not 0 or positive infinity. In this way, the differentiable activation function obtained after training is also differentiable.

[0065] Next, the process of how to construct a convolutional neural network is explained. According to the number of layers where the initial differentiable activation function is located, the convolutional neural network can be constructed in the following way.

[0066] Determine at least one original activation function layer from the original neural network; and replace the at least one original activation function layer with a differentiable activation function layer to obtain the convolutional neural network, wherein the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0067] In this embodiment, the original neural network can be any convolutional neural network, for example, a neural network with a ShuffleNetV2 structure, a neural network with a Mobile Net structure, a neural network with a Mobile NetV2 structure, a ResNet neural network, etc. Among them, the original activation function layer includes an original activation function, which can be a ReLU activation function, a Sigmoid function, or a tanh (hyperbolic tangent function) function.

[0068] Among them, an original neural network may include multiple original activation function layers. In an embodiment of the present application, one of the multiple original activation function layers can be replaced by an initial differentiable activation function layer, or all or part of the multiple original activation function layers can be replaced by an initial differentiable activation function layer. The specific replaceable layers can be determined based on experience or actual needs.

[0069] In this embodiment, the ReLU activation function in any original activation function layer in the original neural network can be replaced with a differentiable activation function. Figure 3A As shown in FIG, a schematic diagram of building a convolutional neural network is shown, taking the original neural network as the ResNet network as an example. Figure 3AAs shown, for example, the ReLU activation function in the original activation function layer connected to the output of the convolution layer "1×1,64" can be replaced with a differentiable activation function, thereby replacing the original activation function layer with a differentiable activation function layer. Of course, the ReLU activation function in the original activation function layer connected to the output of the convolution layer "3×3,64" can also be replaced with the initial differentiable activation function at the same time, thereby achieving the replacement of the two original activation function layers.

[0070] Optionally, in one example, the replaced original activation function layer may refer to the original activation function layer configured between every two adjacent original convolutional layers to be processed in the original neural network.

[0071] Accordingly, when constructing a convolutional neural network, a plurality of original convolutional layers to be processed can be determined from the original neural network, wherein an original activation function layer is configured between each two adjacent original convolutional layers to be processed in the plurality of original convolutional layers to be processed; the original activation function layer configured between one or more groups of adjacent original convolutional layers to be processed is replaced with a differentiable activation function layer to obtain the convolutional neural network, wherein the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0072] Among them, an original convolution layer to be processed can be understood as a layer for performing a single convolution process on an image. In this embodiment, the original neural network generally includes multiple original convolution layers, and an original activation function layer can be configured between each two adjacent original convolution layers, that is, an original activation function layer is configured between the output end of the current original convolution layer and the input end of the next adjacent original convolution layer, and the output of the current original convolution layer is input to the next adjacent original convolution layer after the original activation function layer.

[0073] Thus, in this embodiment, the original activation function layer configured between one or more groups of two adjacent original convolutional layers to be processed can be replaced with a differentiable activation function layer.

[0074] Reference Figure 3B As shown in FIG. 1 , a schematic diagram of target network construction is shown in this example, taking the original neural network as the ResNet network as an example. Figure 3B As shown, the convolution layer "1×1,64" and the convolution layer "3×3,64" are two adjacent convolution layers, and the original activation function layer is configured between the two, wherein the original activation function layer connected between "1×1,64" and the convolution layer "3×3,64" can be replaced by a differentiable activation function layer.

[0075] Alternatively, in one example, the replaced original activation function layer may also refer to the original activation function layer connected to the output end of the feature processing layer in the original neural network.

[0076] Accordingly, the feature processing layer to be processed can be determined from the original neural network, the output end of the feature processing layer is connected to the input end of the original activation function layer, and the feature processing layer is a residual layer or a feature concatenation layer; the original activation function layer connected to the output end of the feature processing layer is replaced with the differentiable activation function layer to obtain the convolutional neural network, and the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0077] Among them, the feature processing layer can have different types according to the type of the original neural network. For example, when the original neural network is a ResNet network, the feature processing layer is a residual layer, which is used to fuse multiple input features and original features. When the original neural network is a MasNet neural network, the feature processing layer is a feature splicing layer, which is used to splice multiple input features.

[0078] In this embodiment, the original activation function layer connected to the output end of each feature processing layer in the original neural network can be replaced with a differentiable activation function layer, that is, the original activation function in the original activation function layer is replaced with an initial differentiable activation function.

[0079] Reference Figure 3C As shown in FIG. 1 , a schematic diagram of target network construction is shown in this example, taking the original neural network as the ResNet network as an example. Figure 3C As shown, the feature processing layer is shown by the "+" sign in the figure, specifically a residual layer, which is used to fuse the input to the convolution layer "1×1,64" and the output of the convolution layer "1×1,256". An original activation function layer is configured after the feature processing layer. The original function in the original activation function layer is a ReLU activation function, and the ReLU activation function can be replaced by the initial differentiable activation function described above in this embodiment.

[0080] It should be noted that, in the specific implementation, while replacing the original activation function layer connected to the output end of each feature processing layer in the original neural network with a differentiable activation function layer, the original activation function layer configured between each two adjacent original convolutional layers to be processed in the original neural network can also be replaced with a differentiable activation function layer, thereby achieving the replacement of all the original activation function layers in the original neural network.

[0081] In this embodiment, after a convolutional neural network is constructed by any one or more combinations, the convolutional neural network can be trained using training samples. In order to dynamically and adaptively control the upper and lower limits of the gradient, the initial differentiable activation function in this embodiment includes learnable parameters, such as p in the initial differentiable activation function. 1 and p 2; Wherein, the training steps of the target network can be as follows:

[0082] First, a sample image is input into a convolutional neural network to obtain a processing result output by the convolutional neural network, wherein the activation function of at least one layer in the convolutional neural network is an initial differentiable activation function, and the parameters of the convolutional neural network include learnable parameters.

[0083] Secondly, according to the residual between the processing result output by the convolutional neural network and the standard processing result, the parameters of the convolutional neural network are updated until the preset training end conditions are met to obtain the target network. The standard processing result is a classification label pre-annotated for the sample image or a label of an object contained in the sample image.

[0084] The preset training end condition may be that the residual meets the requirement or that the parameter is updated enough times. The learnable parameters can be understood as the parameters that can control the upper and lower limits of the gradient during the training of the convolutional neural network, which can be continuously updated during the training process.

[0085] In this embodiment, sample images can be collected according to actual object recognition needs. For example, if object recognition is face recognition, the sample images can be images collected of faces. For another example, if object recognition is animal classification recognition, the sample images can be images collected of various animals.

[0086] The standard processing result can refer to the correct recognition result of the input sample image, or it can be understood as the real result. For example, if the object recognition is animal classification recognition, the standard processing result can be the correct category to which the animal in the sample image actually belongs, and the processing result output by the convolutional neural network can be understood as: the predicted result of the recognition of the sample image.

[0087] The residual between the output processing result and the standard processing result may refer to the difference between the processing result and the standard processing result, and the loss of the convolutional neural network may be obtained according to the difference, and the parameters of the convolutional neural network may be updated according to the loss. The parameters of the convolutional neural network may include learnable parameters in the initial differentiable activation function.

[0088] In an optional example, the learnable parameters include a first parameter that makes the upper and lower limits of the first-order derivative of the initial differentiable activation function adjustable, and / or a second parameter for adjusting the degree of smoothness and differentiability of the initial differentiable activation function.

[0089] Among them, the first parameter can control the upper and lower limits of the first-order derivative of the initial differentiable activation function, including parameters that can control the upper limit of the first-order derivative of the initial differentiable activation function, such as p in the above initial differentiable activation function 1 , as well as parameters that can control the lower limit of the first-order derivative of the initial differentiable activation function, such as p in the above initial differentiable activation function 2 .

[0090] In another optional example, the second parameter is at the pixel level, and the values ​​of the second parameter corresponding to different pixels are independent of each other; or the second parameter is at the channel level, and each pixel point in the same channel shares the same value of the second parameter; or the second parameter is at the layer level, and each pixel point in each channel of the same layer shares the same value of the second parameter.

[0091] When the second parameter is a pixel-level parameter, the values ​​of the second parameters of different pixels are independent of each other. In this case, the values ​​of the second parameters of different pixels can be the same or different. Since the second parameter is a pixel-level parameter, the smoothness and differentiability of the initial differentiable activation function can be more finely controlled.

[0092] When the second parameter is a channel-level parameter, the values ​​of the second parameters of different channels are independent of each other, but the pixels of the same channel share the same value of the second parameter. In this case, since the second parameter is a channel-level parameter, the smoothness and differentiability of the initial differentiable activation function can be controlled regionally.

[0093] When the second parameter is a parameter of the hierarchy, the values ​​of the second parameters of different hierarchical levels are independent of each other. In this case, the values ​​of the second parameters of each pixel in the same hierarchy are the same. Since the second parameter is a parameter of the hierarchy, the smoothness and differentiability of the initial differentiable activation function can be controlled hierarchically.

[0094] Of course, in specific implementation, the second parameter can be set to be pixel-level, channel-level, or layer-level according to actual needs or experience.

[0095] Through the above process, the target network is obtained, wherein the activation function of at least one layer of the target network is a differentiable activation function, so that the target network can be used for image processing. Figure 4 As shown, a flowchart of the steps of an image processing method provided by an embodiment of the present invention is shown. Figure 4 As shown, the following steps may be specifically included:

[0096] Step S401: Obtain an image to be processed.

[0097] The image to be processed may refer to an image that needs to be identified, and may include an image of the object to be identified. The process of obtaining the image to be processed may refer to an image captured in real time of the object to be identified, or may refer to a frame of video image extracted from a video stream, or may refer to a pre-stored image found in an image library.

[0098] Step S402: input the image to be processed into a target network to obtain a processing result output by the target network.

[0099] Among them, the activation function of at least one layer in the target network is a differentiable activation function, and the differentiable activation function is obtained by training the initial differentiable activation function; the processing result is the classification result of the image to be processed or the recognition result of the object contained in the image to be processed.

[0100] In this embodiment, the target network can classify or identify the image to be processed to obtain a classification result or an identification result of the image to be processed.

[0101] Compared with the prior art, some points in the activation function (e.g., ReLU activation function) used by the convolutional neural network are not differentiable. For example, the intersection of y=x and y=0 in the ReLU activation function is not differentiable. The gradients of these non-differentiable points cannot be returned during the back propagation process, and only approximate gradients can be taken. This leads to problems such as low training quality and inflexible model training. However, after adopting the technical solution implemented by the present invention, since the activation function of at least one layer in the target network is a differentiable activation function, each point of the activation function is differentiable, so the precise value of the gradient of each point can be returned during the back propagation process. In this way, the training intensity of the convolutional neural network can be enhanced, so that the parameters of the network can be updated, so that the target network has a higher accuracy rate when performing object recognition, thereby significantly improving the accuracy rate of the neural network model when performing object recognition, and achieving a higher accuracy rate.

[0102] Among them, in order to facilitate intuitive understanding, the applicant processed the activation functions in various original neural networks to replace the activation functions with differentiable activation functions, thereby constructing various different target networks. The applicant used these different target networks to perform image recognition, and statistically analyzed the recognition error rate of each target network and the recognition error rate of the corresponding original neural network. The statistical results can be shown as follows: Figure 5 shown.

[0103] Figure 5 The original neural networks included are mainly ShuffleNetV2, Mobile Net, Mobile NetV2, and ResNet. Accordingly, four different target networks are constructed. Figure 5 The last line "ACON-C" in the figure represents the error rate of the target network after replacing the activation function with a differentiable activation function, and the corresponding line "max(x,0)" in the second to last line represents the error rate of the original neural network (before replacing the activation function with a differentiable activation function) for object recognition.

[0104] It can be seen that for each original neural network, after replacing the activation function with a differentiable activation function, the error rate of object recognition is greatly reduced. It is understandable that when the error rate of object recognition is greatly reduced, the accuracy of object recognition is greatly improved.

[0105] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0106] Reference Figure 6 As shown, a structural block diagram of an image processing device according to an embodiment of the present invention is shown. Figure 6 As shown, the device may specifically include the following modules:

[0107] An image acquisition module 601 is used to obtain an image to be processed;

[0108] An image input module 602 is used to input the image to be processed into a target network to obtain a processing result output by the target network;

[0109] Among them, the activation function of at least one layer in the target network is a differentiable activation function, and the differentiable activation function is obtained by training the initial differentiable activation function; the processing result is the classification result of the image to be processed or the recognition result of the object contained in the image to be processed.

[0110] Optionally, the initial differentiable activation function contains learnable parameters.

[0111] The device may also include the following modules:

[0112] A sample input module, used for inputting a sample image into a convolutional neural network to obtain a processing result output by the convolutional neural network, wherein the activation function of at least one layer in the convolutional neural network is the initial differentiable activation function, and the parameters of the convolutional neural network include learnable parameters;

[0113] A parameter updating module is used to update the parameters of the convolutional neural network according to the residual between the processing result output by the convolutional neural network and the standard processing result until the preset training end condition is met to obtain the target network, wherein the standard processing result is a classification label pre-annotated for the sample image or a label of an object contained in the sample image.

[0114] Optionally, the learnable parameters include a first parameter that makes the upper and lower limits of the first-order derivative of the initial differentiable activation function adjustable, and / or a second parameter for adjusting the degree of smoothness and differentiability of the initial differentiable activation function.

[0115] Optionally, the second parameter is at a pixel level, and the values ​​of the second parameter corresponding to different pixels are independent of each other; or

[0116] The second parameter is at a channel level, and each pixel point in the same channel shares the same value of the second parameter; or

[0117] The second parameter is hierarchical, and each pixel point of each channel of the same layer shares the same value of the second parameter.

[0118] Optionally, the initial differentiable activation function is:

[0119] S β (p 1 x,p 2 x)=(p 1 -p 2 )x·σ[β(p 1 -p 2 )x]+p 2 x;

[0120] Among them, p 1 is a parameter that makes the upper limit of the first-order derivative of the initial differentiable activation function adjustable, p 2 is a parameter that makes the lower limit of the first-order derivative of the initial differentiable activation function adjustable, β is a parameter used to adjust the smoothness and differentiability of the initial differentiable activation function, σ is the Sigmoid function, and x represents the input.

[0121] Optionally, the device may further include a first convolutional neural network construction module, which is used to construct the convolutional neural network, and may specifically include the following units:

[0122] A first determining unit, used to determine at least one original activation function layer from an original neural network;

[0123] The first replacement unit is used to replace the at least one original activation function layer with a differentiable activation function layer to obtain the convolutional neural network, wherein the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0124] Optionally, the device may further include a second convolutional neural network construction module, which is used to construct the convolutional neural network, and may specifically include the following units:

[0125] A second determining unit is used to determine a plurality of original convolutional layers to be processed from the original neural network, wherein an original activation function layer is configured between every two adjacent original convolutional layers to be processed in the plurality of original convolutional layers to be processed;

[0126] The second replacement unit is used to replace the original activation function layer configured between one or more groups of two adjacent original convolutional layers to be processed with a differentiable activation function layer to obtain the convolutional neural network, and the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0127] Optionally, the device may further include a third convolutional neural network construction module, which is used to construct the convolutional neural network, and may specifically include the following units:

[0128] A third determining unit is used to determine a feature processing layer to be processed from the original neural network, wherein an output end of the feature processing layer is connected to an input end of the original activation function layer, and the feature processing layer is a residual layer or a feature concatenation layer;

[0129] The third replacement unit is used to replace the original activation function layer connected to the output end of the feature processing layer with the differentiable activation function layer to obtain the convolutional neural network, and the activation function in the differentiable activation function layer is the initial differentiable activation function.

[0130] It should be noted that the device embodiment is similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the method embodiment.

[0131] An embodiment of the present invention further provides an electronic device that can be used to execute an image processing method, and may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the image processing method.

[0132] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program that enables a processor to execute the image processing method as described in the embodiment of the present invention.

[0133] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0134] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0135] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0139] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0140] The above is a detailed introduction to an image processing method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An image processing method, characterized in that: include: Obtaining an image to be processed; Inputting the image to be processed into a target network to obtain a processing result output by the target network; Wherein, the activation function of at least one layer in the target network is a differentiable activation function, and the differentiable activation function is obtained by training the initial differentiable activation function; The processing result is a classification result of the image to be processed or a recognition result of an object contained in the image to be processed; Wherein, the initial differentiable activation function includes learnable parameters, and the learnable parameters include a first parameter that makes the upper and lower limits of the first-order derivative of the initial differentiable activation function adjustable, and / or a second parameter for adjusting the degree of smoothness and differentiability of the initial differentiable activation function; Wherein, the second parameter is at the pixel level, and the values ​​of the second parameter corresponding to different pixels are independent of each other; or The second parameter is at a channel level, and each pixel point in the same channel shares the same value of the second parameter; or The second parameter is hierarchical, and each pixel point of each channel of the same layer shares the same value of the second parameter; The initial differentiable activation function is: S β (p1x,p2x)=(p1-p2)x·σ[β(p1-p2)x]+p2x; Among them, p1 is a parameter that makes the upper limit of the first-order derivative of the initial differentiable activation function adjustable, p2 is a parameter that makes the lower limit of the first-order derivative of the initial differentiable activation function adjustable, β is a parameter used to adjust the smoothness and differentiability of the initial differentiable activation function, σ is the Sigmoid function, and x represents the input.

2. The method according to claim 1, characterized in that The training steps of the target network are as follows: A sample image is input into a convolutional neural network to obtain a processing result output by the convolutional neural network, wherein the activation function of at least one layer in the convolutional neural network is the initial differentiable activation function, and the parameters of the convolutional neural network include learnable parameters; according to the residual between the processing result output by the convolutional neural network and the standard processing result, the parameters of the convolutional neural network are updated until a preset training end condition is met to obtain the target network, wherein the standard processing result is a classification label pre-annotated for the sample image or a label of an object contained in the sample image.

3. The method according to claim 2, characterized in that The convolutional neural network is constructed by the following steps: Determine at least one original activation function layer from the original neural network; The at least one original activation function layer is replaced with a differentiable activation function layer to obtain the convolutional neural network, wherein the activation function in the differentiable activation function layer is the initial differentiable activation function.

4. The method according to claim 2, characterized in that: The convolutional neural network is constructed by the following steps: Determine a plurality of original convolutional layers to be processed from the original neural network, wherein an original activation function layer is configured between every two adjacent original convolutional layers to be processed in the plurality of original convolutional layers to be processed; The original activation function layer configured between one or more groups of two adjacent original convolutional layers to be processed is replaced with a differentiable activation function layer to obtain the convolutional neural network, wherein the activation function in the differentiable activation function layer is the initial differentiable activation function.

5. The method according to claim 2, characterized in that: The convolutional neural network is constructed by the following steps: Determine a feature processing layer to be processed from the original neural network, wherein the output end of the feature processing layer is connected to the input end of the original activation function layer, and the feature processing layer is a residual layer or a feature concatenation layer; The original activation function layer connected to the output end of the feature processing layer is replaced with the differentiable activation function layer to obtain the convolutional neural network, and the activation function in the differentiable activation function layer is the initial differentiable activation function.

6. An image processing device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image to be processed; An image input module, used to input the image to be processed into a target network to obtain a processing result output by the target network; Wherein, the activation function of at least one layer in the target network is a differentiable activation function, and the differentiable activation function is obtained by training the initial differentiable activation function; The processing result is a classification result of the image to be processed or a recognition result of an object contained in the image to be processed; Wherein, the initial differentiable activation function includes learnable parameters, and the learnable parameters include a first parameter that makes the upper and lower limits of the first-order derivative of the initial differentiable activation function adjustable, and / or a second parameter for adjusting the degree of smoothness and differentiability of the initial differentiable activation function; Wherein, the second parameter is at the pixel level, and the values ​​of the second parameter corresponding to different pixels are independent of each other; or The second parameter is at a channel level, and each pixel point in the same channel shares the same value of the second parameter; or The second parameter is hierarchical, and each pixel point of each channel of the same layer shares the same value of the second parameter; The initial differentiable activation function is: S β (p1x,p2x)=(p1-p2)x·σ[β(p1-p2)x]+p2x; Among them, p1 is a parameter that makes the upper limit of the first-order derivative of the initial differentiable activation function adjustable, p2 is a parameter that makes the lower limit of the first-order derivative of the initial differentiable activation function adjustable, β is a parameter used to adjust the smoothness and differentiability of the initial differentiable activation function, σ is the Sigmoid function, and x represents the input.

7. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the image processing method according to any one of claims 1 to 5, the image processing method is implemented.

8. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the image processing method as described in any one of claims 1 to 5.

Images