Image processing methods, apparatus, computer equipment and storage media

Abstract

This application discloses an image processing method, apparatus, computer device, and storage medium, belonging to the field of computer technology. The method includes: acquiring an original image processing model comprising multiple nodes, multiple operators between each pair of nodes, and a mapping function; after training to obtain first weight parameters for each operator, mapping the first weight parameters of each operator to second weight parameters based on the mapping function, so that the second weight parameters of the multiple operators between each pair of nodes are discretely distributed; selecting a target operator between each pair of nodes based on the second weight parameters of each operator; constructing a target image processing model based on the multiple nodes and the target operators between each pair of nodes; and calling the target image processing model to process any image. The discrete distribution of the multiple second weight parameters in this method facilitates accurate selection of the target operator, improving the accuracy of the target operator and thus improving the accuracy of the target image processing model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of computer technology and artificial intelligence, the application of network models is becoming increasingly widespread. For example, in the field of image processing, image processing models are used for denoising, color equalization, segmentation, and other processing of images. In these technologies, engineers typically design the network structure of the model, train it using training data, and then use the trained model. However, network models designed by engineers rely on subjective experience, leading to relatively poor accuracy. Summary of the Invention

[0003] This application provides an image processing method, apparatus, computer device, and storage medium, which can improve the accuracy of selected target operators and the accuracy of the constructed image processing model. The technical solution is as follows:

[0004] According to one aspect of the embodiments of this application, an image processing method is provided, the method comprising:

[0005] Obtain the original image processing model, which includes multiple nodes, multiple operators and mapping functions between every two nodes;

[0006] After obtaining the first weight parameters of each operator in the original image processing model through training, the first weight parameters of each operator are mapped to second weight parameters based on the mapping function, so that the second weight parameters of multiple operators between every two nodes are discretely distributed.

[0007] Based on the second weight parameter of each operator, the target operator between every two nodes is selected;

[0008] Based on the multiple nodes and the target operators between every two nodes, a target image processing model is constructed.

[0009] The target image processing model is invoked to process any image.

[0010] According to another aspect of the embodiments of this application, an image processing apparatus is provided, the apparatus comprising:

[0011] The model acquisition module is used to acquire the original image processing model, which includes multiple nodes, multiple operators and mapping functions between every two nodes;

[0012] The weight acquisition module is used to map the first weight parameters of each operator to second weight parameters based on the mapping function after the first weight parameters of each operator in the original image processing model are obtained through training, so that the second weight parameters of multiple operators between every two nodes are discretely distributed.

[0013] The operator selection module is used to select the target operator between every two nodes based on the second weight parameter of each operator;

[0014] The model building module is used to build a target image processing model based on the multiple nodes and the target operators between every two nodes;

[0015] The image processing module is used to call the target image processing model to process any image.

[0016] According to another aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory; the memory stores at least one piece of program code, the at least one piece of program code being executed by the processor to implement the image processing method as described above.

[0017] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the storage medium storing at least one piece of program code, the at least one piece of program code being executed by a processor to implement the image processing method as described above.

[0018] According to another aspect of the embodiments of this application, a computer program product is provided, which stores at least one piece of program code, said at least one piece of program code being loaded and executed by a processor to implement the image processing method described above.

[0019] According to another aspect of the embodiments of this application, a chip is provided, the chip including programmable logic circuits and / or program instructions, which, when the chip is run on a computer device, are used to implement the image processing method described above.

[0020] The image processing method, apparatus, computer device, and storage medium provided in this application optimize the first weight parameters of each operator by training the original image processing model. Based on a mapping function, the optimized first weight parameters of multiple operators are mapped to discretely distributed second weight parameters. Then, a target operator is selected based on the second weight parameters of each operator to construct a target image processing model. This achieves automatic construction of the target image processing model without the need for manual design of the model's network structure. Furthermore, the discretely distributed second weight parameters do not interfere with each other, avoiding the convergence of multiple second weight parameters to the same value after mapping. This ensures that there are significant differences between different second weight parameters, thereby accurately selecting the target operator and improving the accuracy of the target operator, thus improving the accuracy of the constructed target image processing model. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown;

[0023] Figure 2 A flowchart illustrating an image processing method provided in an exemplary embodiment of this application is shown;

[0024] Figure 3 A flowchart illustrating a method for constructing a target image processing model provided in an exemplary embodiment of this application is shown;

[0025] Figure 4 This illustration shows a schematic diagram of a network structure provided by an exemplary embodiment of this application;

[0026] Figure 5 This illustration shows a schematic diagram of a training of a raw image processing model provided in an exemplary embodiment of this application;

[0027] Figure 6 This illustration shows a schematic diagram of a target image processing model construction process provided by an exemplary embodiment of this application;

[0028] Figure 7 A schematic diagram of an image processing method provided in an exemplary embodiment of this application is shown;

[0029] Figure 8A schematic diagram of a target image processing model provided in an exemplary embodiment of this application is shown;

[0030] Figure 9 This invention provides a structural block diagram of an image processing apparatus according to an exemplary embodiment of the present application.

[0031] Figure 10 This invention provides a structural block diagram of an image processing apparatus according to an exemplary embodiment of the present application.

[0032] Figure 11 This invention provides a structural block diagram of a terminal according to an exemplary embodiment of the present application.

[0033] Figure 12 The diagram shows a structural block diagram of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0035] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0036] This application provides an image processing method, in which a computer device is the executing entity. The computer device first constructs an original image processing model including multiple nodes and multiple operators, then trains the original image processing model, selects a target operator from the multiple operators based on the training results, and then constructs a target image processing model based on the multiple nodes and the selected target operator. After that, the target image processing model can be called to process any image.

[0037] In some embodiments, the computer device is a terminal, such as a mobile phone, desktop computer, laptop computer, tablet computer, smart TV, smart speaker, in-vehicle terminal, smart robot, or other types of terminals. Alternatively, the computer device is a server, which can be a single server, a server cluster consisting of several servers, or a cloud computing server center. Or, see [link to relevant documentation]. Figure 1The computer device includes a terminal 101 and a server 102. The terminal 101 trains the original image processing model to obtain the target image processing model, and then uses the target image processing model. Alternatively, after obtaining the target image processing model, the terminal 101 can send the target image processing model to the server 102 or other terminals for use by the server 102 or other terminals. Or, the server 102 trains the original image processing model to obtain the target image processing model, and then uses the target image processing model, or sends the target image processing model to the terminal 101 for use by the terminal 101.

[0038] Figure 2 A flowchart illustrating an image processing method provided in an exemplary embodiment of this application is shown. See also: Figure 2 The method is performed by a computer device and includes:

[0039] 201. Obtain the original image processing model.

[0040] The computer device acquires the original image processing model to be trained. This original image processing model includes multiple nodes, multiple operators between each pair of nodes, and a mapping function. The mapping function maps the weight parameters of each operator in the original image processing model. These weight parameters represent the importance of the operator; the larger the weight parameter, the more important the operator. Nodes represent features to be processed, and operators transform the features to be processed in one node into another feature. The transformed feature is the feature represented by the other node connected to that operator.

[0041] 202. After obtaining the first weight parameters of each operator in the original image processing model through training, the first weight parameters of each operator are mapped to the second weight parameters based on the mapping function, so that the second weight parameters of multiple operators between every two nodes are discretely distributed.

[0042] The computer equipment trains the original image processing model. During the training process, the first weight parameter of each operator is continuously updated until the first weight parameter of each operator converges. Then, based on the mapping function, each first weight parameter is mapped to a second weight parameter. The difference between the second weight parameter and the first weight parameter is that the second weight parameters of multiple operators between every two nodes are discretely distributed, so that the magnitude of multiple second weight parameters can be clearly distinguished.

[0043] 203. Based on the second weight parameter of each operator, select the target operator between every two nodes.

[0044] Since the original image processing model has multiple operators between every two nodes, but the final target image processing model does not need to use each of these operators, the target operator is selected from the multiple operators according to their importance. Furthermore, since the second weight parameters of the multiple operators are discretely distributed, it is easy to determine the larger second weight parameter among them.

[0045] 204. Construct a target image processing model based on multiple nodes and the target operator between every two nodes.

[0046] After the computer equipment determines the target operator between every two nodes, it constructs a target image processing model based on multiple nodes and the target operators between every two nodes. Subsequently, when using this target image processing model, for each node, the features corresponding to that node are used as input to the target operator between that node and the next node. The target operator is then used to process the features, and the processed features are input to the next node. This process, in turn, enables feature processing using target operators within the target image processing model.

[0047] 205. Call the target image processing model to process any image.

[0048] The constructed target image processing model can perform denoising, depigmentation, color equalization, segmentation, restoration, cloud removal, rain removal, and fog removal on any image. The embodiments of this application do not limit the function of the target image processing model.

[0049] This application provides an image processing method that optimizes the first weight parameters of each operator by training an original image processing model. Based on a mapping function, the optimized first weight parameters of multiple operators are mapped to discretely distributed second weight parameters. Then, a target operator is selected based on the second weight parameters of each operator to construct a target image processing model. This achieves automatic construction of the target image processing model without the need for manual design of the model's network structure. Furthermore, the discretely distributed second weight parameters do not interfere with each other, preventing multiple second weight parameters from converging to the same value after mapping. This ensures that different second weight parameters are clearly distinguishable, thereby accurately selecting the corresponding target operator and improving the accuracy of the target operator. This, in turn, improves the accuracy of the constructed target image processing model, enabling the target image processing model to process the original image more accurately.

[0050] The above Figure 2 The illustrated example only briefly introduces the construction process of the target image processing model. The following will explain... Figure 3 The illustrated embodiment provides a detailed description of the training process of the original image processing model.

[0051] Figure 3 A flowchart illustrating a method for constructing a target image processing model according to an exemplary embodiment of this application is shown below. Figure 3 The method is performed by a computer device and includes:

[0052] 301. Obtain the original image processing model.

[0053] The computer device acquires the original image processing model to be trained, which includes multiple nodes, multiple operators between each pair of nodes, and mapping functions.

[0054] The mapping function is used to map the first weight parameter of each operator in the original image processing model. For example, the mapping function is the sigmoid function, which can map the input value to the range of 0-1. Furthermore, when multiple values ​​are input simultaneously, the sigmoid function can map each value separately, meaning the mappings of multiple values ​​do not affect each other. In this embodiment, the sigmoid function can map each first weight parameter to a corresponding second weight parameter, resulting in a discrete distribution of the mapped second weight parameters.

[0055] The first weight parameter indicates the importance of the operator; the larger the first weight parameter, the more important the operator. In some embodiments, the first weight parameter of each operator in the original image processing model is any value, such as 1, 2, 3, etc., and the first weight parameter can be updated during the training process of the original image processing model.

[0056] An operator transforms a feature to be processed in one connected node into another feature. The transformed feature is the feature represented by the other node connected by the operator. For example, operators include various types such as convolution operators, pooling operators, and fully connected operators. This application does not limit the type of operator. For each operator, the operator includes operator parameters. In the original image processing model, the operator parameters of each operator are arbitrary values, and the operator parameters can be updated during the training process of the original image processing model.

[0057] In some embodiments, the computer device first determines the nodes needed in the model to be built, then determines multiple operators between every two nodes, and determines a first weight parameter for each operator. Based on the multiple nodes, the multiple operators between every two nodes, and the first weight parameter for each operator, the original image processing model is constructed.

[0058] For multiple nodes, the computer device first determines the number of nodes needed, and then determines the arrangement of these nodes. For each node, the computer device determines the operators between that node and other nodes, thereby determining multiple operators between every two nodes. In some embodiments, there is a sequential order between every two nodes; for each pair of nodes, the first node is used as the input of the operator, and the second node is used as the output of the operator. It should be noted that since each node is connected to multiple operators, and when the operator precedes the node, the node is the output of the operator; when the operator follows the node, the node is the input of the operator.

[0059] For example, see Figure 4 The diagram shows a network structure with two inputs, three nodes, and one output: input node A, input node B, node 1, node 2, node 3, and output node C. Input nodes A, B, and C are also nodes in this network structure, while nodes 1, 2, and 3 are intermediate nodes. The lines connecting any two nodes represent operators, and the arrows indicate the feature processing order. For nodes 1 and 2, node 1 is used as the input to the operator between them, and the output of the operator is node 2.

[0060] In some embodiments, each operator has corresponding operator parameters. The computer device constructs an original image processing model based on multiple nodes, multiple operators between every two nodes, a first weight parameter for each operator, and the operator parameters. The initial operator parameters are arbitrary values, and the operator parameters can be updated during the training process of the original image processing model.

[0061] In some embodiments, the computer device constructs an original image processing model in a search space that includes multiple nodes and multiple operators, and selects nodes and operators in the search space to construct the original image processing model. For example, the original image processing model is constructed using DARTS (Differentiable Architecture Search), which is a neural network architecture search algorithm based on relaxation continuity and using gradient descent for search.

[0062] 302. Based on the training data, train the original image processing model to obtain the first weight parameter and operator parameter of each operator.

[0063] The computer device acquires training data and trains the first weight parameters and operator parameters of each operator in the original image processing model based on the training data. The training data includes sample images and the corresponding target images.

[0064] In some embodiments, the computer device inputs a sample image into an original image processing model, processes the sample image based on the original image processing model, and obtains a predicted image corresponding to the sample image. Then, the difference between the predicted image and the target image is determined; the smaller the difference, the higher the accuracy of the original image processing model. Therefore, based on this difference, the computer device adjusts the first weight parameter and operator parameters of at least one operator in the original image processing model, and then continues to train the original image processing model based on other sample images and the target image to reduce the difference between the predicted image and the target image, thereby obtaining a more accurate original image processing model. For example, gradient descent, Adam (Adaptive Moment Estimation) optimization algorithm, sharpness-aware minimization optimization algorithm, or other optimization algorithms are used to adjust the first weight parameter and operator parameters.

[0065] In some embodiments, since the original image includes multiple nodes with a sequential order, the computer device inputs the sample image into the first node of the original image processing model, processes the sample image based on each operator between the first node and the next node, and performs a weighted sum of the processing results of the multiple operators according to the first weight parameter of the multiple operators to obtain the processing result of the next node, until the processing result of the last node of the original image processing model is obtained, and the processing result of the last node is determined as the predicted image.

[0066] In some embodiments, the computer device keeps the first weight parameters and the first type of operator parameters in the original image processing model unchanged, trains the second type of parameters in the original image processing model to obtain the second type of parameters for each operator, keeps the second type of parameters for each operator unchanged, trains the first type of parameters in the original image processing model to obtain the first type of parameters for each operator, until all obtained operator parameters and each first weight parameter converge. Wherein, when the first type of parameter is the first weight parameter, the second type of parameter is the operator parameter; when the first type of parameter is the operator parameter, the second type of parameter is the first weight parameter.

[0067] In other words, the computer equipment, with the first weight parameter in the original image processing model unchanged, trains the operator parameters in the original image processing model based on training data to obtain the operator parameters for each operator. Then, with the operator parameters of each operator unchanged, it trains the weight parameters in the original image processing model based on training data to obtain the first weight parameter for each operator, until all the obtained operator parameters and each first weight parameter converge. That is, the first weight parameters and operator parameters are trained alternately until all operator parameters and each first weight parameter converge. For example, see... Figure 5 Keeping the operator parameters α, β, γ, etc. of each operator unchanged, update the weight parameters of operator 1, operator 2, operator 3 and operator 4 respectively. Then, keeping the weight parameters of each operator unchanged, update the operator parameters α, β, γ, etc.

[0068] The above implementation is only an example of alternating training of the first weight parameter and then the operator parameter. In another embodiment, the operator parameters can be trained multiple times while keeping the first weight parameter unchanged, and then the first weight parameter can be trained multiple times while keeping the operator parameters unchanged. That is, the computer device trains the original image processing model based on training data while keeping the first weight parameter unchanged, obtaining the operator parameters for each operator. After the number of training iterations for the operator parameters reaches the first number, the original image processing model is trained again based on training data while keeping the operator parameters unchanged, obtaining the first weight parameter for each operator. Then, after the number of training iterations for the first weight parameter reaches the second number, the operator parameters are trained again. Here, the first number and the second number are both any number greater than 1, and the first number and the second number can be the same or different. In some embodiments, considering that the number of first weight parameters in the original image processing model is much smaller than the number of operator parameters, the first number is greater than the second number.

[0069] It should be noted that the above embodiments are only illustrated by taking the training of operator parameters first as an example. In another embodiment, the first weight parameter can be trained first. That is, the operator parameters of each operator in the original image processing model are kept unchanged. Based on the training data, the first weight parameter of each operator in the original image processing model is trained to obtain the first weight parameter of each operator. When the training number of the first weight parameter reaches the second number, the first weight parameter of each operator is kept unchanged. Based on the training data, the operator parameters of each operator in the original image processing model are trained to obtain the operator parameters of each operator until the obtained operator parameters and the first weight parameter are converged.

[0070] Alternatively, in another embodiment, the first weight parameter and operator parameter of each operator are trained simultaneously. This application embodiment does not limit the training method of the original image processing model.

[0071] In some embodiments, the computer device trains the original image processing model based on a loss function and training data, obtaining the first weight parameter and operator parameters for each operator when the output of the loss function reaches its minimum value. For example, the loss function is:

[0072] L = L ISP +τ·L re

[0073] Where L represents the loss function, L ISP Let L represent the first loss function. re Let τ represent the second loss function, and τ be the reference weight. The first loss function can be the mean squared error loss function, cross-entropy loss function, or other loss function. The second loss function is a loss function applied to the weight parameters. The reference weight indicates the weight of the second loss function within the overall loss function; for example, the reference weight can be 0.3, 0.5, or other values ​​greater than 0 and less than 1. For example, the second loss function can be a constraint function.

[0074] 303. Based on the mapping function, the first weight parameter of each operator is mapped to the second weight parameter, so that the second weight parameters of multiple operators between every two nodes are discretely distributed.

[0075] In this embodiment, a mapping function is used to map the first weight parameter of each operator obtained during training, resulting in a second weight parameter corresponding to each first weight parameter. The value range of the second weight parameter is 0-1, and the mapping function maps each first weight parameter independently. This means that multiple first weight parameters are independently mapped to the 0-1 range. During the mapping process, any two first weight parameters do not affect each other. Therefore, the resulting second weight parameters of the multiple operators are discretely distributed, leading to significant differences between the obtained second weight parameters. For example, the mapping function in this application is the sigmoid function.

[0076] In some embodiments, after obtaining the second weight parameters, the computer device performs regularization on the second weight parameters of multiple operators between every two nodes, and further discretizes the second weight parameters of the multiple operators. For each second weight parameter, the square of the difference between the second weight parameter and a first threshold is obtained to determine the regularized second weight parameter, wherein the first threshold is greater than 0 and less than 1.

[0077] For example, a computer device performs regularization on multiple second weight parameters based on a constraint function. The constraint function satisfies the following conditions: (1) The constraint function is differentiable within its domain; (2) The constraint function reaches its maximum when the second weight parameter is 0.5, and the constraint function monotonically decreases on both sides at 0.5; (3) When the second weight parameter is 0.5, the reciprocal of the constraint function approaches 0.

[0078] For example, the constraint function is:

[0079]

[0080] Among them, L re Let N represent the constraint function, N represent the number of operators between any two nodes, and i represent the i-th operator. σ(α) i ) represents the second weight parameter corresponding to the i-th operator, σ(α) i -0.5 represents the second weight parameter after regularization.

[0081] The constraint function is differentiable within its domain. When the value of the second weight parameter is 0.5, the output value of the constraint function is 0. When the value of the second weight parameter is greater than 0.5 or less than 0.5, the output value is less than 0 and monotonically decreasing. When the value of the second weight parameter is 0.5, the reciprocal of the constraint function is 0. Therefore, the constraint function satisfies the above conditions.

[0082] 304. Based on the second weight parameter of each operator, select the target operator from the multiple operators between every two nodes.

[0083] Since the original image processing model has multiple operators between every two nodes, but the final target image processing model only needs to use at least one of these operators, the target operator is selected from the multiple operators according to their importance. Furthermore, since the second weight parameters of the multiple operators are discretely distributed, it is easy to determine the larger second weight parameter among them.

[0084] In some embodiments, the computer device selects target operators based on the magnitude of the second weight parameter of each operator. That is, the computer device selects a target number of target operators from a plurality of operators between every two nodes, wherein the second weight parameter of the selected target operators is greater than the second weight parameter of the unselected operators. This target number can be any number, for example, 1, 2, 3, 4, or other numbers. Optionally, the computer device arranges the second weight parameters of the plurality of operators between every two nodes in descending order, and sequentially selects a target number of operators as target operators starting from the operator corresponding to the first second weight parameter. Alternatively, the computer device can also use other methods to select the target number of target operators from a plurality of operators.

[0085] In the case where the target number is 1, the computer device selects the target operator with the largest second weight parameter from multiple operators between every two nodes.

[0086] In some embodiments, the computer device selects target operators based on whether the second weight parameter of each operator has an upward trend. That is, the computer device selects target operators from multiple operators between every two nodes whose second weight parameter has an upward trend. An upward trend in the second weight parameter means that the second weight parameter gradually increases as the number of training iterations increases during the training process of the original image processing model.

[0087] Optionally, an upward trend in the second weight parameter means that, starting from the first training iteration of the original image processing model, the second weight parameter after each training iteration is greater than the second weight parameter after the previous training iteration. For example, after 5 training iterations, the second weight parameter is 1 after the first training iteration, and the second weight parameter after the subsequent training iterations is 1.2, 1.4, 1.6, and 1.8 respectively. In this case, the second weight parameter of the operator is determined to have an upward trend. Alternatively, an upward trend in the second weight parameter means that the second weight parameter obtained in the last training iteration is greater than the initial second weight parameter, and the magnitude of the second weight parameter may fluctuate during the intermediate training process. For example, after 5 training sessions, the second weight parameter is 1 after the first training session, and the second weight parameter after the subsequent training sessions is 1.2, 1.4, 1.6, and 1.4 respectively; or, the second weight parameter has an upward trend, which means that in the continuous training process before the final second weight parameter is obtained, the second weight parameter after each training session is greater than the second weight parameter after the previous training session. For example, the second weight parameter increases in three consecutive training sessions. After 5 training sessions, the second weight parameter is 1 after the first training session, and the second weight parameter after the subsequent training sessions is 0.8, 1, 1.2, and 1.4 respectively.

[0088] It should be noted that when the computer device selects an operator whose second weight parameter has an upward trend as the target operator, it does not limit the number of target operators selected between any two nodes. For example, if two of the five operators between two nodes have a second weight parameter with an upward trend, then those two operators with an upward trend are selected as the target operators. If one of the five operators between the other two nodes has a second weight parameter with an upward trend, then that one operator with an upward trend is selected as the target operator.

[0089] In some embodiments, the computer device selects target operators based on the magnitude of the second weight parameter of each operator and whether it exhibits an upward trend. Specifically, the computer device selects multiple candidate operators whose second weight parameters exhibit an upward trend from among the multiple operators between every two nodes, and then selects a target number of target operators from among these candidate operators. The second weight parameter of the target operator is greater than the second weight parameter of any unselected candidate operator. The method for selecting candidate operators with an upward trend is similar to the method for selecting target operators with an upward trend described above. Similarly, the method for selecting target operators with larger second weight parameters from among multiple candidate operators is similar to the method for selecting multiple target operators with larger second weight parameters from among multiple operators described above, and will not be repeated here.

[0090] In related technologies, after obtaining the first weight parameters of multiple operators, a softmax function is used to map these first weight parameters. This softmax function can map the input values ​​to the range of 0-1, and when multiple values ​​are input simultaneously, the softmax function can guarantee that the sum of the mapped values ​​is 1. However, because the multiple first weight parameters are interdependent during the mapping process, and because the final converged values ​​of the first weight parameters of multiple operators are quite similar, the second weight parameters obtained after mapping all converge to 0.167, making it difficult to select the target operator from multiple operators based on the second weight parameters. In contrast, the mapping function used in this application can discretize the multiple first weight parameters, resulting in a discrete distribution of the obtained second weight parameters. Therefore, there is no situation where multiple second weight parameters converge to the same value, making it easier to select the target operator from multiple operators.

[0091] For example, see Figure 6 The diagram shown illustrates the construction process of the target image processing model. The original image processing model includes nodes 1, 2, 3, and 4, with three operators between each pair of nodes. Figure 6In this model, the connecting lines between two nodes represent the corresponding operators, with the arrows indicating the input and output of the operators. Each operator initially has a first weight parameter of 1. The computer trains the first weight parameters of each operator in the original image processing model to obtain the second weight parameters of each operator in the trained model. Then, the target operator with the largest second weight parameter is selected from the multiple operators between each pair of nodes. Subsequently, the target image processing model can be determined based on the four nodes and the target operators between each pair of nodes.

[0092] 305. Construct a target image processing model based on multiple nodes and the target operator between every two nodes.

[0093] After the computer equipment determines the target operator between every two nodes, it constructs a target image processing model based on multiple nodes and the target operators between every two nodes. The target operator selected between every two nodes can be one or more.

[0094] In some embodiments, when the target operator between every two nodes is the operator with the largest second weight parameter between every two nodes, the target operator is more important than other operators. That is, it is more appropriate to process the output of the previous node based on the target operator. Therefore, the accuracy of the target image processing model constructed based on the multiple nodes, the target operator between every two nodes, and the operator parameters of each target operator is higher.

[0095] In some embodiments, since the original image processing model includes a target operator and other operators during the previous training process, the operator parameters of the target operator are affected by the other operators when training. Therefore, in order to improve the accuracy of the target image processing model and avoid the influence of other operators on the operator parameters of the target operator, after the computer device constructs the target image processing model, it continues to use training data to train the operator parameters in the target image processing model until the operator parameters in the target image processing model converge. The target image processing model obtained at this time has higher accuracy.

[0096] It should be noted that after constructing the target image processing model, the training data used to further train the target image processing model can be the same as or different from the training data used to train the original image processing model.

[0097] This application provides an image processing method that optimizes the first weight parameters of each operator by training an original image processing model. Based on a mapping function, the optimized first weight parameters of multiple operators are mapped to discretely distributed second weight parameters. Then, a target operator is selected based on the second weight parameters of each operator to construct a target image processing model. This achieves automatic construction of the target image processing model without the need for manual design of the model's network structure. Furthermore, the discretely distributed second weight parameters do not constrain each other, preventing multiple second weight parameters from converging to the same value after mapping. This ensures that different second weight parameters are clearly distinguishable, thereby accurately selecting the target operator with the largest corresponding second weight parameter, improving the accuracy of the target operator, and thus improving the accuracy of the constructed target image processing model.

[0098] Furthermore, after mapping the first weight parameter to the second weight parameter, the second weight parameter is further regularized to make the second weight parameters of multiple operators more discrete, which makes it easier to select the target operator based on the second weight parameter and further improves the accuracy of the target operator.

[0099] Furthermore, the alternating training of weight parameters and operator parameters in this application can avoid mutual interference between weight parameters and operator parameters during training. With sufficient training of operator parameters, it ensures that the weight parameters will not overfit, thereby improving the accuracy of the original image processing model.

[0100] It should be noted that steps 301-305 above describe the process of constructing the target image processing model; see [link / reference]. Figure 7 After constructing the target image processing model, the computer device can also call the target image processing model to process the image.

[0101] 306. Call the target image processing model to process the first image and obtain the second image corresponding to the first image.

[0102] The first image can be any image. The target image processing model can be applied to any image processing scenario. For example, the target image processing model can be used to denoise an image. The first image is input into the target image processing model to obtain a denoised second image; or, the target image processing model can be used to perform color equalization on an image. The first image is input into the target image processing model to obtain a color-equalized second image; or, the target image processing model can be used to remove mosaic effects. The first image is input into the target image processing model to obtain a de-mosaic second image. Of course, the target image processing model can also perform other processing on the first image, such as segmentation, recognition, restoration, cloud removal, rain removal, and fog removal. The embodiments of this application do not limit the function of the target image processing model.

[0103] In some embodiments, the computer device determines the performance of the second image processing model based on the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) between the first and second images. PSNR indicates whether an image is distorted; a higher PSNR indicates better image quality, meaning a higher performance of the target image processing model. SSIM indicates the similarity between two images; a higher SSIM indicates better image quality, meaning a higher performance of the target image processing model.

[0104] In some embodiments, the computer device inputs a first image into a target image processing model. Based on the target operator between the first and second nodes in the target image processing model, it processes the node data of the first node to obtain the processing result of the target operator. The processing result of the target operator is then determined as the node data of the second node. If the node data of the last node in the target image processing model is obtained, the processing result of the last node is determined as the second image corresponding to the first image. Here, the first node is the first node in the target image processing model, and the second node is the first node connected to the first node. When the first node is the first node in the target image processing model, the node data of the first node is the input first image. When the first node is not the first node, the node data of the first node is the processing result output by the target operator connected to and preceding the first node; that is, the node data is an image or image features corresponding to the image.

[0105] For example, see Figure 6The target image processing model shown includes four nodes. A first image is input into node 1. Following the processing flow indicated by the connecting lines between each pair of nodes, the first image is first processed using the target operator between nodes 1 and 2, resulting in processing result 1 at node 2. Then, the first image is processed again using the target operator between nodes 1 and 3, and processing result 2 is input into node 3. Finally, processing result 1 is processed using the target operator between nodes 2 and 3, resulting in processing result 3, which is then input into node 3. Result 2 and processing result 3 are used as processing result 4 of node 3. Then, the first image is processed based on the target operator between node 1 and node 4, and the resulting processing result 5 is input to node 4. Processing result 1 is processed based on the target operator between node 2 and node 4, and the resulting processing result 6 is input to node 5. Processing result 4 is processed based on the target operator between node 3 and node 4, and the resulting processing result 7 is input to node 4. Processing result 5, processing result 6 and processing result 7 are used as the processing result of node 4. Then, the corresponding second image is determined based on the processing result of node 4.

[0106] In some embodiments, when there are multiple target operators between the first node and the second node, the computer device processes the node data of the first node based on the multiple target operators, obtaining the processing results of the multiple target operators. Based on the second weight parameters of the multiple target operators, the processing results of the multiple target operators are weighted and summed, and the weighted sum is determined as the node data of the second node. Each target operator processes the node data of the first node independently, and the target operators do not interfere with each other.

[0107] In some embodiments, the target image processing model is a U-shaped network, a deep learning framework where the input and output sizes remain constant. This target image processing model includes n downsampling networks and n upsampling networks. When a computer device invokes the target image processing model to process a first image and obtain a second image corresponding to the first image, the process includes: inputting the first image into the first downsampling network; downsampling the first image based on the first downsampling network to obtain a first downsampling feature; downsampling the downsampling feature output by the previous downsampling network based on the k-th downsampling network to obtain a k-th downsampling feature, until the n-th downsampling network outputs the n-th downsampling feature; and performing an identity mapping on the n-th downsampling feature. The process involves obtaining the nth mapping feature; inputting the nth mapping feature into the first upsampling network, and upsampling the nth mapping feature based on the first upsampling network to obtain the first upsampled feature; then, based on the kth upsampling network, upsampling the upsampled feature output by the previous upsampling network and the (n+1-k)th mapping feature to obtain the kth upsampled feature, and so on, until the second image is obtained from the nth upsampling network output; wherein, the n downsampling networks and n upsampling networks in the target image processing model are connected sequentially, where n is a positive integer greater than 1, and k is a first positive integer greater than 1 and not greater than n. The nth downsampling feature is identical to the corresponding nth mapping feature.

[0108] For each downsampling network and each upsampling network in the target image processing model, both the downsampling network and the upsampling network include multiple nodes and target operators between every two nodes. The processing procedure of each downsampling network and each upsampling network is similar to the processing procedure of the target image processing model based on the first node, the second node, and the target operators between the first node and the second node for the input first image.

[0109] For example, see Figure 8 The target image processing model includes downsampling networks 801, 802, 803, and 804, and upsampling networks 805, 806, 807, and 808, which are connected sequentially. The first image is input into downsampling network 801, where multiple downsampling networks process it sequentially. The processing result of each downsampling network is then input into the corresponding upsampling network after an identity mapping, enabling the reuse of the downsampling network's processing results. Finally, the last upsampling network 808 in the target image processing model outputs the second image.

[0110] Taking the upsampling network 808 as an example, the output of the downsampling network 801 is used as input 1 of the upsampling network 808, and the output of the previous upsampling network 807 is used as input 2 of the upsampling network 808. Input 1 is processed based on target operator 1 between input node A and node 1, and input 2 is processed based on target operator 2 between input node B and node 1. The outputs of target operator 1 and target operator 2 are used as node data of node 1. Similarly, node data of node 2 is obtained based on target operators between node 2 and input node A, input node B, and node 1. Node data of node 3 is obtained based on target operators between node 3 and input nodes A, B, 1, and 2. Node data of output node C is obtained based on target operators between output node C and nodes 1, 2, and 3. The node data of output node C is the second image corresponding to the input image. For other upsampling or downsampling networks, the corresponding processing results are obtained in a similar manner to those of the upsampling network 808. Figure 8 This example illustrates the concept of having one target operator between every two nodes. In another embodiment, multiple target operators may be included between every two nodes. The node data of the previous node is processed based on the multiple target operators to obtain the node data of the next node.

[0111] In some embodiments, the downsampling features and upsampling features are in the form of feature maps, see [link to documentation]. Figure 8 Before the first downsampling network, a combination of a convolution with a stride of 2 and a linear activation operator is used to reduce the feature map size of the first image by a factor of 2, thereby reducing the amount of feature map data processed. After the last upsampling network, a combination of a transposed convolution with a stride of 2 and a linear activation operator is used to increase the feature map size by a factor of 2, in order to restore the amount of feature map data. Between the last downsampling network and the first upsampling network, a combination of a convolution with a stride of 1 and a linear activation operator is used to further linearly combine the feature map output by the last downsampling network, thereby outputting the first upsampling network.

[0112] In some embodiments, the target image processing model obtained in this application can be applied to an ISP (Image Signal Processing) module, which refers to a module configured inside a computer device for processing captured images. For example, if the computer device is a terminal with a shooting function, the target image processing model is applied to the ISP module in the terminal. After the terminal captures a first image through the camera, the captured image is processed by the ISP module to obtain a second image.

[0113] This application provides an image processing method that optimizes the first weight parameters of each operator by training an original image processing model. Based on a mapping function, the optimized first weight parameters of multiple operators are mapped to discretely distributed second weight parameters. Then, a target operator is selected based on the second weight parameters of each operator to construct a target image processing model. This achieves automatic construction of the target image processing model without the need for manual design of the model's network structure. Furthermore, the discretely distributed second weight parameters do not constrain each other, preventing multiple second weight parameters from converging to the same value after mapping. This ensures that different second weight parameters are clearly distinguishable, thereby accurately selecting the target operator with the largest corresponding second weight parameter, improving the accuracy of the target operator, and thus improving the accuracy of the constructed target image processing model.

[0114] The following are embodiments of the apparatus of this application, which can be used to execute the embodiments of the method of this application. For details not disclosed in the embodiments of the apparatus of this application, please refer to the embodiments of the method of this application.

[0115] Figure 9 A structural block diagram of an image processing apparatus provided in an exemplary embodiment of this application is shown. This image processing apparatus is applied to a computer device; see [link to documentation]. Figure 9 The image processing apparatus includes:

[0116] The model acquisition module 901 is used to acquire the original image processing model, which includes multiple nodes, multiple operators and mapping functions between every two nodes;

[0117] The weight acquisition module 902 is used to map the first weight parameters of each operator to the second weight parameters based on the mapping function after the first weight parameters of each operator in the original image processing model are obtained through training, so that the second weight parameters of multiple operators between every two nodes are discretely distributed.

[0118] The operator selection module 903 is used to select the target operator between every two nodes based on the second weight parameter of each operator;

[0119] Model building module 904 is used to build a target image processing model based on multiple nodes and the target operators between every two nodes;

[0120] The image processing module 905 is used to call the target image processing model to process any image.

[0121] The apparatus provided in this application optimizes the first weight parameters of each operator by training the original image processing model. Based on a mapping function, it maps the optimized first weight parameters of multiple operators to discretely distributed second weight parameters. Then, based on the second weight parameters of each operator, it selects a target operator to construct a target image processing model. This achieves automatic construction of the target image processing model without the need for manual design of the model's network structure. Furthermore, the discretely distributed second weight parameters do not interfere with each other, avoiding the convergence of multiple second weight parameters to the same value after mapping. This ensures that there are significant differences between different second weight parameters, thereby accurately selecting the target operator with the largest corresponding second weight parameter, improving the accuracy of the target operator, and thus improving the accuracy of the constructed target image processing model.

[0122] In some embodiments, see Figure 10 Operator selection module 903 is used for:

[0123] From the multiple operators between every two nodes, select the target operator with the target number of operators, where the second weight parameter of the target operator is greater than the second weight parameter of the unselected operators; or,

[0124] From the multiple operators between every two nodes, select the target operator whose second weight parameter has an upward trend. An upward trend in the second weight parameter means that the second weight parameter gradually increases with the number of training iterations during the training process of the original image processing model; or...

[0125] From the multiple operators between every two nodes, select multiple candidate operators whose second weight parameter has an upward trend. From the multiple candidate operators, select a target number of target operators. The second weight parameter of the target operator is greater than the second weight parameter of the unselected candidate operators.

[0126] In some embodiments, see Figure 10 The device also includes:

[0127] Regularization module 906 is used to regularize the second weight parameters of multiple operators between every two nodes.

[0128] In some embodiments, see Figure 10 The regularization module 906 is used to obtain the square of the difference between the second weight parameter and the first threshold for each second weight parameter, and to determine the regularized second weight parameter. The first threshold is greater than 0 and less than 1.

[0129] In some embodiments, see Figure 10 The device also includes:

[0130] The first training module 907 is used to train the original image processing model to obtain the first weight parameters and operator parameters of each operator, so that the first weight parameters and operator parameters of each operator converge.

[0131] In some embodiments, see Figure 10 The device also includes:

[0132] The first training module 907 is used to keep the first weight parameter and the first type of parameter in the operator parameters of the original image processing model unchanged, train the second type of parameter in the original image processing model to obtain the second type of parameter for each operator, keep the second type of parameter of each operator unchanged, train the first type of parameter in the original image processing model to obtain the first type of parameter for each operator, until each operator parameter and each first weight parameter are converged. Wherein, when the first type of parameter is the first weight parameter, the second type of parameter is the operator parameter, and when the first type of parameter is the operator parameter, the first type of parameter is the first weight parameter.

[0133] In some embodiments, see Figure 10 The device also includes:

[0134] The second training module 908 is used to train the operator parameters of each target operator in the target image processing model based on the training data until the operator parameters of each target operator converge. The training data includes sample images and target images corresponding to the sample images.

[0135] In some embodiments, the image processing module 905 is configured to:

[0136] The first image is input into the target image processing model;

[0137] Based on the target operator between the first node and the second node in the target image processing model, the node data of the first node is processed to obtain the processing result of the target operator. The processing result of the target operator is then used as the node data of the second node. The first node can be any node in the target image processing model.

[0138] Given the node data of the last node in the target image processing model, the node data of the last node is determined as the target image corresponding to the first image.

[0139] In some embodiments, multiple target operators are present between the first node and the second node, and the image processing module 905 is used for:

[0140] Based on multiple target operators between the first node and the second node, the node data of the first node is processed respectively to obtain the processing results of multiple target operators;

[0141] Based on the second weight parameters of multiple target operators, the processing results of multiple target operators are weighted and summed, and the processing result obtained by weighted summation is determined as the node data of the second node.

[0142] In some embodiments, the target image processing model includes n downsampling networks and n upsampling networks, see [link to documentation]. Figure 10 Both the downsampling network and the upsampling network include multiple nodes and target operators between every two nodes. This image processing module 905 is used for:

[0143] The first image is input into the first downsampling network, and the image is downsampled based on the first downsampling network to obtain the first downsampled feature;

[0144] Based on the k-th downsampling network, the downsampling features output by the previous downsampling network are downsampled to obtain the k-th downsampling feature, until the n-th downsampling network outputs the n-th downsampling feature;

[0145] Perform an identity mapping on the nth downsampled feature to obtain the nth mapped downsampled feature;

[0146] The nth mapping downsampled feature is input into the first upsampled network, and the nth mapping downsampled feature is upsampled based on the first upsampled network to obtain the first upsampled feature;

[0147] Based on the k-th upsampling network, the upsampling features output by the previous upsampling network and the (n+1-k)-th mapping downsampling features are upsampled to obtain the k-th upsampling feature, until the second image corresponding to the first image output by the n-th upsampling network is obtained;

[0148] Where n is a positive integer greater than 1, and k is any positive integer greater than 1 and not greater than n.

[0149] In some embodiments, see Figure 10 The device also includes:

[0150] Evaluation module 909 is used to obtain the peak signal-to-noise ratio (PSNR) between the first image and the second image, whereby the PSNR indicates whether the second image is distorted; or,

[0151] Evaluation module 909 is used to obtain the structural similarity between the first image and the second image, and the structural similarity is used to represent the similarity between the first image and the second image.

[0152] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0153] This application provides a computer device including a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the image processing method provided in the above-described method embodiments.

[0154] In some embodiments, the computer device is a terminal; please refer to [reference needed]. Figure 11 This diagram illustrates a structural block diagram of a terminal provided in an exemplary embodiment of this application. In some embodiments, the terminal 1100 is a smartphone, tablet, wearable device, or other terminal capable of accessing a wireless local area network as a wireless station. The terminal 1100 in this application includes at least one or more of the following components: a processor 1110, a memory 1120, and at least two wireless links 1130.

[0155] In some embodiments, processor 1110 includes one or more processing cores. Processor 1110 connects to various parts within terminal 1100 using various interfaces and lines, and performs various functions and processes data of terminal 1100 by running or executing program code stored in memory 1120 and calling data stored in memory 1120. In some embodiments, processor 1110 is implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1110 can integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; the NPU is used to implement Artificial Intelligence (AI) functions; and the modem is used for wireless communication. It is understandable that the aforementioned modem could also be implemented as a separate chip without being integrated into the processor 1110.

[0156] In some embodiments, the processor 1110 is used to control the operating status of at least two wireless links 1130. Accordingly, the processor 1110 is a processor integrating a Wireless Fidelity (Wi-Fi) chip. This Wi-Fi chip is a chip with dual Wi-Fi processing capabilities. For example, the Wi-Fi chip is a dual-band dual-concurrent (DBDC) chip, or a dual-band simultaneous (DBS) chip, etc.

[0157] In some embodiments, memory 1120 includes random access memory (RAM), and in some embodiments, memory 1120 includes read-only memory (ROM). In some embodiments, memory 1120 includes non-transitory computer-readable storage medium. Memory 1120 can be used to store program code. Memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created based on the use of terminal 1100 (such as audio data, phonebook, etc.).

[0158] In some embodiments, the memory 1120 stores reception schemes for different wireless links 1130 receiving beacon frames, as well as identifiers of access nodes connected to different wireless links 1130, identifiers of wireless links 1130, etc.

[0159] The at least two wireless links 1130 are used to connect different access points (APs). They receive downlink data from the APs. These different access points can be access points within the same router or access points within different routers.

[0160] In some embodiments, the terminal 1100 further includes a display screen. The display screen is a display component used to display a user interface. In some embodiments, the display screen is a touch-enabled display screen, allowing users to perform touch operations on the display screen using fingers, styluses, or any suitable object. In some embodiments, the display screen is typically located on the front panel of the terminal 1100. In some embodiments, the display screen is designed as a full-screen, curved screen, irregularly shaped screen, dual-sided screen, or foldable screen. In some embodiments, the display screen is also designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen, etc., which are not limited in this embodiment.

[0161] In addition, those skilled in the art will understand that the structure of the terminal 1100 shown in the above figures does not constitute a limitation on the terminal 1100. The terminal 1100 may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal 1100 may also include components such as a microphone, speaker, input unit, sensor, audio circuit, module, power supply, and Bluetooth module, which will not be described in detail here.

[0162] In some embodiments, the computer device is a server; please refer to [reference needed]. Figure 12 This diagram illustrates a structural block diagram of a server provided in an exemplary embodiment of this application. The server 1200 can vary significantly due to different configurations or performance characteristics. It may include one or more Central Processing Units (CPUs) 1201 and one or more memories 1202. The memories 1202 store at least one computer program, which is loaded and executed by the processor 1201 to implement the methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input / output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0163] This application also provides a computer-readable medium storing at least one piece of program code, which is loaded and executed by the processor to implement the image processing methods shown in the above embodiments.

[0164] This application also provides a computer program product that stores at least one piece of program code, which is loaded and executed by the processor to implement the image processing methods shown in the above embodiments.

[0165] According to another aspect of the embodiments of this application, a chip is provided, the chip including programmable logic circuits and / or program instructions, which, when the chip is run on a terminal device, is used to implement the image processing method described above.

[0166] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0167] Those skilled in the art will understand that all or part of the steps in the image processing method of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. The above descriptions are merely preferred embodiments of this application and are not intended to limit the application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the original image processing model, which includes multiple nodes, multiple operators and mapping functions between every two nodes; After training to obtain the first weight parameters of each operator in the original image processing model, the first weight parameters of each operator are mapped to second weight parameters based on the mapping function. During the mapping process, any two first weight parameters do not affect each other, so that the second weight parameters of the multiple operators between each two nodes are discretely distributed. Based on the second weight parameter of each operator, the target operator between every two nodes is selected; Based on the multiple nodes and the target operators between every two nodes, a target image processing model is constructed. The target image processing model is invoked to process any image.

2. The method according to claim 1, characterized in that, The step of selecting the target operator between every two nodes based on the second weight parameter of each operator includes: From a plurality of operators between every two nodes, a target number of target operators are selected, wherein the second weight parameter of the target operator is greater than the second weight parameter of the unselected operators; or, From among multiple operators between every two nodes, select the target operator whose second weight parameter has an upward trend. An upward trend in the second weight parameter means that during the training process of the original image processing model, the second weight parameter gradually increases with the increase of training iterations; or... From a plurality of operators between every two nodes, select a plurality of candidate operators whose second weight parameter has an upward trend, and select a target number of target operators from the plurality of candidate operators, wherein the second weight parameter of the target operator is greater than the second weight parameter of the unselected candidate operators.

3. The method according to claim 1, characterized in that, After mapping the first weight parameter of each operator to the second weight parameter based on the mapping function, the method further includes: The second weight parameters of multiple operators between every two nodes are regularized.

4. The method according to claim 3, characterized in that, The regularization process for the second weight parameters of multiple operators between every two nodes includes: For each second weight parameter, the square of the difference between the second weight parameter and the first threshold is obtained to determine the regularized second weight parameter, where the first threshold is greater than 0 and less than 1.

5. The method according to claim 1, characterized in that, After obtaining the first weight parameters of each operator in the original image processing model through training, and before mapping the first weight parameters of each operator to second weight parameters based on the mapping function, the method further includes: The original image processing model is trained to obtain the first weight parameter and operator parameter of each operator, so that the first weight parameter and operator parameter of each operator converge.

6. The method according to claim 1, characterized in that, After obtaining the first weight parameters of each operator in the original image processing model through training, and before mapping the first weight parameters of each operator to second weight parameters based on the mapping function, the method further includes: Keeping the first weight parameter and the first type of parameter in the operator parameters of the original image processing model unchanged, the second type of parameter in the original image processing model is trained to obtain the second type of parameter for each operator. Keeping the second type of parameter for each operator unchanged, the first type of parameter in the original image processing model is trained to obtain the first type of parameter for each operator, until each operator parameter and each first weight parameter converge. Wherein, when the first type of parameter is the first weight parameter, the second type of parameter is the operator parameter, and when the first type of parameter is the operator parameter, the first type of parameter is the first weight parameter.

7. The method according to claim 1, characterized in that, After constructing the target image processing model based on the multiple nodes and the target operator between every two nodes, the method further includes: Based on the training data, the operator parameters of each target operator in the target image processing model are trained until the operator parameters of each target operator converge. The training data includes sample images and target images corresponding to the sample images.

8. The method according to claim 1, characterized in that, The step of calling the target image processing model to process any image includes: The first image is input into the target image processing model; Based on the target operator between the first node and the second node in the target image processing model, the node data of the first node is processed to obtain the processing result of the target operator, and the processing result of the target operator is determined as the node data of the second node, wherein the first node is any node in the target image processing model; If the node data of the last node in the target image processing model is obtained, the node data of the last node is determined as the second image corresponding to the first image.

9. The method according to claim 8, characterized in that, There are multiple target operators between the first node and the second node. The step of processing the node data of the first node based on the target operators between the first node and the second node in the target image processing model to obtain the processing result of the target operators, and determining the processing result of the target operators as the node data of the second node, includes: Based on multiple target operators between the first node and the second node, the node data of the first node is processed respectively to obtain the processing results of the multiple target operators; Based on the second weight parameters of the multiple target operators, the processing results of the multiple target operators are weighted and summed, and the processing result obtained by weighted summation is determined as the node data of the second node.

10. The method according to claim 1, characterized in that, The target image processing model includes n downsampling networks and n upsampling networks. Each downsampling network and each upsampling network includes multiple nodes and a target operator between every two nodes. Calling the target image processing model to process any image includes: The first image is input into the first downsampling network, and the first image is downsampled based on the first downsampling network to obtain the first downsampled feature; Based on the k-th downsampling network, the downsampling features output by the previous downsampling network are downsampled to obtain the k-th downsampling feature, until the n-th downsampling network outputs the n-th downsampling feature; Perform an identity mapping on the nth downsampled feature to obtain the nth mapped feature; The nth mapping feature is input into the first upsampling network, and the nth mapping feature is upsampled based on the first upsampling network to obtain the first upsampled feature; Based on the k-th upsampling network, the upsampling features output by the previous upsampling network and the (n+1-k)-th mapping features are upsampled to obtain the k-th upsampling feature, until the second image corresponding to the first image output by the n-th upsampling network is obtained; Wherein, n is a positive integer greater than 1, and k is any positive integer greater than 1 and not greater than n.

11. The method according to claim 9 or 10, characterized in that, The method further includes: Obtain the peak signal-to-noise ratio (PSNR) between the first image and the second image, whereby the PSNR indicates whether the second image is distorted; or... Obtain the structural similarity between the first image and the second image, wherein the structural similarity is used to represent the similarity between the first image and the second image.

12. An image processing apparatus, characterized in that, The device includes: The model acquisition module is used to acquire the original image processing model, which includes multiple nodes, multiple operators and mapping functions between every two nodes; The weight acquisition module is used to map the first weight parameters of each operator in the original image processing model to second weight parameters based on the mapping function after the first weight parameters of each operator are obtained through training. In the mapping process, any two first weight parameters among the multiple first weight parameters do not affect each other, so that the second weight parameters of the multiple operators between each two nodes are discretely distributed. The operator selection module is used to select the target operator between every two nodes based on the second weight parameter of each operator; The model building module is used to build a target image processing model based on the multiple nodes and the target operators between every two nodes; The image processing module is used to call the target image processing model to process any image.

13. A computer device, characterized in that, The computer device includes a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the image processing method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The storage medium stores at least one piece of program code, which is executed by a processor in a computer device to implement the image processing method as described in any one of claims 1 to 11.

15. A chip, characterized in that, The chip includes programmable logic circuitry and / or program instructions, which, when the chip is run on a computer device, are used to implement the image processing method as described in any one of claims 1 to 11.

Images