An image processing method, apparatus and device

Abstract

The application provides an image processing method, device and equipment, the method comprising: obtaining input features corresponding to a to-be-processed image; determining a shift operation feature based on the input features, determining a limited range feature based on the input features and the shift operation feature; determining a mapping feature corresponding to the to-be-processed image based on the shift operation feature and the limited range feature; performing a normalization operation based on the mapping feature to obtain a normalized feature, and determining output features corresponding to the to-be-processed image based on the normalized feature; and determining an image processing result corresponding to the to-be-processed image based on the output features. Through the technical solution of the application, the computational complexity can be reduced, the amount of calculation is small, and the resource consumption is small.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to an image processing method, apparatus, and device. Background Technology

[0002] Machine learning is a pathway to achieving artificial intelligence. It is an interdisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. Machine learning studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning places great emphasis on algorithm design, enabling computers to automatically learn patterns from data and use these patterns to predict unknown data.

[0003] Machine learning has been widely used in fields such as deep learning, data mining, computer vision, natural language processing, biometrics, search engines, medical diagnosis, speech and handwriting recognition, etc.

[0004] In order to use machine learning to achieve artificial intelligence processing, the server needs to acquire a large amount of sample data, train a machine learning model based on this sample data, and deploy the machine learning model to the terminal device (such as a camera) so that the terminal device can perform artificial intelligence processing based on the machine learning model.

[0005] However, because the processing of machine learning models involves a large number of complex calculations, such as exponential operations, the computational load and resource consumption of machine learning models are large. Summary of the Invention

[0006] This application provides an image processing method, the method comprising:

[0007] Obtain the input features corresponding to the image to be processed;

[0008] Based on the input features, a shift operation feature is determined; based on the input features and the shift operation feature, a defined range feature is determined; based on the shift operation feature and the defined range feature, a mapping feature corresponding to the image to be processed is determined; based on the mapping feature, a normalization operation is performed to obtain a normalized feature; and based on the normalized feature, an output feature corresponding to the image to be processed is determined.

[0009] The image processing result corresponding to the image to be processed is determined based on the output features.

[0010] This application provides an image processing apparatus, the apparatus comprising:

[0011] The acquisition module is used to acquire the input features corresponding to the image to be processed;

[0012] The determination module is used to determine shift operation features based on the input features, determine a defined range feature based on the input features and the shift operation features, determine a mapping feature corresponding to the image to be processed based on the shift operation features and the defined range feature, perform a normalization operation based on the mapping feature to obtain a normalized feature, and determine an output feature corresponding to the image to be processed based on the normalized feature.

[0013] The processing module is used to determine the image processing result corresponding to the image to be processed based on the output features.

[0014] This application provides an image processing apparatus, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the image processing method disclosed in the above example.

[0015] As can be seen from the above technical solutions, in this embodiment, the exponentiation operation of e can be replaced by shift operations and linear operations. That is, the exponentiation of e is approximated as a combination of shift operations (powers of 2) and linear operations, which greatly reduces resource overhead. Furthermore, by introducing piecewise linear operations, the accuracy of the approximation is improved. The above method can reduce computational complexity, reduce computational load, and reduce resource consumption, thereby reducing chip resource overhead and improving the performance of the target network model. It can be widely used in various business scenarios of artificial intelligence, such as image classification, object detection, segmentation, and pose estimation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0017] Figure 1 This is a flowchart illustrating an image processing method according to one embodiment of this application;

[0018] Figure 2A This is a schematic diagram of the classification network structure in one embodiment of this application;

[0019] Figure 2B This is a schematic diagram of the detection network structure in one embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the target network model in one embodiment of this application;

[0021] Figure 4 This is a flowchart illustrating an image processing method according to one embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the processing procedure of the target network layer in one embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the target network model in one embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the structure of an image processing apparatus according to one embodiment of this application;

[0025] Figure 8 This is a hardware structure diagram of an image processing device according to one embodiment of this application. Detailed Implementation

[0026] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0027] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0028] This application proposes an image processing method that can be applied to any type of device, such as a terminal device (e.g., a camera) that needs to deploy a target network model; there are no limitations on this. See also Figure 1 The diagram shown is a flowchart of the image processing method, which may include:

[0029] Step 101: Obtain the input features corresponding to the image to be processed.

[0030] Step 102: Determine the shift operation feature based on the input feature; determine the limited range feature based on the input feature and the shift operation feature; determine the mapping feature corresponding to the image to be processed based on the shift operation feature and the limited range feature; perform a normalization operation based on the mapping feature to obtain the normalized feature; and determine the output feature corresponding to the image to be processed based on the normalized feature.

[0031] Step 103: Determine the image processing result corresponding to the image to be processed based on the output feature.

[0032] For example, determining the mapping features corresponding to the image to be processed based on the shift operation feature and the defined range feature may include, but is not limited to: determining the exponential operation value based on the shift operation feature; determining the linear operation value based on the defined range feature; and determining the mapping features based on the exponential operation value and the linear operation value.

[0033] For example, determining the linear operation value based on the limited range feature may include, but is not limited to: determining the linear operation value using a K-piecewise linear function based on the limited range feature, a preset threshold, and the calibrated target parameter value, where K can be a positive integer greater than 1, such as 2, 3, etc.

[0034] In one possible implementation, based on the defined range characteristics, a preset threshold, and the calibrated target parameter value, the linear operation value is determined using a K-piecewise linear function, which may include, but is not limited to:

[0035] When K is 2, the linear operation value is determined using the following formula:

[0036]

[0037] Where s represents the linear operation value, a1, b1, a2, b2 represent the target parameter values, p represents the limited range feature, and 0.5*ln2 represents the preset threshold.

[0038] Alternatively, when K is 2, the linear operation value can be determined using the following formula:

[0039]

[0040] Where s represents the linear operation value, a1, b1, a2, b2 represent the target parameter values, p represents the limited range feature, and 0.3*ln2 represents the preset threshold.

[0041] Alternatively, when K is 3, the linear operation value can be determined using the following formula:

[0042]

[0043] Where s represents the linear operation value, a1, b1, a2, b2, a3, b3 represent the target parameter values, p represents the limited range feature, and 0.33*ln2, 0.67*ln2, ln2 represent the preset thresholds.

[0044] In one possible implementation, the target network model may include multiple network layers, including a target network layer for implementing normalization. The network layers preceding the target network layer form a first sub-network, and the network layers following the target network layer form a second sub-network. Based on this:

[0045] For step 101, the image to be processed can be input into the first sub-network to obtain the input features corresponding to the image to be processed; for example, after obtaining the image to be processed, the first sub-network can process the image to be processed without any restrictions on the processing procedure to obtain the input features corresponding to the image to be processed.

[0046] Regarding step 102, after obtaining the input features corresponding to the image to be processed, the input features can be input to the target network layer. The target network layer determines the shift operation features based on the input features, determines the limited range features based on the input features and the shift operation features, determines the mapping features corresponding to the image to be processed based on the shift operation features and the limited range features, performs a normalization operation based on the mapping features to obtain normalized features, and determines the output features corresponding to the image to be processed based on the normalized features.

[0047] Regarding step 103, after obtaining the output features corresponding to the image to be processed, these output features can be input into the second sub-network to obtain the image processing result corresponding to the image to be processed. For example, after obtaining the output features, the second sub-network can process the output features to obtain the image processing result.

[0048] For example, the target network layer can be a sigmoid network layer in the target network model; or, the target network layer can be a softmax network layer in the target network model. Of course, the above sigmoid network layer and softmax network layer are just two examples of target network layers, and there is no limitation on the target network layer.

[0049] For example, in order to obtain the target network model, a trained network model can be obtained. After obtaining the trained network model, the e-exponential operator of the target network layer in the trained network model is replaced with the target type operator to obtain the target network model. Wherein, the target type operator is used to implement the functions of determining shift operation features based on input features, determining limited range features based on input features and shift operation features, and determining mapping features based on shift operation features and limited range features.

[0050] As can be seen from the above technical solutions, in this embodiment, the exponentiation operation of e can be replaced by shift operations and linear operations. That is, the exponentiation of e is approximated as a combination of shift operations (powers of 2) and linear operations, which greatly reduces resource overhead. Furthermore, by introducing piecewise linear operations, the accuracy of the approximation is improved. The above method can reduce computational complexity, reduce computational load, and reduce resource consumption, thereby reducing chip resource overhead and improving the performance of the target network model. It can be widely used in various business scenarios of artificial intelligence, such as image classification, object detection, segmentation, and pose estimation.

[0051] The technical solutions of the embodiments of this application will be described below in conjunction with specific application scenarios.

[0052] Before introducing the technical solution of this application, let's first introduce the concepts related to the embodiments of this application.

[0053] Machine learning: Machine learning is a pathway to achieving artificial intelligence. It studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Deep learning and neural networks are subcategories of machine learning. Deep learning is a process of using mathematical models to model specific problems in the real world to solve similar problems within that field. Neural networks are mathematical models that mimic the behavioral characteristics of animal neural networks, performing distributed parallel information processing. These neural networks rely on the complexity of the system, adjusting the connections between a large number of internal nodes to achieve the purpose of processing information. For ease of description, we will use the structure and function of neural networks as an example; other subcategories of machine learning have similar structures and functions.

[0054] Neural networks: Neural networks can include, but are not limited to, convolutional neural networks (CNN), recurrent neural networks (RNN), fully connected networks, etc. The structural units of neural networks can include, but are not limited to, convolutional layers (Conv), pooling layers (Pool), activation layers, fully connected layers (FC), etc.

[0055] In practical applications, neural networks can be constructed by combining one or more convolutional layers, one or more pooling layers, one or more activation layers, and one or more fully connected layers, depending on different needs.

[0056] In a convolutional layer, the input data features are enhanced by performing convolution operations on the convolution kernel. The convolution kernel can be an m*n matrix. The input data features of the convolutional layer are convolved with the convolution kernel to obtain the output data features of the convolutional layer. The convolution operation is actually a filtering process.

[0057] In the pooling layer, operations such as taking the maximum, minimum, and average values ​​of the input data features (such as the output of the convolutional layer) are performed. This utilizes the principle of local correlation to subsample the input data features, reducing the amount of processing and maintaining feature invariance. The pooling layer operation is actually a downsampling process.

[0058] In the activation layer, activation functions (such as nonlinear functions) can be used to map the features of the input data, thereby introducing nonlinear factors and enabling the neural network to enhance its expressive power through nonlinear combinations.

[0059] The activation function may include, but is not limited to, the ReLU (Rectified Linear Units) function, which is used to set features less than 0 to 0, while features greater than 0 remain unchanged.

[0060] In a fully connected layer, all data features input to the fully connected layer are processed in a fully connected manner to obtain a feature vector, which may include multiple data features.

[0061] Network Model: A network model can be pre-built, called the initial network model. The initial network model is a network model that has not been fully trained. The initial network model can be a machine learning model, such as a machine learning model based on deep learning or a machine learning model based on neural networks, without any restrictions.

[0062] The training process for the initial network model involves using sample data to train the initial network model, which is essentially the process of adjusting and optimizing the model weights (network parameters) in the initial network model. For example, the initial network model may include multiple network layers, each of which includes model weights. For a neural network-based initial network model, the model weights could be convolutional layer parameters (such as convolutional kernel parameters), pooling layer parameters, activation layer parameters, fully connected layer parameters, etc. During the training process of the initial network model, sample data can be used to adjust and optimize the model weights in multiple network layers of the initial network model.

[0063] After the initial network model is trained, a trained network model is obtained. The model weights in the trained network model are the adjusted and optimized model weights. Once the trained network model is obtained, it can be deployed to terminal devices, enabling the devices to perform artificial intelligence processing based on it. For example, the trained network model can be widely applied to various business scenarios of artificial intelligence, such as image classification, object detection, segmentation, and pose estimation. Taking object detection as an example, functions such as face detection, human detection, and vehicle detection can be implemented based on the trained network model. For instance, for face detection, an image including a face is input into the trained network model, which then performs artificial intelligence processing on the image, and the result is the face detection result. For vehicle detection, an image including a vehicle is input into the trained network model, which then performs artificial intelligence processing on the image, and the result is the vehicle detection result.

[0064] In one possible implementation, the initial network model includes an e-exponential operator. After training the initial network model to obtain a trained network model, the trained network model also includes an e-exponential operator. After the trained network model is deployed to the terminal device, when the terminal device performs artificial intelligence processing based on the trained network model, it needs to perform calculations based on the e-exponential operator. The calculation process of the e-exponential operator is a complex calculation, which will bring a relatively large resource overhead and requires dedicated resources to support it. As a result, the trained network model has a large amount of computation and high resource consumption.

[0065] To address the above findings, this application proposes an e-exponential approximation method. This method replaces the e-exponential operation with shift operations and linear operations, approximating the e-exponential as a combination of shift operations (powers of 2) and linear operations. This significantly reduces resource overhead, and the introduction of piecewise linear operations improves the accuracy of the approximation. This reduces computational complexity, minimizes computational load, and reduces resource consumption, thus reducing chip resource consumption. The e-exponential approximation method proposed in this embodiment can be widely applied to various artificial intelligence business scenarios, such as image classification, object detection, segmentation, and pose estimation.

[0066] In this embodiment, an initial network model can be pre-constructed, which may include multiple network layers. A training set can also be pre-constructed, which may include a large amount of sample data and the corresponding calibration data for each sample. The initial network model can be trained based on this training set. After training is complete, a trained network model can be obtained. After obtaining the trained network model, the e-exponential operator of the target network layer in the trained network model can be replaced with a target type operator to obtain the target network model, which can then be deployed to the terminal device.

[0067] For example, the multiple network layers of the trained network model may include a target network layer. The target network layer is a network layer used to implement the normalization function, that is, to normalize any value to a specified value range, such as the value range of 0-1. The target network layer used to implement the normalization function usually uses the e-exponential operation operator to implement the normalization function. In this embodiment, the e-exponential operation operator of the target network layer can be replaced with the target type operation operator to implement the normalization function.

[0068] For example, regarding the process of implementing normalization function through target type operation operator, the target type operation operator is used to implement the functions of determining shift operation features based on input features, determining limited range features based on input features and shift operation features, and determining mapping features based on shift operation features and limited range features. For the process of implementing normalization function for target type operation operator, please refer to the following embodiments.

[0069] For example, in artificial intelligence fields such as object detection, segmentation, and image classification, both the sigmoid function and the softmax function can use the e-exponential operator to achieve normalization. Therefore, the target network layer used to achieve normalization can be a sigmoid network layer and / or a softmax network layer.

[0070] For ease of distinction, a network layer that uses the sigmoid function can be called a sigmoid network layer. In a sigmoid network layer, the sigmoid function is used to map variables to the numerical range of 0-1.

[0071] For ease of distinction, the network layer that uses the softmax function can be called the softmax network layer. In the softmax network layer, the softmax function is also called the normalization exponential function. It is a generalization of the binary classification function to multi-class classification and is used to present the results of multi-class classification in the form of probabilities.

[0072] For example, in image classification tasks, the penultimate layer of a classification network often uses the softmax function to achieve classification. (See [link to relevant documentation]). Figure 2A The diagram shows the structure of a classification network; therefore, the penultimate network layer can be used as the target network layer. For image detection tasks, see [link to relevant documentation]. Figure 2B As shown, this is a schematic diagram of the detection network structure. The classification network layer often uses the sigmoid function or the softmax function to implement the classification function. Therefore, the classification network layer can be used as the target network layer.

[0073] In summary, in this embodiment, the target network model includes multiple network layers, including a target network layer for implementing normalization. This target network layer is either a sigmoid network layer or a softmax network layer within the target network model. Furthermore, the target network layer may include a target type operation operator for implementing normalization. This target type operation operator is used to determine shift operation features based on input features, determine limited range features based on the input features and shift operation features, and determine mapping features based on the shift operation features and limited range features.

[0074] In this embodiment, all network layers preceding the target network layer can be grouped into a first sub-network, and the network layers following the target network layer can be grouped into a second sub-network. Therefore, the target network model includes a first sub-network, a target network layer, and a second sub-network. The first sub-network includes at least one network layer, and the second sub-network includes at least one network layer. See also Figure 3 The diagram shows the structure of the target network model. The target network model can include network layer 1, network layer 2, network layer 3, network layer 4, and network layer 5. Here, we are only using 5 network layers as an example. The number of network layers in the target network model can be greater than 5. Assuming that network layer 4 is a softmax network layer, then network layer 4 can be used as the target network layer. In this case, the first sub-network can include network layer 1, network layer 2, and network layer 3, and the second sub-network can include network layer 5.

[0075] After obtaining the target network model, it can be deployed to the terminal device, where image processing is performed based on the target network model. The image processing process is essentially an artificial intelligence processing process based on the target network model. This application proposes an image processing method applicable to terminal devices; see [link to relevant documentation]. Figure 4 The diagram shown is a flowchart of the method, which may include:

[0076] Step 401: Obtain the input features corresponding to the image to be processed.

[0077] Exemplarily, after obtaining the image to be processed, the image to be processed can be input into the first sub-network of the target network model, and the first sub-network processes the image to be processed to obtain the input features corresponding to the image to be processed. For example, network layer 1, network layer 2, and network layer 3 process the image to be processed in sequence. There is no limitation on this processing process, and the input features corresponding to the image to be processed are obtained.

[0078] Step 402: Determine the shift operation feature based on the input feature corresponding to the image to be processed, and determine the limited range feature based on the input feature and the shift operation feature; determine the mapping feature corresponding to the image to be processed based on the shift operation feature and the limited range feature; perform a normalization operation based on the mapping feature to obtain a normalized feature, and determine the output feature corresponding to the image to be processed based on the normalized feature.

[0079] Exemplarily, after obtaining the input features corresponding to the image to be processed, the input features can be input into the target network layer, and the target network layer determines the shift operation feature based on the input features, determines the limited range feature based on the input feature and the shift operation feature, determines the mapping feature corresponding to the image to be processed based on the shift operation feature and the limited range feature, performs a normalization operation based on the mapping feature to obtain a normalized feature, and determines the output feature corresponding to the image to be processed based on the normalized feature.

[0080] In a possible implementation manner, for any x, when solving e x , it can be converted into a combination of e p within a limited range, 0 < p < ln2 and a shift operation: e x = 2 z e p = e zln2 e p . Based on the above principle, in this embodiment, for step 402, the following steps can be adopted to implement:

[0081]

[0082] Step 4021: Determine the shift operation feature based on the input feature.

[0083] For example, based on the input feature, the shift operation feature can be determined by using the following formula: In the above formula, x represents the input feature, and z represents the shift operation feature.

[0084]

[0085]

[0086] Step 4022: Determine the limited range feature based on the input feature and the shift operation feature.

[0087] For example, the limited range feature can be determined by using the following formula: p = x - z * ln2, 0 < p < ln2. In the above formula, x represents the input feature, z represents the shift operation feature, and p represents the limited range feature.

[0085] Step 4023: Determine the exponential operation value based on the shift operation characteristics.

[0086] For example, based on the characteristics of shift operations, the exponential operation value can be determined using the following formula: w = 2 z In the above formula, z represents the shift operation characteristic, and w represents the exponential operation value.

[0087] Step 4024: Determine the linear operation value based on the limited range characteristics.

[0088] For example, based on the limited range characteristics, the linear operation value can be determined using the following formula: s = e p In the above formula, p represents the range characteristic and s represents the linear operation value.

[0089] In one possible implementation, for e p Alternatively, piecewise linear functions can be used for fitting calculations. For example, based on the limited range features, preset thresholds and target parameter values, a K-piecewise linear function can be used to determine the linear operation value s, where K can be a positive integer greater than 1, such as 2, 3, etc.

[0090] For example, to determine the value s of a linear operation, it can be done in the following way:

[0091] Method 1: Given that the K-piecewise linear function is a 2-piecewise linear function, based on the defined range characteristics, preset threshold, and target parameter value, the linear operation value s can be determined using the following formula:

[0092]

[0093] In the above formula, s represents the linear operation value, a1, b1, a2, and b2 represent the target parameter values, p represents the limited range feature, and 0.5 * ln2 represents the preset threshold. The values ​​of a1, b1, a2, and b2 can be determined by pre-obtaining multiple sets of calibration data, each set consisting of p and s. For example, for a value of p, s = e p Calculate the s value corresponding to this p value, and this p value and s value constitute a set of calibration data. Substitute multiple sets of calibration data into the above formula to calculate the values ​​of a1, b1, a2, and b2.

[0094] Method 2: Given that the K-piecewise linear function is a 2-piecewise linear function, based on the defined range characteristics, preset threshold, and target parameter value, the linear operation value s can be determined using the following formula:

[0095]

[0096] In the above formula, s represents the linear operation value, a1, b1, a2, and b2 represent the target parameter values, p represents the limited range feature, and 0.3 * ln2 represents the preset threshold. The values ​​of a1, b1, a2, and b2 can be determined by pre-obtaining multiple sets of calibration data, each set consisting of p and s. For example, for a value of p, s = e p Calculate the s value corresponding to this p value, and this p value and s value constitute a set of calibration data. Substitute multiple sets of calibration data into the above formula to calculate the values ​​of a1, b1, a2, and b2.

[0097] Method 3: When the K-piecewise linear function is a 3-piecewise linear function, based on the limited range characteristics, preset threshold, and target parameter value, the linear operation value s can be determined using the following formula:

[0098]

[0099] In the above formula, s represents the linear operation value, a1, b1, a2, b2, a3, b3 represent the target parameter values, p represents the limited range feature, and 0, 0.33*ln2, 0.67*ln2, ln2 all represent preset thresholds.

[0100] Regarding the values ​​of a1, b1, a2, b2, a3, and b3, multiple sets of calibration data can be obtained in advance. Each set of calibration data consists of p and s. For example, for a value of p, the values ​​can be determined by s = e p Calculate the s value corresponding to this p value, and this p value and s value constitute a set of calibration data. Based on this, multiple sets of calibration data can be substituted into the above formula to calculate the values ​​of a1, b1, a2, b2, a3, and b3.

[0101] Of course, the above three methods are just a few examples of determining the linear operation value s. Under a 2-piece piecewise linear function, different preset thresholds correspond to different determination methods. There are no restrictions on these preset thresholds; they can be configured based on experience. Under a 3-piece piecewise linear function, different preset thresholds correspond to different determination methods. There are no restrictions on these preset thresholds; they can be configured based on experience. Of course, a K-piece piecewise linear function can also be a 4-piece piecewise linear function, a 5-piece piecewise linear function, etc. The implementation principle is similar; different preset thresholds can be designed.

[0102] Step 4025: Determine the mapping characteristics based on the exponential and linear operation values.

[0103] For example, based on the exponential and linear operation values, the mapping characteristics can be determined using the following formula: k = w * s = 2 z e p w represents the exponential operand value, and s represents the linear operand value.

[0104] The following combination Figure 5 The processing steps 4021-4025 are explained below. First, the shift operation feature z and the limited range feature p are determined based on the input feature x. Then, the exponential operation value w is determined based on the shift operation feature z. Based on the limited range feature p, a piecewise linear function is used for fitting calculation to obtain the linear operation value s. Finally, the mapping feature can be calculated based on the exponential operation value w and the linear operation value s.

[0105] Step 4026: Perform a normalization operation based on the mapping feature to obtain the normalized feature.

[0106] For example, after obtaining the mapping feature, a normalization operation can be performed based on the mapping feature to obtain the normalized feature. The normalized feature can be a value located in a specified numerical range, such as a value located in the numerical range of 0-1. In this embodiment, the process of this normalization operation is not limited.

[0107] Step 4027: Determine the output features corresponding to the image to be processed based on the normalized features.

[0108] For example, after obtaining the normalized features, the target network layer can process the image based on these normalized features without any restrictions on the processing procedure, thereby obtaining the output features corresponding to the image to be processed.

[0109] For step 403, the image processing result corresponding to the image to be processed is determined based on the output feature.

[0110] For example, after obtaining the output features corresponding to the image to be processed, these output features can be input into the second sub-network to obtain the image processing result corresponding to the image to be processed. For instance, after obtaining the output features, the second sub-network can process the output features to obtain the image processing result.

[0111] See Figure 6 The diagram shows the structure of the target network model. The network layer using the softmax function is the target network layer. Based on the structure of the target network model, the processing of the target network model is explained.

[0112] After obtaining the image to be processed, a first convolutional layer can be used to perform convolution operations on the image to obtain the first convolutional feature. A second convolutional layer can then be used to perform convolution operations on the image to obtain the second convolutional feature. Finally, a third convolutional layer can be used to perform convolution operations on the image to obtain the third convolutional feature. Based on the first and second convolutional features, a first mul network layer can be used to perform a mul operation on the first and second convolutional features to obtain the input features for the softmax network layer.

[0113] After obtaining the input features, these features can be fed into the softmax network layer, which is the target network layer in the above embodiment. Based on these input features, the softmax network layer can use the processing flow of steps 4021-4027 to obtain the output features corresponding to the image to be processed.

[0114] Based on the output features and third convolutional features corresponding to the image to be processed, the output features and third convolutional features can be subjected to mul operation through the second mul network layer to obtain mul operation features. The mul operation features are then input into the FFN network layer, which processes the mul operation features to obtain the image processing result (i.e., the artificial intelligence processing result) corresponding to the image to be processed, thus completing the image processing process.

[0115] As can be seen from the above technical solutions, in this embodiment, the exponentiation operation of e can be replaced by shift operations and linear operations. That is, the exponentiation of e is approximated as a combination of shift operations (powers of 2) and linear operations, which greatly reduces resource overhead. Furthermore, by introducing piecewise linear operations, the accuracy of the approximation is improved. The above method can reduce computational complexity, reduce computational load, and reduce resource consumption, thereby reducing chip resource overhead and improving the performance of the target network model. It can be widely used in various business scenarios of artificial intelligence, such as image classification, object detection, segmentation, and pose estimation.

[0116] Based on the same concept as the above method, this application proposes an image processing apparatus, see [link to previous application]. Figure 7 The diagram shown is a structural schematic of the device, which may include:

[0117] The acquisition module 71 is used to acquire the input features corresponding to the image to be processed;

[0118] The determining module 72 is used to determine shift operation features based on the input features, determine a defined range feature based on the input features and the shift operation features, determine the mapping features corresponding to the image to be processed based on the shift operation features and the defined range features, perform a normalization operation based on the mapping features to obtain normalized features, and determine the output features corresponding to the image to be processed based on the normalized features.

[0119] The processing module 73 is used to determine the image processing result corresponding to the image to be processed based on the output features.

[0120] In one possible implementation, when the determining module 72 determines the mapping feature corresponding to the image to be processed based on the shift operation feature and the defined range feature, it is specifically used to: determine an exponential operation value based on the shift operation feature; and determine a linear operation value based on the defined range feature; and determine the mapping feature based on the exponential operation value and the linear operation value.

[0121] For example, when determining the linear operation value based on the defined range feature, the determining module 72 is specifically used to: determine the linear operation value using a K-piece linear function based on the defined range feature, a preset threshold, and the calibrated target parameter value, where K is a positive integer greater than 1.

[0122] For example, when the determining module 72 determines the linear operation value using a K-piece linear function based on the defined range features, the preset threshold, and the calibrated target parameter value, it is specifically used for:

[0123] When K is 2, the value of the linear operation is determined using the following formula:

[0124]

[0125] Wherein, s represents the linear operation value, a1, b1, a2, b2 represent the target parameter values, p represents the limited range feature, and 0.5*ln2 represents the preset threshold.

[0126] Alternatively, when K is 2, the value of the linear operation can be determined using the following formula:

[0127]

[0128] Where s represents the linear operation value, a1, b1, a2, b2 represent the target parameter values, p represents the limited range feature, and 0.3*ln2 represents the preset threshold.

[0129] Alternatively, when K is 3, the linear operation value can be determined using the following formula:

[0130]

[0131] Wherein, s represents the linear operation value, a1, b1, a2, b2, a3, b3 represent the target parameter values, p represents the limited range feature, and 0.33*ln2, 0.67*ln2, ln2 represent the preset thresholds.

[0132] For example, the target network model includes multiple network layers, including a target network layer for implementing normalization. The network layers before the target network layer form a first sub-network, and the network layers after the target network layer form a second sub-network. The acquisition module 71 is specifically used to: input the image to be processed into the first sub-network to obtain the input features corresponding to the image to be processed; the determination module 72 is specifically used to: input the input features into the target network layer, whereby the target network layer determines shift operation features based on the input features, determines a defined range feature based on the input features and the shift operation features, determines a mapping feature corresponding to the image to be processed based on the shift operation features and the defined range feature, performs a normalization operation based on the mapping features to obtain normalized features, and determines the output features corresponding to the image to be processed based on the normalized features; the processing module 73 is specifically used to: input the output features into the second sub-network to obtain the image processing result corresponding to the image to be processed.

[0133] For example, the target network layer is the sigmoid network layer in the target network model; or, the target network layer is the softmax network layer in the target network model.

[0134] For example, the processing module 73 is further configured to, after obtaining the trained network model, replace the e-exponential operator of the target network layer in the trained network model with a target type operator to obtain the target network model; wherein: the target type operator is used to implement the functions of determining shift operation features based on input features, determining limited range features based on input features and shift operation features, and determining mapping features based on shift operation features and limited range features.

[0135] Based on the same concept as the methods described above, this application proposes an image processing device, see [link to relevant documentation]. Figure 8 As shown, the image processing apparatus includes a processor 81 and a machine-readable storage medium 82, the machine-readable storage medium 82 storing machine-executable instructions that can be executed by the processor 81; the processor 81 is used to execute the machine-executable instructions to implement the image processing method disclosed in the above example.

[0136] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the image processing method disclosed in the above examples of this application.

[0137] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0138] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0139] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0140] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the input features corresponding to the image to be processed; Based on the input features, shift operation features are determined; based on the input features and the shift operation features, a defined range feature is determined; based on the shift operation features and the defined range feature, a mapping feature corresponding to the image to be processed is determined; based on the mapping feature, a normalization operation is performed to obtain normalized features; and based on the normalized features, an output feature corresponding to the image to be processed is determined; wherein, , , This represents the input feature. This indicates the shift operation feature. This indicates the defined range feature; The image processing result corresponding to the image to be processed is determined based on the output features.

2. The method according to claim 1, characterized in that, The step of determining the mapping features corresponding to the image to be processed based on the shift operation features and the defined range features includes: The exponential operation value is determined based on the shift operation characteristics; wherein... , Indicates the value of the exponential operation; The linear operation value is determined based on the defined range characteristics; wherein... , Indicates the value of a linear operation; The mapping feature is determined based on the exponential operation value and the linear operation value.

3. The method according to claim 2, characterized in that, The determination of the linear operation value based on the defined range feature includes: Based on the defined range features, the preset threshold, and the calibrated target parameter values, the linear operation value is determined using a K-piece linear function, where K is a positive integer greater than 1.

4. The method according to claim 3, characterized in that, The step of determining the linear operation value using a K-piecewise linear function based on the defined range features, preset threshold, and calibrated target parameter values ​​includes: When K is 2, the value of the linear operation is determined using the following formula: in, This represents the value of the linear operation. , , , This represents the target parameter value. This indicates the defined range feature. This represents the preset threshold; Alternatively, when K is 2, the value of the linear operation can be determined using the following formula: in, This represents the value of the linear operation. , , , This represents the target parameter value. This indicates the defined range feature. This represents the preset threshold; Alternatively, when K is 3, the linear operation value can be determined using the following formula: in, This represents the value of the linear operation. , , , , , This represents the target parameter value. This indicates the defined range feature. , , This refers to the preset threshold.

5. The method according to any one of claims 1-4, characterized in that, The target network model includes multiple network layers, including a target network layer for implementing normalization. The network layers preceding the target network layer form a first sub-network, and the network layers following the target network layer form a second sub-network. The method specifically includes: The image to be processed is input into the first sub-network to obtain the input features corresponding to the image to be processed; The input features are input to the target network layer, which determines shift operation features based on the input features, determines a defined range feature based on the input features and the shift operation feature, determines the mapping feature corresponding to the image to be processed based on the shift operation feature and the defined range feature, performs a normalization operation based on the mapping feature to obtain normalized features, and determines the output feature corresponding to the image to be processed based on the normalized features. The output features are input into the second sub-network to obtain the image processing result corresponding to the image to be processed.

6. The method according to claim 5, characterized in that, The target network layer is the sigmoid network layer in the target network model; or... The target network layer is the softmax network layer in the target network model.

7. The method according to claim 5, characterized in that, After obtaining the trained network model, the e-exponential operator of the target network layer in the trained network model is replaced with the target type operator to obtain the target network model; wherein: the target type operator is used to implement the functions of determining shift operation features based on input features, determining limited range features based on input features and shift operation features, and determining mapping features based on shift operation features and limited range features.

8. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the input features corresponding to the image to be processed; The determining module is configured to: determine shift operation features based on the input features; determine a defined range feature based on the input features and the shift operation feature; determine a mapping feature corresponding to the image to be processed based on the shift operation feature and the defined range feature; perform a normalization operation based on the mapping feature to obtain a normalized feature; and determine an output feature corresponding to the image to be processed based on the normalized feature. , , This represents the input feature. This indicates the shift operation feature. This indicates the defined range feature; The processing module is used to determine the image processing result corresponding to the image to be processed based on the output features.

9. The apparatus according to claim 8, characterized in that, in, When the determining module determines the mapping features corresponding to the image to be processed based on the shift operation features and the limited range features, it is specifically used to: determine the exponential operation value based on the shift operation features; determine the linear operation value based on the limited range features; and determine the mapping features based on the exponential operation value and the linear operation value; wherein... , Indicates the exponential operation value; where, , Indicates the value of a linear operation; Specifically, when the determining module determines the linear operation value based on the defined range features, it is used to: determine the linear operation value using a K-piece linear function based on the defined range features, a preset threshold, and the calibrated target parameter value, where K is a positive integer greater than 1; Specifically, when the determining module determines the linear operation value using a K-piece linear function based on the defined range features, a preset threshold, and the calibrated target parameter value, it is used for: When K is 2, the value of the linear operation is determined using the following formula: in, This represents the value of the linear operation. , , , This represents the target parameter value. This indicates the defined range feature. This represents the preset threshold; Alternatively, when K is 2, the value of the linear operation can be determined using the following formula: in, This represents the value of the linear operation. , , , This represents the target parameter value. This indicates the defined range feature. This represents the preset threshold; Alternatively, when K is 3, the linear operation value can be determined using the following formula: in, This represents the value of the linear operation. , , , , , This represents the target parameter value. This indicates the defined range feature. , , This represents the preset threshold; The target network model includes multiple network layers, including a target network layer for implementing normalization. The network layers preceding the target network layer form a first sub-network, and the network layers following the target network layer form a second sub-network. The acquisition module is specifically used to: input the image to be processed into the first sub-network to obtain input features corresponding to the image to be processed; the determination module is specifically used to: input the input features into the target network layer, whereby the target network layer determines shift operation features based on the input features, determines a defined range feature based on the input features and the shift operation feature, determines a mapping feature corresponding to the image to be processed based on the shift operation feature and the defined range feature, performs a normalization operation based on the mapping feature to obtain normalized features, and determines the output features corresponding to the image to be processed based on the normalized features; the processing module is specifically used to: input the output features into the second sub-network to obtain the image processing result corresponding to the image to be processed; Wherein, the target network layer is the sigmoid network layer in the target network model; or, the target network layer is the softmax network layer in the target network model; The processing module is further configured to, after obtaining the trained network model, replace the e-exponential operator of the target network layer in the trained network model with a target type operator to obtain the target network model; wherein: the target type operator is used to implement the functions of determining shift operation features based on input features, determining limited range features based on input features and shift operation features, and determining mapping features based on shift operation features and limited range features.

10. An image processing device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-7.

Images