Image processing method and apparatus

Abstract

The present disclosure provides an image processing method and device, which can be applied to the technical field of semiconductor devices and integrated circuits. The image processing method comprises: determining a first number of intermediate feature maps according to a first feature map corresponding to a to-be-processed image, wherein the first feature map comprises a plurality of feature elements; determining an intermediate feature element of each intermediate feature map according to a second number of candidate feature elements of each intermediate feature map; determining a target feature element of each intermediate feature map according to a target threshold and the intermediate feature element of each intermediate feature map; and determining an image processing result corresponding to the to-be-processed image according to the target feature element of each intermediate feature map.

Description

Technical Field

[0001] This disclosure relates to the field of semiconductor devices and integrated circuit technology, and more specifically, to an image processing method and apparatus. Background Technology

[0002] With the development of computer technology, Convolutional Neural Networks (CNNs) have achieved excellent results in fields such as image classification, object tracking, and image segmentation, and have broad application prospects.

[0003] However, due to the enormous computational and parameter requirements of convolutional neural networks (CNNs), traditional general-purpose processors struggle to achieve real-time inference, and the energy consumption required for computation is also high. Many applications, such as the Internet of Things (IoT) and autonomous driving, demand real-time response and have limited power resources, placing stringent requirements on energy efficiency. Therefore, effectively reducing the energy consumption and latency overhead of neural network inference is a pressing technical challenge. Summary of the Invention

[0004] In view of this, the present disclosure provides an image processing method and apparatus.

[0005] According to one aspect of this disclosure, an image processing method is provided, comprising: determining a first number of intermediate feature maps based on a first feature map corresponding to an image to be processed, wherein the first feature map includes a plurality of feature elements; determining an intermediate feature element for each of the intermediate feature maps based on a second number of candidate feature elements for each of the intermediate feature maps; determining a target feature element for each of the intermediate feature maps based on a target threshold and the intermediate feature element for each of the intermediate feature maps; and determining an image processing result corresponding to the image to be processed based on the target feature element for each of the intermediate feature maps.

[0006] According to embodiments of this disclosure, the image processing method further includes, before determining a first number of intermediate feature maps based on a first feature map corresponding to the image to be processed: obtaining a trained convolutional neural network, wherein the trained convolutional neural network includes S max-pooling layers, where S is a positive integer; for each max-pooling layer, determining a candidate threshold and a candidate sliding window size corresponding to the max-pooling layer; and associating and storing the candidate level identifiers of each of the S max-pooling layers, the candidate thresholds and candidate sliding window sizes corresponding to each of the S max-pooling layers, in a data source.

[0007] According to an embodiment of this disclosure, the first feature map is obtained by processing a max pooling layer corresponding to the target level identifier; determining a first number of intermediate feature maps based on the first feature map corresponding to the image to be processed includes: determining a target sliding window size among the S candidate sliding window sizes based on the target level identifier and the S candidate level identifiers; and dividing the first feature map based on the target sliding window size to obtain the first number of intermediate feature maps.

[0008] According to an embodiment of this disclosure, the intermediate feature map includes P*Q feature elements, where P and Q are both positive integers; determining the intermediate feature element of each intermediate feature map based on the second number of candidate feature elements of each intermediate feature map includes: for each intermediate feature map, determining the second number of candidate feature elements among the P*Q feature elements, wherein the second number is less than or equal to P*Q; and determining the candidate feature element with the largest value among the second number of candidate feature elements as the intermediate feature element.

[0009] According to an embodiment of this disclosure, the first feature map is obtained by processing a max pooling layer corresponding to the target level identifier; the method further includes, before determining the target feature element of each intermediate feature map based on the target threshold and the intermediate feature element of each intermediate feature map: determining the target threshold from the S candidate thresholds based on the target level identifier and the S candidate level identifiers.

[0010] According to an embodiment of this disclosure, determining the target feature element of each intermediate feature map based on the target threshold and the intermediate feature element of each intermediate feature map includes: determining a first comparison result for each intermediate feature map based on the target threshold and the intermediate feature element of the intermediate feature map; and determining the target feature element of the intermediate feature map based on the first comparison result.

[0011] According to an embodiment of this disclosure, determining the target feature element of the intermediate feature map based on the first comparison result includes: when the first comparison result indicates that the value of the intermediate feature element is greater than the target threshold, the intermediate feature element is determined as the target feature element of the intermediate feature map.

[0012] According to an embodiment of the present disclosure, the image processing method further includes: when the first comparison result indicates that the value of the intermediate feature element is less than or equal to the target threshold, determining a third number of remaining feature elements of the intermediate feature map based on P*Q feature elements of the intermediate feature map and the second number of candidate feature elements; determining a second comparison result based on the third number of remaining feature elements and the intermediate feature elements; and determining a target feature element of the intermediate feature map based on the second comparison result.

[0013] According to an embodiment of this disclosure, the second comparison result includes a fourth number of feature elements arranged in descending order; determining the target feature element of the intermediate feature map based on the second comparison result includes: determining the element with the largest value among the fourth number of feature elements as the target feature element of the intermediate feature map.

[0014] According to embodiments of this disclosure, determining the image processing result corresponding to the image to be processed based on the target feature elements of each of the intermediate feature maps includes: determining a second feature map corresponding to the image to be processed based on the target feature elements of each of the intermediate feature maps; and determining the image processing result based on the second feature map.

[0015] According to another aspect of this disclosure, an image processing apparatus is provided, comprising: a first determining module, configured to determine a first number of intermediate feature maps based on a first feature map corresponding to an image to be processed, wherein the first feature map includes a plurality of feature elements; a second determining module, configured to determine an intermediate feature element for each of the intermediate feature maps based on a second number of candidate feature elements for each of the intermediate feature maps; a third determining module, configured to determine a target feature element for each of the intermediate feature maps based on a target threshold and the intermediate feature element for each of the intermediate feature maps; and a fourth determining module, configured to determine an image processing result corresponding to the image to be processed based on the target feature element for each of the intermediate feature maps.

[0016] According to another aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more instructions, wherein, when executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described in this disclosure.

[0017] According to another aspect of this disclosure, a computer-readable storage medium is provided having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described in this disclosure.

[0018] According to another aspect of this disclosure, a computer program product is provided, which includes computer-executable instructions that, when executed, are used to perform the methods described in this disclosure.

[0019] According to embodiments of this disclosure, since the intermediate feature elements are determined based on a second number of candidate feature elements for each intermediate feature map, and the candidate feature elements are determined based on the intermediate feature map, which in turn is determined based on a first feature map corresponding to the image to be processed, redundant computations during max pooling can be reduced, thus improving the efficiency of intermediate feature element determination. Furthermore, since the target feature elements are determined based on a target threshold and the intermediate feature elements for each intermediate feature map, by employing a hardware-software co-design approach and a threshold comparison-based judgment method, redundant computations during max pooling can be further reduced, allowing the target feature elements of the max pooling operation to be obtained earlier. Therefore, compared to traditional pooling accelerators, this reduces inference energy consumption and latency overhead, thereby improving the efficiency of subsequent image processing. Attached Figure Description

[0020] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 A flowchart illustrating an image processing method according to an embodiment of the present disclosure is shown schematically.

[0022] Figure 2 This illustration schematically shows an example of a process for obtaining an intermediate feature map according to an embodiment of the present disclosure;

[0023] Figure 3 This schematic diagram illustrates an example of a process for obtaining intermediate feature elements according to an embodiment of the present disclosure;

[0024] Figure 4 This illustration schematically shows an example diagram of a process for obtaining target feature elements according to an embodiment of the present disclosure;

[0025] Figures 5A-5C This schematically illustrates an example of a max-pooling operator acceleration process according to an embodiment of the present disclosure;

[0026] Figure 6 An example schematic diagram of a max-pooling operator acceleration device according to an embodiment of the present disclosure is shown;

[0027] Figure 7 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is schematically shown; and

[0028] Figure 8A block diagram of an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0029] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0032] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0033] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0034] For the max pooling operator, related technologies mainly employ dedicated acceleration circuits for convolutional neural networks to implement the various operations required for max pooling. However, from a hardware perspective, there is a lack of methods for co-designing hardware and software for the max pooling operator, thus limiting the acceleration effect on max pooling.

[0035] To at least partially address the technical problems existing in related technologies, this disclosure provides an image processing method and apparatus that can be applied to the fields of semiconductor devices and integrated circuit technology. The image processing method includes: determining a first number of intermediate feature maps based on a first feature map corresponding to an image to be processed, wherein the first feature map includes multiple feature elements; determining intermediate feature elements for each intermediate feature map based on a second number of candidate feature elements for each intermediate feature map; determining a target feature element for each intermediate feature map based on a target threshold and the intermediate feature elements for each intermediate feature map; and determining an image processing result corresponding to the image to be processed based on the target feature elements for each intermediate feature map.

[0036] It should be noted that the sequence numbers of the operations in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0037] Figure 1 A flowchart illustrating an image processing method according to an embodiment of the present disclosure is shown schematically.

[0038] like Figure 1 As shown, the image processing method 100 includes operations S110 to S140.

[0039] In operation S110, a first number of intermediate feature maps are determined based on the first feature map corresponding to the image to be processed, wherein the first feature map includes multiple feature elements.

[0040] In operation S120, the intermediate feature elements of each intermediate feature map are determined based on the second number of candidate feature elements of each intermediate feature map.

[0041] In operation S130, the target feature element of each intermediate feature map is determined based on the target threshold and the intermediate feature element of each intermediate feature map.

[0042] In operation S140, the image processing result corresponding to the image to be processed is determined based on the target feature elements of each intermediate feature map.

[0043] According to embodiments of this disclosure, the file format of the image to be processed can be configured according to actual business needs, and is not limited herein. For example, the file format of the image to be processed may include at least one of the following: JPG (Joint Photographic Experts Group), TIFF (Tag Image File Format), PNG (Portable Network Graphics), PDF (Portable Document Format), and GIF (Graphics Interchange Format), etc.

[0044] According to embodiments of this disclosure, after obtaining the image to be processed, the image can be processed based on a preset model to obtain a first feature map corresponding to the image to be processed. The preset model can be configured according to actual business needs and is not limited herein. For example, the preset model may include at least one of the following: Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). The preset model may include at least one max-pooling layer, and each max-pooling layer may have its own corresponding layer label. The aforementioned first feature map may correspond to any one of the at least one max-pooling layers.

[0045] According to embodiments of this disclosure, the first feature map may include multiple feature elements, each feature element containing information about the spatial location of the first feature map and the intensity or response level of the feature at the corresponding location. After obtaining the first feature map, the target sliding window size of the max pooling layer corresponding to the first feature map can be determined, and based on the target sliding window size, the first feature map is divided to obtain a first number of intermediate feature maps. For example, if the first feature map includes 4*4 feature elements and the target sliding window size is 2*2, then four intermediate feature maps, each including 2*2 feature elements, can be obtained.

[0046] According to embodiments of this disclosure, an intermediate feature map may include multiple feature elements, and the number of feature elements in the intermediate feature map is less than the number of feature elements in the first feature map. For each intermediate feature map, a second number of candidate feature elements can be determined from the multiple feature elements corresponding to that intermediate feature map. The second number of candidate feature elements can be understood as a subset of the feature elements in the intermediate feature map, and the second number is less than the number of feature elements in the intermediate feature map. For example, if the intermediate feature map includes 2*2 feature elements, then the second number of candidate feature elements can be feature elements in the first row or feature elements in the first column.

[0047] According to embodiments of this disclosure, after obtaining a second number of candidate feature elements corresponding to each intermediate feature map, an intermediate feature element can be determined for each intermediate feature map based on the second number of candidate feature elements. The method for determining the intermediate feature element can be configured according to actual business needs and is not limited here. For example, the maximum value among the second number of candidate feature elements can be determined as the intermediate feature element. Alternatively, the second number of candidate feature elements can be arranged, and the intermediate feature element can be determined from the feature elements that occupy a preset position in the arrangement result.

[0048] According to embodiments of this disclosure, after obtaining the intermediate feature elements corresponding to each intermediate feature map, a target feature element corresponding to each intermediate feature map can be determined based on a target threshold and the intermediate feature elements corresponding to each intermediate feature map. For each intermediate feature map, for example, if the intermediate feature element is greater than the target threshold, the intermediate feature element can be determined as the target feature element. Alternatively, if the intermediate feature element is less than or equal to the target threshold, a third number of feature elements can be determined based on a plurality of feature elements corresponding to the intermediate feature element and a second number of candidate feature elements, and the target feature element can be determined based on the third number of feature elements and the target threshold.

[0049] According to embodiments of this disclosure, after obtaining the target feature elements corresponding to each intermediate feature map, a second feature map corresponding to the image to be processed can be determined based on the target feature elements corresponding to each intermediate feature map. Based on this, an image processing task can be performed according to the second feature map corresponding to the image to be processed to obtain the image processing result.

[0050] According to embodiments of this disclosure, since the target feature elements are determined based on the target threshold and the intermediate feature elements of each intermediate feature map, by adopting a hardware and software co-design approach and a threshold comparison-based judgment method, some redundant operations in the max pooling process can be reduced, and the target feature elements of the max pooling operation result can be obtained in advance. Therefore, compared with traditional pooling accelerators, the power consumption and latency overhead of inference can be reduced, thereby improving the efficiency of subsequent image processing.

[0051] The following is for reference. Figures 2-6 The image processing method 100 according to an embodiment of the present invention will be further described.

[0052] According to embodiments of this disclosure, the image processing method 100 may further include the following operations.

[0053] Obtain a trained convolutional neural network, which includes S max-pooling layers, where S is a positive integer. For each max-pooling layer, determine the corresponding candidate threshold and candidate sliding window size. Associate and store the candidate layer identifiers, corresponding candidate thresholds, and candidate sliding window sizes of the S max-pooling layers with the data source.

[0054] According to embodiments of this disclosure, before obtaining the first feature map corresponding to the image to be processed, a candidate threshold and a candidate sliding window size can be pre-set for each max-pooling layer in the trained convolutional neural network. The candidate threshold can be obtained during the training process of the trained convolutional neural network, and it is known and remains unchanged during the inference process of the trained convolutional neural network. The candidate sliding window size can refer to the size of the sliding window used during the max-pooling operation, and can be determined by the max-pooling algorithm corresponding to the max-pooling layer.

[0055] According to embodiments of this disclosure, after obtaining the candidate threshold and candidate sliding window size corresponding to each max pooling layer, the candidate threshold and candidate sliding window size corresponding to each max pooling layer can be associated and stored in the data source. For example, the data source can be configured with a threshold storage module and a sliding window size storage module. The threshold storage module can be used to store candidate layer identifiers and candidate thresholds, and the sliding window size storage module can be used to store candidate layer identifiers and candidate sliding window sizes.

[0056] According to embodiments of this disclosure, by pre-associating and storing the relevant candidate parameters of each max pooling layer to the data source, the relevant candidate parameters of the trained convolutional neural network can be better managed, reducing the computational cost of the trained convolutional neural network, improving operating efficiency, and enabling the trained convolutional neural network to have better generalization ability.

[0057] According to embodiments of this disclosure, operation S110 may include the following operations.

[0058] Based on the target level identifier and S candidate level identifiers, the target sliding window size is determined from the S candidate sliding window sizes. Based on the target sliding window size, the first feature map is divided to obtain a first number of intermediate feature maps.

[0059] According to embodiments of this disclosure, the first feature map may be obtained by processing a max-pooling layer corresponding to the target level identifier. After obtaining the target level identifier, it can be matched with S candidate level identifiers in the data source to obtain candidate level identifiers that match the target level identifier. The size of the candidate sliding window corresponding to the candidate level identifier can be determined as the target sliding window size.

[0060] According to embodiments of this disclosure, after obtaining the target sliding window size, the first feature map can be divided based on the target sliding window size to obtain a first number of intermediate feature maps. For example, if the target sliding window size corresponding to a 2*2 max pooling operation is 2*2, meaning that each step requires 4 feature elements, the first feature map can be divided based on 2*2 to obtain a first number of intermediate feature maps, each containing 4 feature elements.

[0061] According to embodiments of this disclosure, by determining the target sliding window size based on the target level identifier and candidate level identifiers, the most suitable size can be selected from S candidate sliding window sizes for feature map partitioning, thereby ensuring the precision and accuracy of feature map partitioning. Furthermore, by partitioning the first feature map based on the target sliding window size, the efficiency of feature extraction can be effectively improved, enabling more accurate extraction of key information from the feature map, thereby reducing some redundant computations during the max pooling process.

[0062] Figure 2 The illustration shows an example schematic diagram of an intermediate feature map acquisition process according to an embodiment of the present disclosure.

[0063] like Figure 2 As shown, in 200, the trained convolutional neural network 201 may include S max-pooling layers, for example, max-pooling layer 201_1, max-pooling layer 201_2, ..., max-pooling layer 201_S. For each max-pooling layer, a candidate layer identifier 202, a candidate threshold 203, and a candidate sliding window size 204 corresponding to the max-pooling layer are determined. The candidate layer identifiers 202, the candidate thresholds 203, and the candidate sliding window sizes 204 corresponding to each of the S max-pooling layers are associated and stored in the data source 205.

[0064] After obtaining the first feature map 208 corresponding to the image to be processed, S candidate level identifiers can be defined based on the target level identifier 206 and the data source 205 corresponding to the first feature map 208. The target sliding window size 207 is then determined from the S candidate sliding window sizes. Based on the target sliding window size 207, the first feature map 208 is divided to obtain a first number of intermediate feature maps 209.

[0065] According to embodiments of this disclosure, operation S120 may include the following operations.

[0066] For each intermediate feature map, among P*Q feature elements, determine a second number of candidate feature elements, where the second number is less than or equal to P*Q. The candidate feature element with the largest value among these second number of candidate feature elements is then determined as the intermediate feature element.

[0067] According to embodiments of this disclosure, the intermediate feature map may include P*Q feature elements, where P and Q are both positive integers. In one example, a computation module can be configured to perform computational operations, specifically to compute the first feature map. The first feature map may include multiple feature elements. These feature elements may refer to the input data of the max pooling layer, typically computed by the hardware module of the preceding convolutional layer. The preceding convolutional layer hardware module refers to the hardware module used to perform the convolution operator operation before the max pooling layer. The implementation of the preceding convolutional layer hardware module can be configured according to actual business needs and is not limited here.

[0068] According to embodiments of this disclosure, candidate feature elements can be understood as a subset of feature values ​​required to calculate a single max pooling operation. These subset feature values ​​refer to a portion of all feature values ​​required for a single max pooling operation. It should be noted that the method for determining the second number of candidate feature elements can be configured according to actual business needs and is not limited here. For example, the second number of candidate feature elements can be feature elements from the first row or the first column.

[0069] According to embodiments of this disclosure, after obtaining a second number of candidate feature elements, the maximum value among these second number of candidate feature elements can be determined as the current largest feature value; that is, the intermediate feature element can be understood as the maximum value among a subset of feature values. In one example, a comparison module can be configured to perform a comparison operation, namely, to compare the size of each feature value among the second number of candidate feature elements, find the largest feature value among the multiple feature values, and determine the largest feature value as the intermediate feature element.

[0070] According to embodiments of this disclosure, by selecting a second number of candidate feature elements in each intermediate feature map, the feature representation can be effectively simplified, the number of feature elements reduced, computational complexity and memory consumption decreased, and the model's running efficiency improved. Furthermore, by selecting the candidate feature element with the largest value among the second number of candidate feature elements as the intermediate feature element, the efficiency and accuracy of determining the intermediate feature element can be improved.

[0071] Figure 3 The illustration shows an example schematic diagram of the process for obtaining intermediate feature elements according to an embodiment of the present disclosure.

[0072] like Figure 3As shown, in 300, a first number of intermediate feature maps can be determined based on the first feature map corresponding to the image to be processed 301. For example, the first number of intermediate feature maps may include intermediate feature map 302, intermediate feature map 303, intermediate feature map 304, ..., intermediate feature map 305. It should be noted that the size of the image to be processed 301 and each intermediate feature map in 300 is only an example and is not limited here.

[0073] Taking intermediate feature map 305 as an example, it can include 2*2 feature elements. Among these 2*2 feature elements, a second number of candidate feature elements can be determined. For example, the second number of candidate feature elements includes candidate feature element 305_1 and candidate feature element 305_2. Based on this, the candidate feature element with the largest value among the second number of candidate feature elements can be determined as the intermediate feature element.

[0074] According to embodiments of this disclosure, the image processing method 100 may further include the following operations.

[0075] Based on the target level identifier and S candidate level identifiers, the target threshold is determined from the S candidate thresholds.

[0076] According to embodiments of this disclosure, operation S230 may include the following operations.

[0077] For each intermediate feature map, a first comparison result is determined based on the target threshold and the intermediate feature elements of the intermediate feature map. Based on the first comparison result, the target feature elements of the intermediate feature map are then determined.

[0078] According to embodiments of this disclosure, the first feature map may be obtained by processing a max pooling layer corresponding to the target level identifier. After obtaining the target level identifier, the target level identifier can be matched with S candidate level identifiers to determine the candidate level identifier that matches the target level identifier among the S candidate level identifiers, and the candidate threshold corresponding to the candidate level identifier can be determined as the target threshold.

[0079] According to embodiments of this disclosure, after obtaining a target threshold, the target threshold can be compared with intermediate feature elements of the intermediate feature map to obtain a first comparison result. The first comparison result may include one of the following: the intermediate feature element is greater than the target threshold, or the intermediate feature element is less than or equal to the target threshold. In one example, a threshold comparison module can be configured to perform a comparison operation, i.e., compare the intermediate feature element with the target threshold to obtain the first comparison result. After obtaining the first comparison result, the target feature element of the intermediate feature map can be further determined based on the first comparison result.

[0080] According to embodiments of this disclosure, by adaptively determining target feature elements, unnecessary computation can be reduced, computational efficiency can be improved, and the inference process of the model can be accelerated, thus shortening the response time.

[0081] According to embodiments of this disclosure, determining the target feature elements of the intermediate feature map based on the first comparison result may include the following operations.

[0082] If the value of the intermediate feature element is greater than the target threshold as indicated by the first comparison result, the intermediate feature element is determined as the target feature element of the intermediate feature map.

[0083] According to embodiments of this disclosure, if the first comparison result indicates that the value of the intermediate feature element is greater than the target threshold, subsequent feature value calculations and comparisons can be skipped, and the current intermediate feature element can be used as the calculation result of this max pooling, that is, the intermediate feature element is determined as the target feature element of the intermediate feature map. In one example, a calculation control module can be configured to perform control operations, that is, if the first comparison result indicates that the value of the intermediate feature element is greater than the target threshold, the intermediate feature element is determined as the target feature element of the intermediate feature map.

[0084] According to embodiments of this disclosure, by determining the value of an intermediate feature element as a target feature element when the first comparison result indicates that the value of the intermediate feature element is greater than the target threshold, unnecessary calculations and processing can be reduced and computational efficiency can be improved.

[0085] According to embodiments of this disclosure, determining the target feature elements of the intermediate feature map based on the first comparison result may include the following operations.

[0086] If the value of the intermediate feature element is less than or equal to the target threshold as indicated by the first comparison result, a third number of remaining feature elements of the intermediate feature map are determined based on the P*Q feature elements and the second number of candidate feature elements. A second comparison result is then determined based on the third number of remaining feature elements and the intermediate feature elements. Finally, the target feature element of the intermediate feature map is determined based on the second comparison result.

[0087] According to embodiments of this disclosure, if the first comparison result indicates that the value of the intermediate feature element is less than or equal to the target threshold, subsequent feature value calculations and comparisons can continue. Specifically, based on the P*Q feature elements and the second number of candidate feature elements of the intermediate feature map, a third number of remaining feature elements of the intermediate feature map are determined, and a second comparison result is determined based on the third number of remaining feature elements and the intermediate feature elements. The second comparison result can be used to represent the descending order of the third number of remaining feature elements and the intermediate feature elements. Based on this, the target feature element of the intermediate feature map can be determined according to the second comparison result.

[0088] In one example, a calculation control module can be configured to perform control operations, namely, if the first comparison result indicates that the value of the intermediate feature element is less than or equal to the target threshold, the calculation module is controlled to complete the calculation of subsequent feature values ​​and compare them with the current intermediate feature element.

[0089] According to embodiments of this disclosure, since the third number of remaining feature elements are determined based on the P*Q feature elements of the intermediate feature map and the second number of candidate feature elements, by selecting a small number of remaining feature elements to be compared with the intermediate feature elements, unnecessary computational complexity can be reduced, and all feature elements of the entire intermediate feature map need not be processed, thereby improving computational efficiency.

[0090] According to embodiments of this disclosure, determining the target feature elements of the intermediate feature map based on the second comparison result may include the following operations.

[0091] The element with the largest value among the fourth number of feature elements is determined as the target feature element of the intermediate feature map.

[0092] According to embodiments of this disclosure, the second comparison result may include a fourth number of feature elements arranged in descending order. For example, if the third number of remaining feature elements are 4 and 10, and the intermediate feature element is 8, then the second comparison result can be determined as 10, 8, 4. Based on this, the element with the largest value among the fourth number of feature elements can be used as the calculation result of this max pooling, that is, feature element 10 is determined as the target feature element of the intermediate feature map.

[0093] According to embodiments of this disclosure, by determining the element with the largest value among the fourth number of feature elements as the target feature element, feature selection based on the maximum value is achieved, avoiding complex calculation and processing procedures and improving processing efficiency.

[0094] Figure 4 The illustration shows an example schematic diagram of a process for obtaining target feature elements according to an embodiment of the present disclosure.

[0095] like Figure 4 As shown, in step 400, a target threshold 403 can be determined from S candidate thresholds based on the target level identifier 401 and S candidate level identifiers 402. A first comparison result 405 is determined based on the target threshold 403 and the intermediate feature element 404_1 of the intermediate feature map 404. After obtaining the first comparison result 405, operation S410 can be performed.

[0096] In operation S410, it is determined whether the first comparison result indicates that the value of the intermediate feature element is greater than the target threshold. If so, the intermediate feature element 404_1 can be determined as the target feature element 406 of the intermediate feature map 404. If not, the third number of remaining feature elements 407 of the intermediate feature map 404 can be determined based on the P*Q feature elements and the second number of candidate feature elements of the intermediate feature map. Based on the third number of remaining feature elements 407 and the intermediate feature element 404_1, the second comparison result 408 is determined. Based on the second comparison result 408, the target feature element 406 of the intermediate feature map 404 is determined.

[0097] Figure 5A , Figure 5B and Figure 5C An example schematic diagram illustrating a max-pooling operator acceleration process according to an embodiment of the present disclosure is shown.

[0098] In one example Figures 5A-5C A pooling acceleration algorithm with a 2×2 max pooling window for convolutional neural networks is presented.

[0099] like Figure 5A As shown, after obtaining the image to be processed 501, the image to be processed 501 can be processed using the max pooling layer corresponding to the target level identifier to obtain the first feature map 502.

[0100] Taking the intermediate feature map 502_1 in the first feature map 502 as an example, among the 2*2 feature elements, a second number of candidate feature elements are determined. For example, the second number of candidate feature elements includes the feature values ​​corresponding to the top left and top right positions in the max pooling 2×2 window, namely 8 and 6. Based on this, the candidate feature element with the largest value among the second number of candidate feature elements can be determined as the intermediate feature element, namely 8.

[0101] Based on the target level identifier and S candidate level identifiers, a target threshold (Th) is determined from the S candidate thresholds. A threshold comparison is performed between the target threshold and the intermediate feature elements of the intermediate feature map to determine a first comparison result. The first comparison result may include one of the following: the value of the intermediate feature element is greater than the target threshold, or the value of the intermediate feature element is less than or equal to the target threshold.

[0102] like Figure 5B As shown, if the value of the intermediate feature element is greater than the target threshold as indicated by the first comparison result, the calculation and comparison of subsequent feature values ​​can be skipped, and the intermediate feature element can be determined as the target feature element of the intermediate feature graph 502_1, that is, the calculation result of max pooling is 8.

[0103] like Figure 5CAs shown, when the value of the intermediate feature element is less than or equal to the target threshold, the intermediate feature map 502_1 in the first feature map 502 obtained by processing the image 501 to be processed using the max pooling layer corresponding to the target level identifier can be used. Based on the 2*2 feature elements and the second number of candidate feature elements (i.e., 8 and 6) of the intermediate feature map 502_1, the third number of remaining feature elements, i.e., 4 and 10, are determined, and the feature value calculation for the subsequent lower left and lower right positions is continued. On this basis, based on the third number of remaining feature elements (i.e., 4 and 10) and the intermediate feature element (i.e., 8), the second comparison result is determined to be 10, 8, and 4. Thus, based on the second comparison result, the target feature element of the intermediate feature map 502_1 is determined to be 10.

[0104] Figure 6 An example schematic diagram of a max-pooling operator acceleration device according to an embodiment of the present disclosure is shown.

[0105] like Figure 6 As shown, a pooling acceleration device 600 for a max-pooling layer with a pooling window of 2×2 in a convolutional neural network is presented. The pooling acceleration device 600 may include a threshold register 601, a feature value calculation and comparison unit 602, a comparison circuit 603, and a control unit 604.

[0106] Candidate thresholds corresponding to each max-pooling layer can be pre-stored in threshold register 601. Control unit 604 can be used to generate control commands to control feature value calculation and comparison unit 602. Feature value calculation and comparison unit 602 can be used, under the control of control unit 604, to accept convolutional layer input and calculate the intermediate feature elements of each intermediate feature map. Comparison circuit 603 can be used to determine the target feature element of each intermediate feature map based on the target threshold and the intermediate feature elements of each intermediate feature map, and output the target feature element of each intermediate feature map to control unit 604.

[0107] According to embodiments of this disclosure, operation S140 may include the following operations.

[0108] Based on the target feature elements of each intermediate feature map, a second feature map corresponding to the image to be processed is determined. Based on the second feature map, the image processing result is determined.

[0109] According to embodiments of this disclosure, after obtaining the target feature elements of each intermediate feature map, a second feature map corresponding to the image to be processed can be determined based on the target feature elements of each intermediate feature map. After obtaining the second feature map, an image processing task can be performed on the second feature map to obtain the image processing result. The image processing task may include at least one of the following: image enhancement task, image filtering task, image segmentation task, feature extraction task, image compression task, image synthesis task, image recognition and classification task, and image reconstruction and restoration task.

[0110] For example, image enhancement tasks can refer to improving the quality of an image, making it more visually appealing or better suited for a specific task. Image filtering tasks can refer to applying different types of filters to smooth an image, remove noise, or enhance details in an image. Image segmentation tasks can refer to dividing an image into multiple regions or objects, typically based on the similarity or continuity between pixels. Feature extraction tasks can refer to extracting representative features from an image for subsequent image recognition and classification.

[0111] Alternatively, image compression tasks can refer to tasks that reduce the storage space or transmission bandwidth occupied by images while maintaining image quality as much as possible. Image synthesis tasks can refer to tasks that combine multiple images or parts of images into a new image. Image recognition and classification tasks can refer to tasks that use machine learning or deep learning techniques to identify and classify images. Image reconstruction and restoration tasks can refer to tasks that reconstruct or restore damaged or incomplete images using mathematical models or algorithms.

[0112] According to embodiments of this disclosure, by determining a second feature map corresponding to the image to be processed based on the target feature elements of each intermediate feature map, it helps ensure that each intermediate feature map can extract effective features related to the image to be processed based on its target feature elements, thereby improving the diversity and richness of features. Based on this, by determining the image processing result based on the second feature map, it can be ensured that the effective information extracted from each intermediate feature map is fully utilized during image processing, thereby improving the precision and accuracy of image processing.

[0113] The above are merely exemplary embodiments, but are not limited thereto. Other image processing methods known in the art may also be included, as long as they can reduce the energy consumption and latency overhead of inference.

[0114] Figure 7 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown schematically.

[0115] like Figure 7As shown, the image processing device 700 may include a first determining module 710, a second determining module 720, a third determining module 730, and a fourth determining module 740.

[0116] The first determining module 710 is used to determine a first number of intermediate feature maps based on a first feature map corresponding to the image to be processed, wherein the first feature map includes multiple feature elements.

[0117] The second determining module 720 is used to determine the intermediate feature elements of each intermediate feature map based on the second number of candidate feature elements of each intermediate feature map.

[0118] The third determining module 730 is used to determine the target feature element of each intermediate feature map based on the target threshold and the intermediate feature element of each intermediate feature map.

[0119] The fourth determining module 740 is used to determine the image processing result corresponding to the image to be processed based on the target feature elements of each intermediate feature map.

[0120] According to embodiments of this disclosure, the image processing apparatus 700 may further include an acquisition module, a fifth determination module, and an associated storage module.

[0121] The acquisition module is used to acquire a trained convolutional neural network, wherein the trained convolutional neural network includes S max pooling layers, where S is a positive integer.

[0122] The fifth determination module is used to determine the candidate threshold and candidate sliding window size corresponding to each max pooling layer.

[0123] The associated storage module is used to associate and store the candidate level identifiers of the S max pooling layers, the candidate thresholds and candidate sliding window sizes corresponding to each max pooling layer, and store them in the data source.

[0124] According to an embodiment of this disclosure, the first feature map is obtained by processing a max pooling layer corresponding to the target level identifier.

[0125] According to embodiments of this disclosure, the first determining module 710 may include a first determining submodule and a dividing submodule.

[0126] The first determining submodule is used to determine the target sliding window size from among the S candidate sliding window sizes based on the target level identifier and the S candidate level identifiers.

[0127] The partitioning submodule is used to partition the first feature map based on the target sliding window size to obtain a first number of intermediate feature maps.

[0128] According to embodiments of this disclosure, the intermediate feature map includes P*Q feature elements, where P and Q are both positive integers.

[0129] According to embodiments of this disclosure, the second determining module 720 may include a second determining submodule and a third determining submodule.

[0130] The second determination submodule is used to determine a second number of candidate feature elements from P*Q feature elements for each intermediate feature map, wherein the second number is less than or equal to P*Q.

[0131] The third determination submodule is used to determine the candidate feature element with the largest value among the second number of candidate feature elements as the intermediate feature element.

[0132] According to an embodiment of this disclosure, the first feature map is obtained by processing a max pooling layer corresponding to the target level identifier.

[0133] According to embodiments of the present disclosure, the image processing apparatus 700 may further include a sixth determining module.

[0134] The sixth determination module is used to determine the target threshold from among the S candidate thresholds based on the target level identifier and the S candidate level identifiers.

[0135] According to embodiments of this disclosure, the third determining module 730 may include a fourth determining submodule and a fifth determining submodule.

[0136] The fourth determination submodule is used to determine the first comparison result for each intermediate feature map based on the target threshold and the intermediate feature elements of the intermediate feature map.

[0137] The fifth determination submodule is used to determine the target feature elements of the intermediate feature map based on the first comparison result.

[0138] According to embodiments of this disclosure, the fifth determining submodule may include a first determining unit.

[0139] The first determining unit is used to determine the intermediate feature element as the target feature element of the intermediate feature map when the value of the intermediate feature element is greater than the target threshold as indicated by the first comparison result.

[0140] According to embodiments of this disclosure, the fifth determining submodule may further include a second determining unit, a third determining unit, and a fourth determining unit.

[0141] The second determining unit is used to determine the third number of remaining feature elements of the intermediate feature map based on the P*Q feature elements and the second number of candidate feature elements when the value of the intermediate feature element is less than or equal to the target threshold in the first comparison result.

[0142] The third determining unit is used to determine the second comparison result based on the third number of remaining feature elements and intermediate feature elements.

[0143] The fourth determining unit is used to determine the target feature elements of the intermediate feature map based on the second comparison result.

[0144] According to embodiments of this disclosure, the second comparison result includes a fourth number of feature elements arranged in descending order.

[0145] According to embodiments of this disclosure, the fourth determining unit may include a determining subunit.

[0146] The sub-unit is determined to identify the element with the largest value among the fourth number of feature elements as the target feature element of the intermediate feature map.

[0147] According to embodiments of this disclosure, the fourth determining module 740 may include a sixth determining submodule and a seventh determining submodule.

[0148] The sixth determination submodule is used to determine the second feature map corresponding to the image to be processed based on the target feature elements of each intermediate feature map.

[0149] The seventh determination submodule is used to determine the image processing result based on the second feature map.

[0150] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), Systems-on-Chip, Systems-on-Substrate, Systems-on-Package, Application-Specific Integrated Circuits (ASICs), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0151] For example, any plurality of the first determining module 710, the second determining module 720, the third determining module 730, and the fourth determining module 740 can be combined into one module / unit / subunit, or any one of these modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / units / subunits can be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of this disclosure, at least one of the first determining module 710, the second determining module 720, the third determining module 730, and the fourth determining module 740 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the first determining module 710, the second determining module 720, the third determining module 730, and the fourth determining module 740 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0152] It should be noted that the image processing apparatus part in the embodiments of this disclosure corresponds to the image processing method part in the embodiments of this disclosure. The specific description of the image processing apparatus part is referred to in the image processing method part, and will not be repeated here.

[0153] Figure 8 A block diagram of an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure is shown schematically. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0154] like Figure 8 As shown, a computer electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 809 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0155] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0156] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0157] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0158] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0159] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0160] For example, according to embodiments of this disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0161] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the image processing methods provided in the embodiments of this disclosure.

[0162] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0163] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0164] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0165] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0166] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. An image processing method, comprising: Based on the first feature map corresponding to the image to be processed, a first number of intermediate feature maps are determined, wherein the first feature map includes multiple feature elements; Based on the second number of candidate feature elements for each intermediate feature map, the intermediate feature elements for each intermediate feature map are determined. Based on the target threshold and the intermediate feature elements of each intermediate feature map, determine the target feature elements of each intermediate feature map; and Based on the target feature elements of each intermediate feature map, the image processing result corresponding to the image to be processed is determined.

2. The method according to claim 1, further comprising, before determining a first number of intermediate feature maps based on a first feature map corresponding to the image to be processed: Obtain the trained convolutional neural network, where, The trained convolutional neural network includes S max-pooling layers, where S is a positive integer; For each max pooling layer, a candidate threshold and a candidate sliding window size corresponding to the max pooling layer are determined respectively; as well as The candidate level identifiers of the S max pooling layers, along with the candidate thresholds and candidate sliding window sizes corresponding to each max pooling layer, are associated and stored in the data source.

3. The method according to claim 2, wherein, The first feature map is obtained by processing a max pooling layer corresponding to the target level identifier; The step of determining a first number of intermediate feature maps based on a first feature map corresponding to the image to be processed includes: Based on the target level identifier and the S candidate level identifiers, determine the target sliding window size from the S candidate sliding window sizes; as well as Based on the target sliding window size, the first feature map is divided to obtain the first number of intermediate feature maps.

4. The method according to any one of claims 1 to 3, wherein, The intermediate feature map includes P*Q feature elements, where P and Q are both positive integers; The step of determining the intermediate feature elements of each intermediate feature map based on the second number of candidate feature elements of each intermediate feature map includes: For each of the intermediate feature maps From the P*Q feature elements, a second number of candidate feature elements are determined, wherein the second number is less than or equal to P*Q; and The candidate feature element with the largest value among the second number of candidate feature elements is determined as the intermediate feature element.

5. The method according to claim 2, wherein, The first feature map is obtained by processing a max pooling layer corresponding to the target level identifier; The method further includes, before determining the target feature element of each intermediate feature map based on the target threshold and the intermediate feature element of each intermediate feature map: Based on the target level identifier and the S candidate level identifiers, the target threshold is determined from the S candidate thresholds; The step of determining the target feature element of each intermediate feature map based on the target threshold and the intermediate feature element of each intermediate feature map includes: For each of the intermediate feature maps A first comparison result is determined based on the target threshold and the intermediate feature elements of the intermediate feature map; as well as Based on the first comparison result, the target feature elements of the intermediate feature map are determined.

6. The method according to claim 5, wherein, The step of determining the target feature elements of the intermediate feature map based on the first comparison result includes: If the first comparison result indicates that the value of the intermediate feature element is greater than the target threshold, the intermediate feature element is determined as the target feature element of the intermediate feature map.

7. The method according to claim 6, further comprising: If the first comparison result indicates that the value of the intermediate feature element is less than or equal to the target threshold, then a third number of remaining feature elements of the intermediate feature map are determined based on the P*Q feature elements of the intermediate feature map and the second number of candidate feature elements. Based on the third number of remaining feature elements and the intermediate feature elements, a second comparison result is determined; and Based on the second comparison result, the target feature elements of the intermediate feature map are determined.

8. The method according to claim 7, wherein, The second comparison result includes a fourth number of feature elements arranged in descending order; The step of determining the target feature elements of the intermediate feature map based on the second comparison result includes: The element with the largest value among the fourth number of feature elements is determined as the target feature element of the intermediate feature map.

9. The method according to any one of claims 1 to 7, wherein, The step of determining the image processing result corresponding to the image to be processed based on the target feature elements of each intermediate feature map includes: Based on the target feature elements of each of the intermediate feature maps, a second feature map corresponding to the image to be processed is determined; and The image processing result is determined based on the second feature map.

10. An image processing apparatus, comprising: The first determining module is configured to determine a first number of intermediate feature maps based on a first feature map corresponding to the image to be processed, wherein the first feature map includes multiple feature elements; The second determining module is used to determine the intermediate feature elements of each intermediate feature map based on the second number of candidate feature elements of each intermediate feature map. The third determining module is used to determine the target feature element of each intermediate feature map based on the target threshold and the intermediate feature element of each intermediate feature map; and The fourth determining module is used to determine the image processing result corresponding to the image to be processed based on the target feature elements of each intermediate feature map.

Images