Image processing method and device, storage medium and electronic equipment

Abstract

The invention discloses an image processing method and device, a storage medium and electronic equipment, and relates to the field of artificial intelligence. The method comprises the following steps: receiving a target object image; inputting the target object image into a target model, extracting target features of the target object image according to priori knowledge learned by the target model in a model training stage, and determining a target image corresponding to the target object image based on the target features; wherein the target features are used for representing features of an abnormal region in the target object image, the target model is a model obtained by updating an initial model according to model parameters, the model parameters are parameter combinations meeting preset conditions, and the initial model is a model constructed based on a target mechanism; the target mechanism is used for dynamically adjusting the weight corresponding to the pixel of the model input image and the weight corresponding to the image color channel. The technical problem that the accuracy is low when image abnormity is manually recognized in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically to an image processing method, device, storage medium, and electronic device. Background Art

[0002] With the rapid development of the information society, image analysis technology is playing an increasingly important role in various fields such as finance, healthcare, and security monitoring. In particular, in the financial sector, such as banks and currency exchange institutions, identifying damaged or abnormal images (such as currency images) is a routine yet crucial task. For a long time, this type of identification has relied primarily on manual visual inspection. However, with the expansion of business volume, the limitations of manual identification have become increasingly apparent.

[0003] On the one hand, the human eye has limited resolution, especially for minor damage or anomalies under complex patterns. On the other hand, manual inspection is affected by multiple factors such as individual differences, fatigue status and ambient light, resulting in large fluctuations in the accuracy of recognition results, making it difficult to ensure consistency and reliability.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The present application provides an image processing method, device, storage medium and electronic device to at least solve the technical problem of low accuracy in manual identification of image anomalies in the prior art.

[0006] According to one aspect of the present application, an image processing method is provided, comprising: receiving a target object image; inputting the target object image into a target model, extracting target features of the target object image based on prior knowledge learned by the target model in a model training phase, and determining a target image corresponding to the target object image based on the target features; wherein the target features are used to characterize features of abnormal areas in the target object image, and the target model is a model obtained by updating an initial model according to model parameters, wherein the model parameters are a combination of parameters that meet preset conditions, and the initial model is a model constructed based on a target mechanism, and the target mechanism is used to dynamically adjust weights corresponding to pixels of a model input image and weights corresponding to image color channels, wherein the preset conditions are used to constrain an error value between the target image of the target model and an actual abnormal object image to be less than a preset threshold.

[0007] Optionally, the target model is obtained by the following steps: obtaining N object images, where N is an integer greater than 1, and each object image includes annotated actual abnormality information; performing multiple iterative training and verification operations on the initial model based on the N object images to obtain the target model, where the multiple iterative training is used to determine the model parameters of the target model, and the verification operation is used to verify the performance parameters of the target model.

[0008] Optionally, the initial model is subjected to multiple iterative training and verification operations based on N object images to obtain a target model, including: dividing the N object images into a model training set and a model verification set; inputting the model training set into the initial model for iterative training until the number of model iterations is greater than a preset number or the training error of the model is less than a preset threshold, thereby obtaining model parameters of the initial model; updating the initial model to a first model based on the model parameters; and performing a verification operation on the first model based on the model verification set to obtain a target model.

[0009] Optionally, each iterative training includes the following steps: when there are T training sets in the model training set, each of the T training sets is input into the encoder of the initial model, and T initial feature maps corresponding to the T training sets are determined by the encoder, wherein the encoder includes S levels, each level of the encoder is used to reduce the resolution of the object image, and the feature map corresponding to the object image is determined by the depthwise separable convolution and residual connection in the level, and the obtained feature map is used as the input of the next level, wherein S is an integer greater than or equal to 1, the depthwise separable convolution is used to extract the features of the object image, and the residual connection is used to add the output of the depthwise separable convolution in each level of the encoder to the input of the depthwise separable convolution; based on the T initial feature maps, T recognition images are generated through the output layer of the initial model, wherein the recognition image includes abnormal information corresponding to the object image; and the model parameters of the initial model are updated according to the T recognition images and the actual abnormal object images corresponding to the T training sets.

[0010] Optionally, based on T initial feature maps, T recognition images are generated through the output layer of the initial model, including: using a target mechanism to perform target processing on the T initial feature maps to obtain T first feature maps, wherein the target processing is used to adjust the weights corresponding to each channel and position in the initial feature map; each of the T first feature maps is input into a decoder in the initial model for an upsampling operation to obtain T second feature maps, wherein the decoder includes S levels, the decoder is used to reconstruct the features of the object image corresponding to the first feature map, and the upsampling operation is used to improve the resolution of the first feature map; and T recognition images are generated based on the T second feature maps through the output layer of the initial model.

[0011] Optionally, the target processing includes the following steps: extracting spatial distribution information of T initial feature maps, and determining T first weight matrices based on the spatial distribution information corresponding to each initial feature map, wherein the spatial distribution information is used to characterize the feature intensity and distribution characteristics on each channel in the initial feature map, and the first weight matrix is ​​used to adjust the weight corresponding to each channel in the initial feature map; determining T third feature maps based on each initial feature map in the T initial feature maps and the first weight matrix corresponding to the initial feature map, wherein the third feature map is a weighted feature map obtained by multiplying the initial feature map with the corresponding first weight matrix; extracting position information of the T third feature maps to obtain T position information, wherein the position information is used to characterize the feature distribution in the third feature map; determining T second weight matrices based on the T position information, wherein the second weight matrix is ​​used to adjust the weights corresponding to different positions in the third feature map; determining T first feature maps based on each third feature map in the T third feature maps and the second weight matrix corresponding to the third feature map.

[0012] Optionally, each of the T first feature maps is input into the decoder in the initial model for upsampling operation to obtain T second feature maps, including: inputting each of the T first feature maps into S layers in the decoder in the initial model for upsampling operation to obtain T feature map sets, wherein the feature map sets include feature maps output by the first feature map in each layer in the decoder, wherein the feature maps output in each layer include the feature maps corresponding to the encoder in the initial model at that layer; adjusting the feature maps in each feature map set in the T feature map sets to a preset resolution to obtain T first feature map sets; and fusing the feature maps in each of the T first feature map sets to obtain T second feature maps.

[0013] Optionally, the model parameters of the initial model are updated according to the T recognition images and the actual abnormal object images corresponding to the T training sets, including: determining the error value between each of the T recognition images corresponding to the T training sets and the actual abnormal object image corresponding to the training set, to obtain T error values; based on the T error values, updating the model parameters of the initial model through a first algorithm and back propagation, wherein back propagation is used to determine the influencing factors of the model parameters of the initial model on the model error, and the first algorithm is used to adjust the model parameters of the initial model based on the influencing factors.

[0014] According to another aspect of the present application, an image processing device is also provided, including: a receiving unit for receiving a target object image; a determination unit for inputting the target object image into a target model, extracting target features of the target object image based on prior knowledge learned by the target model in the model training phase, and determining a target image corresponding to the target object image based on the target features; wherein the target features are used to characterize the features of the abnormal area in the target object image, and the target model is a model obtained by updating the initial model according to the model parameters, wherein the model parameters are a parameter combination that meets preset conditions, and the initial model is a model constructed based on a target mechanism, and the target mechanism is used to dynamically adjust the weights corresponding to the pixels of the model input image and the weights corresponding to the image color channels, wherein the preset conditions are used to constrain the error value between the target image of the target model and the actual abnormal object image to be less than a preset threshold.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, which includes a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned image processing method.

[0016] According to another aspect of the present application, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned image processing method.

[0017] According to another aspect of an embodiment of the present application, a computer program product is further provided, comprising computer instructions, which implement the steps of the above-mentioned image processing method when executed by a processor.

[0018] In the present application, a target object image is first received, and then the target object image is input into a target model. Based on the prior knowledge learned by the target model during the model training phase, target features of the target object image are extracted, and a target image corresponding to the target object image is determined based on the target features. The target features are used to characterize the features of abnormal regions in the target object image. The target model is a model obtained by updating the initial model according to model parameters. The model parameters are a combination of parameters that meet preset conditions. The initial model is a model constructed based on a target mechanism. The target mechanism is used to dynamically adjust the weights corresponding to the pixels of the model input image and the weights corresponding to the image color channels. The preset conditions are used to constrain the error value between the target image of the target model and the actual abnormal object image to be less than a preset threshold. That is, by dynamically adjusting the model parameters and using deep learning to process the target object image, the purpose of accurately extracting and locating the features of abnormal regions in the image is achieved, thereby achieving the technical effect of automated high-precision abnormal region detection, thereby solving the technical problem of low accuracy in manual identification of image anomalies in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 This is a flow chart of an optional image processing method according to an embodiment of the present application. Figure 1 ;

[0021] Figure 2 is a schematic diagram of an optional residual connection structure according to an embodiment of the present application;

[0022] Figure 3 is a structural diagram of an optional U-Net model according to an embodiment of the present application;

[0023] Figure 4 This is a flow chart of an optional image processing method according to an embodiment of the present application. Figure 2 ;

[0024] Figure 5 is a schematic diagram of an optional image processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation portals for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation portals for users to choose to agree or refuse the automated decision-making results; if the user chooses to refuse, the expert decision-making process will be entered.

[0028] According to an embodiment of the present application, a method embodiment of an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] It should be noted that an intelligent processing system can be used as the execution subject of the image processing method of the embodiment of the present application. It is understandable that the image processing method provided in the embodiment of the present application can also be executed by other systems or devices, and the embodiment of the present application does not specifically limit this.

[0030] Figure 1 This is a flow chart of an optional image processing method according to an embodiment of the present application. Figure 1 ,like Figure 1 As shown, the method includes the following steps:

[0031] Step S101: receiving a target object image.

[0032] Optionally, the target object image refers to an image to be detected, which may be acquired by a camera, uploaded by a user, or from an internal database of the system.

[0033] In step S102 , the target object image is input into the target model, and target features of the target object image are extracted based on the prior knowledge learned by the target model during the model training phase. The target image corresponding to the target object image is determined based on the target features.

[0034] In step S102 , the target features are used to characterize the features of abnormal areas in the target object image, and the target model is a model obtained by updating the initial model according to the model parameters.

[0035] In step S102, the model parameters are a combination of parameters that meet preset conditions, and the initial model is a model constructed based on a target mechanism, which is used to dynamically adjust the weights corresponding to the pixels of the model input image and the weights corresponding to the image color channels.

[0036] In step S102 , the preset condition is used to constrain the error value between the target image of the target model and the actual abnormal object image to be less than a preset threshold.

[0037] Optionally, the target model refers to a multi-scale fusion U-Net model based on an attention mechanism (a convolutional neural network architecture specifically used for image segmentation), which has learned rules and patterns for identifying abnormal images during the training phase.

[0038] Optionally, prior knowledge: abnormal feature patterns learned by the model from the labeled dataset during training, such as the shape, size, and location information of cracks, stains, pits, etc. in the image.

[0039] Optionally, the target feature refers to the feature of the abnormal area in the damaged banknote image extracted by the model, which is used to describe information such as the shape, size, and position of the abnormal area.

[0040] Optionally, the target image refers to an image generated by the model based on the extracted target features, which highlights the abnormal area.

[0041] Optionally, a targeting mechanism, namely the Channel Attention Module and Spatial Attention Module (CBAM), dynamically adjusts the model's weighting of image pixels and color channels, improving the efficiency and accuracy of feature extraction. This targeting mechanism enables the model to more intelligently focus on key features while suppressing irrelevant information.

[0042] Optionally, the preset condition is a model performance indicator defined during training, such as the minimum value of a loss function, to ensure that the degree of match between the model output and the actual abnormal area is higher than a certain set standard, thereby improving the overall performance and practicality of the model.

[0043] Optionally, the intelligent processing system feeds the received image into a trained model for processing. The model uses prior knowledge learned during training to identify and extract features of potential abnormal regions in the image. Based on the extracted target features, the model generates a new image highlighting the abnormal regions. This step leverages the model's experience in identifying abnormal images during training, ensuring accurate location and characterization of abnormal regions.

[0044] As can be seen from steps S101 to S102, in this application, a target object image is first received, then the target object image is input into a target model. Based on the prior knowledge learned by the target model during the model training phase, target features of the target object image are extracted, and a target image corresponding to the target object image is determined based on the target features. The target features are used to characterize the characteristics of abnormal regions in the target object image. The target model is a model obtained by updating the initial model based on model parameters. The model parameters are parameter combinations that meet preset conditions. The initial model is a model constructed based on a target mechanism. The target mechanism is used to dynamically adjust the weights corresponding to the pixels of the model input image and the weights corresponding to the image color channels. The preset conditions are used to constrain the error value between the target image of the target model and the actual abnormal object image to be less than a preset threshold. That is, by dynamically adjusting the model parameters and using deep learning to process the target object image, the purpose of accurately extracting and locating the features of abnormal regions in the image is achieved, thereby achieving the technical effect of automated high-precision abnormal region detection, thereby solving the technical problem of low accuracy in manual image anomaly identification in the prior art.

[0045] In an optional embodiment, the target model is obtained through the following steps: the intelligent processing system first obtains N object images, where N is an integer greater than 1, and each object image includes annotated actual abnormality information, and then performs multiple iterative training and verification operations on the initial model based on the N object images to obtain the target model, wherein the multiple iterative training is used to determine the model parameters of the target model, and the verification operation is used to verify the performance parameters of the target model.

[0046] Optionally, the intelligent system initially collects a set of images containing abnormal regions, denoted as N, where N is a positive integer greater than 1, indicating that the number of training samples is at least two. Each image contains manually annotated anomaly information, which guides the model to identify abnormal regions that require special attention. Based on the N collected object images (abnormal images), the intelligent system uses the annotated anomaly information as a supervisory signal to repeatedly train the "initial model" (i.e., an unoptimized multi-scale fusion U-Net model based on the attention mechanism). During this process, the initial model continuously adjusts its "model parameters" (including weights and biases at each layer) through forward and backpropagation until the model can effectively identify and predict the annotated abnormal regions. This training process typically involves minimizing the loss function and applying optimization algorithms such as gradient descent to ensure that the model parameters meet the preset optimization conditions.

[0047] Optionally, in addition to training, the intelligent system will also use an independent "validation set" to evaluate model performance. This validation set also contains images of abnormal areas, but the image set is independent of the training set and is used to objectively evaluate the generalization ability of the model. After each training iteration, the model will be applied to the images in the validation set. By comparing the abnormal areas predicted by the model with the actual anomaly information, the system can calculate a series of "performance parameters" such as accuracy, recall rate, F1 score, etc. The goal of the validation operation is to ensure that the model maintains good recognition performance on data that has never been seen before, and to avoid overfitting.

[0048] Optionally, the intelligent system continuously adjusts model parameters through multiple iterations of training and validation until predetermined performance metrics are achieved. Under pre-defined conditions, the error between the target image output by the model and the actual image of the abnormal object must remain low, below a pre-set threshold, to ensure the accuracy and robustness of the model in identifying abnormal areas in the image.

[0049] As can be seen above, through iterative training and validation on a large number of images with annotated abnormal regions, the intelligent system was able to find an optimal set of model parameters, enabling the "target model" to not only accurately identify abnormal regions in the training set but also maintain stable high performance on unseen validation data. This approach combines the powerful feature extraction capabilities of deep learning with the adaptive weight adjustment properties of the attention mechanism, significantly improving the accuracy of abnormal region identification in images.

[0050] In an optional embodiment, the intelligent processing system divides N object images into a model training set and a model verification set, and then inputs the model training set into the initial model for iterative training until the number of model iterations is greater than a preset number or the training error of the model is less than a preset threshold, thereby obtaining the model parameters of the initial model. The initial model is then updated to a first model based on the model parameters, and finally the first model is verified based on the model verification set to obtain a target model.

[0051] Optionally, the intelligent processing system divides the collected N object images into two parts: a model training set and a model validation set. This division usually follows a certain ratio, for example, 80% of the images are used as a training set for model learning; the remaining 20% ​​are used as a validation set to evaluate the performance of the model. The processing system then inputs the model training set into the initial model built based on the target mechanism for iterative training. During the iterative training process, the model continuously attempts to predict the damaged area of ​​the input image, compares it with the true annotation, and calculates the loss (error). The model parameters are adjusted through the backpropagation algorithm to reduce the loss. The training loop continues until any of the following conditions is met: the number of model iterations exceeds the preset number, ensuring that the model has been fully trained, or the training error of the model is less than the preset threshold, which indicates that the performance of the model on the training set has reached the expected standard.

[0052] Optionally, after training, the intelligent processing system obtains the model parameters of the initial model. Model parameters are the numerical values ​​and weights used within the model to calculate predictions. Finally, based on these model parameters, the intelligent processing system updates the initial model to a first model, i.e., a model that initially possesses the ability to identify abnormal regions in images.

[0053] Optionally, the updated first model undergoes further validation. The intelligent processing system verifies the first model against the model validation set, evaluating the model's generalization ability by comparing the first model's predictions on the validation set with the actual annotations. If the model performs well on the validation set, meaning the prediction error is less than a preset threshold, the first model is optimized to become the target model.

[0054] As can be seen from the above, the intelligent processing system is able to efficiently train and validate the multi-scale fusion U-Net model based on the attention mechanism through the above steps. The model not only learns the characteristics of abnormal images in the training set, but also verifies its recognition ability on unknown data through testing on the validation set. The iterative training and parameter update process, combined with pre-set constraints, enables the model to more intelligently process input images, dynamically adjust the weights of pixels and color channels, and improve the accuracy of identifying damaged areas. The resulting target model can accurately and quickly identify and locate abnormal areas in images while meeting pre-set thresholds, significantly improving the performance and practicality of the intelligent recognition system.

[0055] In an optional embodiment, each iterative training includes the following steps: when there are T training sets in the model training set, the intelligent processing system inputs each training set in the T training sets into the encoder of the initial model, and determines T initial feature maps corresponding to the T training sets through the encoder, wherein the encoder includes S levels, each level of the encoder is used to reduce the resolution of the object image, and determines the feature map corresponding to the object image through the depthwise separable convolution and residual connection in the level, and uses the obtained feature map as the input of the next level, wherein S is an integer greater than or equal to 1, the depthwise separable convolution is used to extract the features of the object image, and the residual connection is used to add the output of the depthwise separable convolution in each level of the encoder to the input of the depthwise separable convolution, and then based on the T initial feature maps, T recognition images are generated through the output layer of the initial model, wherein the recognition image includes abnormal information corresponding to the object image, and then the model parameters of the initial model are updated according to the T recognition images and the actual abnormal object images corresponding to the T training sets.

[0056] Optionally, when the model training set contains T training samples (i.e., T abnormal images), the intelligent processing system begins iterative training. The system's first task is to feed these T training samples into the encoder of the initial model. The encoder consists of S layers, where S is an integer greater than or equal to 1, representing the depth of feature extraction. The encoder at each layer performs the following operations:

[0057] Use depth-wise separable convolution to extract image features. This operation can efficiently process image data, reduce the amount of computation, and maintain the depth and richness of features.

[0058] The residual connection is introduced to add the output of the depthwise separable convolution to the input, which helps alleviate the gradient vanishing problem in deep models and ensures the effective transmission of information between network layers and the full utilization of deep features.

[0059] Through the above process, the encoder can generate T initial feature maps from each of the T training sets. These feature maps carry key information of the abnormal image, including the shape, size, and position of the abnormal area.

[0060] Optionally, after the initial feature maps are generated, the intelligent processing system uses the output layer of the initial model to convert these feature maps into T recognition images. These recognition images contain the model's predictions of abnormal information in the object image, specifically, the possible abnormal regions within the abnormal image. This step is a key part of model learning and prediction. The recognition images generated by the output layer, using features extracted by the encoder, directly reflect the model's ability to understand and locate abnormal information.

[0061] Optionally, to enable the model to more accurately identify abnormal areas, the system needs to compare the identified images with actual abnormal object images. Here, actual abnormal object images refer to manually annotated abnormal images that indicate the actual location and extent of the abnormality. Based on the T identified images and the actual abnormal object images corresponding to the T training sets, the system calculates the error (loss) between the model predictions and the actual annotations. It then uses a backpropagation algorithm to update the model parameters and optimize model performance, ensuring that the model can more accurately identify abnormal areas in subsequent predictions and reduce false positives and negatives.

[0062] Optionally, depthwise separable convolution is an efficient convolution operation that splits standard convolution into depthwise convolution and pointwise convolution, reducing the number of parameters and computational complexity; residual connection: In neural networks, residual connection skips one or several layers and directly adds the input to the output, which helps avoid gradient vanishing in deep networks and improves the stability of model learning.

[0063] Optionally, Figure 2 is a schematic diagram of an optional residual connection structure according to an embodiment of the present application, such as Figure 2 As shown, x is the input data of the model, which is an image or a feature representation of an image. In this embodiment, the input can be an image of a paper currency to be detected; two 3x3 depthwise separable convolution layers (3x3 conv): Depthwise separable convolution is an efficient convolution operation that decomposes the standard convolution into two independent steps: depthwise convolution and pointwise convolution, thereby reducing the amount of calculation and the number of parameters. The two 3x3 depthwise separable convolution layers process the input features separately, aiming to extract local features in the image while reducing computational complexity; a 1x1 convolution layer (1x1 conv): 1x1 convolution is used to change the number of input channels or the depth of the feature map without changing its size. Figure 2It is used to match the feature dimension of the residual map to facilitate the subsequent feature addition operation; F(x) represents the residual map, which is the core idea of ​​the residual network. It designs a shortcut connection so that the network can directly pass part of the information from the input to the deeper layer, thereby alleviating the gradient disappearance or gradient explosion problem in the deep network; the output H(x) is the feature representation after the residual map and element-wise addition operation, which combines the original input with the new features learned by the depth-separable convolutional layer, aiming to make more efficient use of information while avoiding the training difficulties of the deep network.

[0064] Optionally, the labeled abnormal image dataset is input into the model, and the input feature x is passed through two 3×3 depthwise separable convolutional layers to obtain a channel feature map, which is then matched to the feature dimension through a 1×1 convolution. The residual map F(x) is obtained by element-wise addition, and then used as the input of the next module to improve the feature learning ability. The calculation method of H(x) is shown in formula (1):

[0065] H(x)=F(x)+x (1)

[0066] Here, F(x) represents the residual mapping, H(x) is the output after residual mapping, and x is the input feature. The residual mapping captures the complex differences between the input and output through F(x), while H(x) is obtained by adding the result of the residual mapping to the input feature to obtain the final output.

[0067] As can be seen from the above content, through the above detailed iterative training steps, the intelligent processing system ensures that the model can accurately extract key features and optimize model parameters to minimize prediction errors when processing a large number of abnormal images. This is not only a model training process, but also a way for the model to learn and understand the characteristics of abnormal areas in abnormal images. Through the combination of depthwise separable convolution and residual connection, the encoder can efficiently extract multi-scale features, and the update of model parameters ensures that the model can continuously improve its prediction ability until the expected recognition accuracy is achieved. Overall, this implementation method improves the model's ability to identify abnormal areas in images in complex environments through the designed encoder structure and model parameter optimization strategy, ensuring that even in low-quality images or multiple forms of abnormalities, the model can stably and accurately detect abnormal areas, providing a reliable basis for subsequent processing and decision-making. This not only improves the recognition speed, but also significantly improves the recognition accuracy.

[0068] In an optional embodiment, the intelligent processing system first uses a target mechanism to perform target processing on T initial feature maps to obtain T first feature maps, wherein the target processing is used to adjust the weights corresponding to each channel and position in the initial feature map, and then each of the T first feature maps is input into the decoder in the initial model for upsampling operation to obtain T second feature maps, wherein the decoder includes S levels, the decoder is used to reconstruct the features of the object image corresponding to the first feature map, the upsampling operation is used to improve the resolution of the first feature map, and finally, T recognition images are generated based on the T second feature maps through the output layer of the initial model.

[0069] Optionally, the initial feature map is the feature output by the encoder, which contains preliminary feature information of the abnormal area in the image; the first feature map is the feature map enhanced by CBAM, which has higher feature discrimination and representation ability, and can more accurately reflect the characteristics of the abnormal area.

[0070] Optionally, the intelligent processing system first processes the T initial feature maps using a "targeted mechanism," or CBAM. The CBAM mechanism dynamically adjusts the weights of each feature map based on its channel and location. This process allows the model to focus on features highly correlated with abnormal regions while suppressing the influence of irrelevant or background features, generating the T first feature maps. "Targeted processing" here refers to using CBAM to enhance the representation of key information in the feature maps.

[0071] Optionally, next, the intelligent processing system inputs these T first feature maps into the decoder part of the initial model. The decoder consists of S layers, and the decoder at each layer is responsible for gradually restoring the image resolution reduced by the encoding process, while reconstructing the features of the object image corresponding to the first feature map. Through upsampling operations (such as bilinear interpolation, transposed convolution, etc.), the decoder can increase the resolution of the first feature map to a level close to that of the input object image, generating T second feature maps. These second feature maps not only have improved resolution, but also integrate feature information from different levels, enabling the model to restore image details more meticulously, providing richer feature support for subsequent object recognition. Finally, the intelligent processing system uses the output layer of the initial model to generate T recognition images based on these T second feature maps. The recognition images are the model's final prediction of the abnormal information of the input object image. They intuitively show the model's judgment on the abnormal area and provide direct visual feedback for subsequent abnormality recognition and processing.

[0072] As can be seen from the above, the intelligent processing system significantly enhances the model's performance in identifying damaged banknotes by introducing the CBAM attention mechanism and upsampling operations in the decoder. The CBAM mechanism enables the model to more intelligently focus on key features in abnormal areas, enhancing feature differentiation; while the upsampling operation in the decoder restores the resolution of the feature map, improving the model's accuracy in locating abnormal areas. This series of optimization measures not only improves the model's accuracy in identifying damaged banknotes, but also enhances its robustness and generalization capabilities, ensuring that the model can stably and efficiently complete the recognition task even under varying conditions. Through this implementation, the model can more intelligently and accurately identify anomalies in images.

[0073] In an optional embodiment, the target processing includes the following steps: the intelligent processing system first extracts the spatial distribution information of T initial feature maps, and determines T first weight matrices based on the spatial distribution information corresponding to each initial feature map, wherein the spatial distribution information is used to characterize the feature intensity and distribution characteristics on each channel in the initial feature map, and the first weight matrix is ​​used to adjust the weight corresponding to each channel in the initial feature map, and then determines T third feature maps based on each initial feature map in the T initial feature maps and the first weight matrix corresponding to the initial feature map, wherein the third feature map is a weighted feature map obtained by multiplying the initial feature map with the corresponding first weight matrix, and then extracts the position information of the T third feature maps to obtain T position information, wherein the position information is used to characterize the feature distribution in the third feature map, and then determines T second weight matrices based on the T position information, wherein the second weight matrix is ​​used to adjust the weights corresponding to different positions in the third feature map, and finally determines T first feature maps based on each third feature map in the T third feature maps and the second weight matrix corresponding to the third feature map.

[0074] Optionally, the intelligent processing system first processes the T initial feature maps generated by the encoder from the T training samples. For each feature map, the system will focus on extracting its spatial distribution information, which includes the feature intensity at each pixel and its distribution characteristics in the entire image. The extraction of spatial distribution information helps the system understand the relative importance of each area in the image. Based on the extracted spatial distribution information, the intelligent processing system calculates T first weight matrices through a specific algorithm (usually an attention mechanism based on MLP or convolution operations). Each first weight matrix corresponds to the corresponding initial feature map, and its function is to adjust the weight of each channel in the feature map. This means that, guided by the first weight matrix, the system can more effectively focus on channels that carry key information, while reducing the influence of background or irrelevant channels, thereby improving the pertinence and efficiency of feature extraction.

[0075] Optionally, after determining the first weight matrix, the system performs an element-by-element multiplication of the weight matrix with the corresponding initial feature map to generate T third feature maps. These weighted feature maps (third feature maps) not only retain the spatial structure of the original feature map, but also reflect the feature intensity distribution after channel-level weighting adjustment, so that the feature importance of each channel is reflected in the final feature map.

[0076] Optionally, the intelligent processing system then continues to process the third feature map and extract its location information. The location information reflects the distribution of features in different areas of the feature map, helping the system understand which locations may be key clues to damage. The system then determines T second weight matrices based on the location information, which are used to adjust the weights at different locations in the third feature map. In this way, the system can further enhance its attention to key areas while reducing resource waste in non-key areas, thereby improving the recognition efficiency and accuracy of the model.

[0077] Optionally, the intelligent processing system multiplies each third feature map by the corresponding second weight matrix to generate T first feature maps. These first feature maps, after undergoing two attention mechanism adjustments, contain more precise feature representations of damaged banknotes, enabling the model to make more accurate judgments in the final recognition stage.

[0078] Alternatively, assume that the size of the input feature map F (the first feature map) is H×W×C, where H represents length, W represents width, and C represents the number of channels. In the channel attention module in CBAM, first, the feature map F is max-pooled and average-pooled to extract the spatial distribution information of the features, generating two 1×1×C feature maps. and These two feature maps are fed into a multilayer perceptron (MLP) network, and 1×1 convolution is used to compress the channels to (r is the compression rate), and then use 1×1 convolution to expand the channel to C; add the features output by MLP element by element, and then pass the sigmoid activation function to generate the final channel attention feature map, that is, M1 (the first weight matrix). The generated M1 is element-wise multiplied with the input feature map F as the input F of the spatial attention module ′ (Third feature map). The calculation method of the channel attention module is shown in formula (2):

[0079]

[0080] Among them, M1(F) refers to the channel attention feature map corresponding to the feature map F, σ is the sigmoid activation function, and w0 and w1 are two different weights.

[0081] Optionally, in the spatial attention module, the input features F are centered along the channel axis. ′ Perform maximum pooling and average pooling to extract feature position information and generate two 1×H×W feature maps and Next, the two feature maps are concatenated in the channel dimension; then a 7×7 convolution is used to compress them into one channel; finally, the spatial attention feature map is generated by the sigmoid activation function, namely M2 (the second weight matrix). The calculation method of the spatial attention module is shown in formula (3):

[0082]

[0083] Among them, M2(F) refers to the spatial attention feature map corresponding to the feature map F, f 7×7 It is a 7×7 convolution.

[0084] Optionally, the input feature map F passes through the channel attention module and the spatial attention module in sequence. During the training process, more weight parameters are allocated to the damage cracks to improve the feature extraction capability and reduce the influence of background noise.

[0085] As can be seen from the above, the intelligent processing system significantly enhances the model's ability to identify features in abnormal regions by embedding a two-step attention mechanism. First, by adjusting the first weight matrix, different channels in the feature map are weighted, enhancing the model's ability to capture key features. Subsequently, by determining the second weight matrix, different locations in the feature map are re-evaluated, further highlighting the importance of abnormal regions. The application of this two-step attention mechanism not only increases the model's sensitivity to damaged features but also improves its ability to resist interference in complex backgrounds.

[0086] In an optional embodiment, the intelligent processing system inputs each of the T first feature maps into S layers in the decoder in the initial model for upsampling operations to obtain T feature map sets, wherein the feature map sets include feature maps output by the first feature map in each layer in the decoder, wherein the feature maps output in each layer include the feature maps corresponding to the encoder in the initial model at that layer, and then adjusts the feature maps in each feature map set in the T feature map sets to a preset resolution to obtain T first feature map sets, and finally fuses the feature maps in each of the T first feature map sets to obtain T second feature maps.

[0087] Optionally, the intelligent processing system feeds the T first feature maps, i.e., the feature maps extracted by the encoder, into the S layers of the decoder of the initial model for upsampling. The decoder layers correspond to the encoder layers, aiming to gradually restore the image resolution while incorporating the feature maps extracted by the encoder at different layers into the decoding process to recover richer details.

[0088] Optionally, at each level, the decoder not only upsamples the feature maps but also fuses them with the corresponding feature maps from the encoder at that level. This step, called a "skip connection," helps preserve and utilize the detailed features learned by the encoder. Upsampling is typically implemented using techniques such as bilinear interpolation, nearest neighbor interpolation, or transposed convolution. Its purpose is to amplify the size of the feature maps to restore lost spatial information and provide a more detailed image foundation for subsequent feature fusion and abnormal area localization.

[0089] Optionally, after upsampling and feature fusion, the intelligent processing system obtains the output feature maps from each level of the decoder to form T feature map sets corresponding to the T first feature maps. Each set contains feature information of different scales, reflecting the various details of the abnormal area in the image. The system then adjusts the feature maps in each feature map set in the T feature map sets to a preset resolution (same resolution), that is, sets a standard output size to ensure that all feature maps have the same size to facilitate the next step of feature fusion. The choice of preset resolution is based on the size and accuracy requirements of the model output image. Finally, the intelligent processing system fuses the feature maps in the adjusted T first feature map sets to obtain T second feature maps. The fusion process may be achieved through technologies such as weighted averaging, splicing or attention mechanism. Its core purpose is to find the optimal feature representation in feature maps of different scales to improve the model's recognition accuracy and robustness for abnormal areas.

[0090] Optionally, the fusion operation ensures that the model can not only identify large-scale abnormal areas, such as cracks in the image, but also capture small-scale details such as spots and pits, thereby outputting more comprehensive and accurate abnormal area positioning results.

[0091] Optionally, the intelligent processing system uses bilinear interpolation to upsample the different resolutions of each layer of scale features to a common resolution, then concatenates all feature maps of different scales into a tensor, obtains the features of multi-scale feature prediction fusion through a 1×1 convolution, and finally outputs the abnormal area in the image.

[0092] Optionally, Figure 3 is a structural diagram of an optional U-Net model according to an embodiment of the present application, such as Figure 3As shown, the U-Net model includes multi-level encoders, multi-level decoders, channel-space attention mechanisms, and multi-scale feature prediction fusion modules. Among them, the encoder of the U-Net model is responsible for converting the input image into a compact feature representation. In this embodiment, the encoder part contains multiple residual connection modules, each of which is composed of a depth-separable convolution layer. Depth-separable convolution reduces the number of parameters and computational complexity by decomposing the standard convolution into depth-wise convolution and point-by-point convolution, while the residual connection helps the model learn deeper features while avoiding the problem of gradient disappearance or explosion, ensuring stable training of the model; the decoder part is responsible for restoring the features extracted by the encoder to the resolution of the original image in order to make accurate pixel-level predictions. By connecting with the features of the encoder (usually achieved through skip connections), the decoder can utilize multi-scale information to improve the accuracy of predictions; the multi-scale feature prediction fusion module is responsible for fusing the multi-scale features output by the decoder. First, the features of different resolutions are upsampled to the same size through bilinear interpolation, and then they are spliced ​​together. Finally, a 1x 1. The convolution layer integrates features to generate the final fused feature map, enabling the model to utilize information from different levels to enhance its understanding of image details and its ability to detect anomalies. CBAM is a lightweight attention mechanism located between the encoder and decoder, including a channel attention module and a spatial attention module. The channel attention module first performs a pooling operation on the feature map to extract spatial information, and then adjusts the weight of each channel through a multi-layer perceptron network to highlight important features. The spatial attention module generates two feature maps through maximum pooling and average pooling, and then splices them in the channel dimension. After the convolution operation, a spatial weight map is generated through a sigmoid activation function to enhance attention to specific locations and suppress interference from irrelevant background.

[0093] As can be seen from the above, the intelligent processing system effectively utilizes the decoder portion of the initial model to perform multi-scale feature fusion, generating a highly detailed feature map of abnormal regions. This process not only restores the spatial details of the image but also, by fusing features at different scales, enhances the model's ability to capture abnormal details, including large-scale cracks and small-scale spots and pits. The resulting second feature map contains more comprehensive abnormality information, providing richer and more accurate feature descriptions for automated recognition of abnormal images, significantly improving the model's recognition effectiveness and application range. This implementation is key to the model's ability to accurately identify and locate abnormal regions on damaged banknotes, providing a solid technical foundation for subsequent decision-making and processing.

[0094] In an optional embodiment, the intelligent processing system updates the model parameters of the initial model based on T recognition images and actual abnormal object images corresponding to T training sets, including: first determining the error value between each of the T recognition images corresponding to the T training sets and the actual abnormal object image corresponding to the training set to obtain T error values, and then based on the T error values, updating the model parameters of the initial model through a first algorithm and back propagation, wherein back propagation is used to determine the influencing factors of the model parameters of the initial model on the model error, and the first algorithm is used to adjust the model parameters of the initial model based on the influencing factors.

[0095] Optionally, the intelligent processing system compares the identified images generated for each training set with the corresponding actual abnormal object images (with manually annotated damage locations). Using a loss function (such as cross entropy or mean squared error), the system calculates the error between each identified image and the actual abnormal object image, i.e., the degree of difference between the identified result and the actual abnormal area. This process yields T error values ​​for each of the T identified images in the T training sets. After calculating these T error values, the intelligent processing system uses a backpropagation algorithm to determine which model parameters (such as weights and biases) have the greatest impact on these error values. The backpropagation algorithm uses a gradient descent mechanism to calculate the gradient of the loss function with respect to the model parameters, i.e., the degree of influence of the error on the parameters. Next, the system uses a first algorithm (typically an optimization algorithm, such as Adam (Adaptive Moment Estimation)) to adjust the model parameters based on the calculated gradient to minimize the error value. This algorithm updates the parameters according to a preset learning rate, enabling the model to gradually learn and optimize its ability to identify abnormal areas in the image.

[0096] Optionally, the above process is repeated in each training iteration until the number of model iterations exceeds a preset number or the model's training error falls below a preset threshold. This cycle ensures that model parameter optimization continues until stable performance and accurate recognition are achieved. After each training iteration, the system also evaluates model performance using a validation set to ensure that the model not only performs well on the training set but also maintains good generalization capabilities on unseen image sets.

[0097] From the above content, it can be seen that by accurately calculating the error, the intelligent system can adjust the model parameters in a targeted manner, reduce the deviation of the model prediction, and improve the recognition accuracy of abnormal areas. Ultimately, the target model can more accurately detect and locate various abnormalities on the image, such as cracks, stains, pits, defects, etc., and at the same time has strong generalization capabilities and can run stably on different image data, significantly improving the efficiency and accuracy of automatic recognition of abnormalities. Through iterative training and error-based parameter updates, the intelligent processing system not only optimizes the performance of the model, but also enhances the model's sensitivity to abnormal features, and maintains a high level of recognition capability even in complex situations such as lighting changes and uneven image quality. This provides a solid technical guarantee for financial institutions to handle the automatic recognition of large numbers of abnormal images (such as damaged paper currency), reduces human misjudgment, and improves service quality and customer satisfaction.

[0098] In an optional embodiment, Figure 4 This is a flow chart of an optional image processing method according to an embodiment of the present application. Figure 2 ,like Figure 2 As shown in the figure, firstly, a dataset of object images with different degrees of abnormality is collected, where the dataset can come from artificially taken abnormal object images and crack datasets in papers, including damaged cracks, stains, pits, defects, etc.; secondly, the collected dataset is finely annotated through professional annotation software and the basic framework of the U-Net model is built, where the annotated target is the abnormal area to guide the model to learn which parts need to be focused on and generate an XML file containing abnormal key points; based on the U-Net structure, residual connections are first used in each basic block of the deep U-Net model, and the ordinary convolution in the U-Net is replaced by the depthwise separable convolution, which improves the utilization of feature information. At the same time, the complexity of the model parameters is reduced; then the CBAM attention mechanism is introduced in the middle part of the model to improve the representation ability of the model; then, the fusion decoder outputs features of different scales to obtain richer feature information; then the preprocessed data set is input into the improved U-Net model for training, allowing the model to learn how to identify and locate abnormal areas, and finally optimize the model parameters to improve recognition accuracy; once the model training is completed, it can be used for real-time image input to detect unknown abnormal images. The real-time input value of the abnormal image can be collected and analyzed in the improved U-Net model. After model processing, the output is the result of segmentation of the input abnormal image, that is, a clear mark of the abnormal area.

[0099] The present application also provides an image processing device. It should be noted that the image processing device of the present application can be used to execute the image processing method provided in the present application. The image processing device provided in the present application is introduced below.

[0100] According to an embodiment of the present application, a device for implementing the above-mentioned image processing method is also provided. Figure 5 is a schematic diagram of an optional image processing device according to an embodiment of the present application, such as Figure 5 As shown, the device includes: a receiving unit 501 and a determining unit 502.

[0101] Optionally, the receiving unit 501 is used to receive the target object image; the determining unit 502 is used to input the target object image into the target model, extract the target features of the target object image based on the prior knowledge learned by the target model in the model training stage, and determine the target image corresponding to the target object image based on the target features; wherein the target features are used to characterize the features of the abnormal area in the target object image, and the target model is a model obtained by updating the initial model according to the model parameters, wherein the model parameters are a parameter combination that meets preset conditions, and the initial model is a model constructed based on a target mechanism, and the target mechanism is used to dynamically adjust the weights corresponding to the pixels of the model input image and the weights corresponding to the image color channels, wherein the preset conditions are used to constrain the error value between the target image of the target model and the actual abnormal object image to be less than a preset threshold.

[0102] Optionally, the determination unit 502 includes: a first acquisition subunit and a first processing subunit. The first acquisition subunit is configured to acquire N object images, where N is an integer greater than 1, and each object image includes annotated actual abnormality information; and the first processing subunit is configured to perform multiple iterative training and verification operations on the initial model based on the N object images to obtain a target model, wherein the multiple iterative training operations are used to determine model parameters of the target model, and the verification operations are used to verify performance parameters of the target model.

[0103] Optionally, the first processing subunit includes: a first partitioning module, a first training module, a first updating module, and a first verification module. The first partitioning module is configured to partition N object images into a model training set and a model verification set; the first training module is configured to input the model training set into an initial model for iterative training until the number of model iterations exceeds a preset number or the model training error is less than a preset threshold, thereby obtaining model parameters of the initial model; the first updating module is configured to update the initial model to a first model based on the model parameters; and the first verification module is configured to verify the first model based on the model verification set to obtain a target model.

[0104] Optionally, the first training module includes: a first determination submodule, a first generation submodule, and a first update submodule. The first determination submodule is used to input each of the T training sets into the encoder of the initial model when there are T training sets in the model training set, and determine T initial feature maps corresponding to the T training sets through the encoder, wherein the encoder includes S levels, each level of the encoder is used to reduce the resolution of the object image, and determine the feature map corresponding to the object image through the depthwise separable convolution and residual connection in the level, and use the obtained feature map as the input of the next level, wherein S is an integer greater than or equal to 1, the depthwise separable convolution is used to extract the features of the object image, and the residual connection is used to add the output of the depthwise separable convolution in each level of the encoder to the input of the depthwise separable convolution; the first generation submodule is used to generate T recognition images through the output layer of the initial model based on the T initial feature maps, wherein the recognition images include abnormality information corresponding to the object image; the first update submodule is used to update the model parameters of the initial model according to the T recognition images and the actual abnormal object images corresponding to the T training sets.

[0105] Optionally, the first generation submodule includes: a first processing component, a second processing component, and a first generation component. The first processing component is used to perform target processing on T initial feature maps using a target mechanism to obtain T first feature maps, wherein the target processing is used to adjust the weights corresponding to each channel and position in the initial feature map; the second processing component is used to input each first feature map in the T first feature maps into the decoder in the initial model for upsampling operation to obtain T second feature maps, wherein the decoder includes S layers, the decoder is used to reconstruct the features of the object image corresponding to the first feature map, and the upsampling operation is used to improve the resolution of the first feature map; the first generation component is used to generate T recognition images based on the T second feature maps through the output layer of the initial model.

[0106] Optionally, the first processing component includes: a first extraction subcomponent, a first determination subcomponent, a second extraction subcomponent, a second determination subcomponent, and a third determination subcomponent. Among them, the first extraction subcomponent is used to extract the spatial distribution information of T initial feature maps, and determine T first weight matrices based on the spatial distribution information corresponding to each initial feature map, wherein the spatial distribution information is used to characterize the feature intensity and distribution characteristics on each channel in the initial feature map, and the first weight matrix is ​​used to adjust the weight corresponding to each channel in the initial feature map; the first determination subcomponent is used to determine T third feature maps based on each initial feature map in the T initial feature maps and the first weight matrix corresponding to the initial feature map, wherein the third feature map is a weighted feature map obtained by multiplying the initial feature map with the corresponding first weight matrix; the second extraction subcomponent is used to extract the position information of the T third feature maps to obtain T position information, wherein the position information is used to characterize the feature distribution in the third feature map; the second determination subcomponent is used to determine T second weight matrices based on the T position information, wherein the second weight matrix is ​​used to adjust the weights corresponding to different positions in the third feature map; the third determination subcomponent is used to determine T first feature maps based on each third feature map in the T third feature maps and the second weight matrix corresponding to the third feature map.

[0107] Optionally, the second processing component includes: a first processing subcomponent, a first adjustment subcomponent, and a first fusion subcomponent. The first processing subcomponent is used to input each of the T first feature maps into the S layers of the decoder in the initial model for upsampling operations to obtain T feature map sets, wherein the feature map sets include feature maps output by the first feature map in each layer of the decoder, wherein the feature maps output in each layer include the feature maps corresponding to the encoder in the initial model at that layer; the first adjustment subcomponent is used to adjust the feature maps in each feature map set in the T feature map sets to a preset resolution to obtain T first feature map sets; the first fusion subcomponent is used to fuse the feature maps in each of the T first feature map sets to obtain T second feature maps.

[0108] Optionally, the first updating submodule includes: a first determining component and a first updating component. The first determining component is configured to determine the error between each of the T recognition images corresponding to the T training sets and the actual abnormal object image corresponding to the training sets, thereby obtaining T error values; and the first updating component is configured to update the model parameters of the initial model based on the T error values ​​using a first algorithm and back propagation, wherein the back propagation is configured to determine the factors affecting the model error caused by the model parameters of the initial model, and the first algorithm is configured to adjust the model parameters of the initial model based on the factors affecting the model error.

[0109] According to another aspect of the present application, a computer-readable storage medium is provided, which includes a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned image processing method.

[0110] According to another aspect of the present application, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned image processing method.

[0111] According to another aspect of an embodiment of the present application, a computer program product is further provided, comprising computer instructions, which implement the steps of the above-mentioned image processing method when executed by a processor.

[0112] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0113] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0115] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0116] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0118] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: include: receiving a target object image; Inputting the target object image into a target model, extracting target features of the target object image based on prior knowledge learned by the target model during a model training phase, and determining a target image corresponding to the target object image based on the target features; In which, the target features are used to characterize the features of the abnormal area in the target object image, the target model is a model obtained by updating the initial model according to the model parameters, wherein the model parameters are a parameter combination that meets the preset conditions, and the initial model is a model constructed based on the target mechanism, and the target mechanism is used to dynamically adjust the weights corresponding to the pixels of the model input image and the weights corresponding to the image color channels, wherein the preset conditions are used to constrain the error value between the target image of the target model and the actual abnormal object image to be less than a preset threshold.

2. The image processing method according to claim 1, wherein: The target model is obtained by the following steps: Acquire N object images, where N is an integer greater than 1, and each of the object images includes annotated actual abnormality information; The initial model is subjected to multiple iterative training and verification operations according to N object images to obtain the target model, wherein the multiple iterative trainings are used to determine the model parameters of the target model, and the verification operations are used to verify the performance parameters of the target model.

3. The image processing method according to claim 2, wherein: Performing multiple iterative training and validation operations on the initial model according to N object images to obtain the target model includes: Dividing the N object images into a model training set and a model verification set; Inputting the model training set into the initial model for iterative training until the number of model iterations is greater than a preset number or the training error of the model is less than a preset threshold, thereby obtaining the model parameters of the initial model; Updating the initial model to a first model according to the model parameters; The verification operation is performed on the first model according to the model verification set to obtain the target model.

4. The image processing method according to claim 3, wherein: Each training iteration consists of the following steps: When there are T training sets in the model training set, each of the T training sets is input into the encoder of the initial model, and T initial feature maps corresponding to the T training sets are determined by the encoder, wherein the encoder includes S levels, each level of the encoder is used to reduce the resolution of the object image, and determine the feature map corresponding to the object image through the depthwise separable convolution and residual connection in the level, and use the obtained feature map as the input of the next level, wherein S is an integer greater than or equal to 1, the depthwise separable convolution is used to extract features of the object image, and the residual connection is used to add the output of the depthwise separable convolution in each level of the encoder to the input of the depthwise separable convolution; Based on the T initial feature maps, generating T recognition images through the output layer of the initial model, wherein the recognition images include abnormality information corresponding to the object image; The model parameters of the initial model are updated according to the T recognition images and the actual abnormal object images corresponding to the T training sets.

5. The image processing method according to claim 4, wherein: Based on the T initial feature maps, generating T recognition images through the output layer of the initial model, including: Performing target processing on the T initial feature maps using the target mechanism to obtain T first feature maps, wherein the target processing is used to adjust the weight corresponding to each channel and position in the initial feature maps; Inputting each of the T first feature maps into a decoder in the initial model for an upsampling operation to obtain T second feature maps, wherein the decoder includes S layers, the decoder is used to reconstruct features of the object image corresponding to the first feature map, and the upsampling operation is used to increase the resolution of the first feature map; Generate T recognition images based on the T second feature maps through the output layer of the initial model.

6. The image processing method according to claim 5, characterized in that The target processing includes the following steps: Extracting spatial distribution information of T initial feature maps, and determining T first weight matrices based on the spatial distribution information corresponding to each initial feature map, wherein the spatial distribution information is used to characterize the feature intensity and distribution characteristics of each channel in the initial feature map, and the first weight matrix is ​​used to adjust the weight corresponding to each channel in the initial feature map; Determining T third feature maps according to each of the T initial feature maps and the first weight matrix corresponding to the initial feature map, wherein the third feature map is a weighted feature map obtained by multiplying the initial feature map by the corresponding first weight matrix; Extracting position information of T pieces of the third feature maps to obtain T pieces of position information, wherein the position information is used to characterize feature distribution in the third feature map; Determining T second weight matrices based on the T pieces of position information, wherein the second weight matrices are used to adjust weights corresponding to different positions in the third feature map; T first feature maps are determined according to each of the T third feature maps and a second weight matrix corresponding to the third feature map.

7. The image processing method according to claim 5, characterized in that: Inputting each of the T first feature maps into the decoder in the initial model for upsampling operation to obtain T second feature maps, including: Inputting each of the T first feature maps into the S layers of the decoder in the initial model for upsampling, to obtain T feature map sets, wherein the feature map sets include feature maps output by the first feature map in each layer of the decoder, wherein the feature map output by each layer includes the feature map corresponding to the encoder at the layer in the initial model; Adjusting the feature map in each of the T feature map sets to a preset resolution to obtain T first feature map sets; The feature maps in each of the T first feature map sets are fused to obtain T second feature maps.

8. The image processing method according to claim 4, wherein: Updating the model parameters of the initial model according to the T recognition images and the T actual abnormal object images corresponding to the training sets, including: Determine an error value between each of the T recognition images corresponding to the T training sets and an actual abnormal object image corresponding to the training set, to obtain T error values; Based on T error values, the model parameters of the initial model are updated through a first algorithm and back propagation, wherein the back propagation is used to determine the influencing factors of the model parameters of the initial model on the model error, and the first algorithm is used to adjust the model parameters of the initial model based on the influencing factors.

9. An image processing device, characterized in that: include: A receiving unit, configured to receive an image of a target object; a determination unit, configured to input the target object image into a target model, extract target features of the target object image based on prior knowledge learned by the target model during a model training phase, and determine a target image corresponding to the target object image based on the target features; In which, the target features are used to characterize the features of the abnormal area in the target object image, the target model is a model obtained by updating the initial model according to the model parameters, wherein the model parameters are a parameter combination that meets the preset conditions, and the initial model is a model constructed based on the target mechanism, and the target mechanism is used to dynamically adjust the weights corresponding to the pixels of the model input image and the weights corresponding to the image color channels, wherein the preset conditions are used to constrain the error value between the target image of the target model and the actual abnormal object image to be less than a preset threshold.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the image processing method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: The apparatus comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the image processing method according to any one of claims 1 to 8.

12. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

Images