Method and apparatus for model training, terminal and storage medium

Abstract

Embodiments of the present application relate to the technical field of digital medical treatment, and specifically provide a training method and device of an image processing model, a terminal and a storage medium. The method comprises: obtaining a first picture and a second picture; performing image filtering processing on the first picture by using an initial image processing model to obtain a third picture; performing image segmentation on the second picture to obtain a first foreground image and a first background image, and performing image segmentation on the third picture to obtain a second foreground image and a second background image; obtaining a contour segmentation error according to the first foreground image, the first background image, the second foreground image and the second background image; calculating a pixel error of the second picture and the third picture; constructing a loss function according to the contour segmentation error and the pixel error; and iteratively updating the initial image processing model based on target training data and the loss function to obtain a target image processing model. The model improves the reliability of image filtering and provides good support for subsequent image tasks.

Description

Technical Field

[0001] This application relates to the field of digital medical technology, and in particular to a training method, apparatus, terminal and storage medium for an image processing model. Background Technology

[0002] In recent decades, the rapid development of algorithm research in computer architecture has led to the widespread application of digital image processing technology in fields such as medicine, investigation, and industry. Image filtering is a crucial aspect of digital image processing and a vital step in image preprocessing. Compared to other preprocessing steps in computer vision tasks such as classification, object detection, and object segmentation, its primary purpose is to improve image quality and reduce noise. For example, in the financial sector, when recognizing invoices, preprocessing the invoice image is necessary to obtain a clearer image, facilitating subsequent object detection and segmentation. Reducing image noise is crucial for invoice recognition. Similarly, in the medical field, when classifying medical images, reducing noise during image data preprocessing is essential to ensure accurate classification results and improve classification accuracy.

[0003] There are many existing image filtering methods, including median filtering, mean filtering, bilateral filtering, Gaussian filtering, etc. However, there is a lack of unified evaluation criteria for selecting filtering methods. Usually, many related experiments are required to determine the filter. When carrying out new tasks or using new datasets, similar experiments need to be repeated, which is extremely cumbersome. Summary of the Invention

[0004] The main objective of this application is to provide a training method, apparatus, terminal, and storage medium for an image processing model, aiming to solve the problem of cumbersome filter selection in image filtering when processing patient images in the field of digital healthcare, thereby improving the versatility of image filters and thus improving the efficiency of patient image processing.

[0005] In a first aspect, embodiments of this application provide a method for training an image processing model, comprising:

[0006] Obtain target training data, wherein the target training data includes at least a first image and a second image, wherein the second image is an image obtained by filtering the first image.

[0007] The first image is processed by image filtering using the initial image processing model to obtain the third image.

[0008] The second image is segmented to obtain a first foreground image and a first background image of the second image, and the third image is segmented to obtain a second foreground image and a second background image of the third image.

[0009] The contour segmentation error between the second image and the third image is obtained based on the first foreground image, the first background image, the second foreground image, and the second background image.

[0010] Calculate the pixel difference between the pixels at the same position in the second image and the third image to obtain the pixel error between the second image and the third image.

[0011] A loss function is constructed based on the contour segmentation error and the pixel error.

[0012] The initial image processing model is iteratively updated based on the target training data and the loss function to obtain the target image processing model.

[0013] Secondly, embodiments of this application also provide an image processing model training apparatus, comprising:

[0014] The data acquisition module is used to obtain target training data, wherein the target training data includes at least a first image and a second image, and the second image is an image obtained by filtering the first image.

[0015] The data recognition module is used to perform image filtering on the first image using the initial image processing model to obtain the third image.

[0016] The data segmentation module is used to perform image segmentation on the second image to obtain a first foreground image and a first background image of the second image, and to perform image segmentation on the third image to obtain a second foreground image and a second background image of the third image.

[0017] The first data processing module is used to obtain the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image.

[0018] The second data processing module is used to calculate the pixel difference between pixels at the same position in the second image and the third image, so as to obtain the pixel error between the second image and the third image.

[0019] The data analysis module is used to construct a loss function based on the contour segmentation error and the pixel error.

[0020] The data update module is used to iteratively update the initial image processing model based on the target training data and the loss function to obtain the target image processing model.

[0021] Thirdly, embodiments of this application also provide a terminal device, the terminal including a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of a training method for any of the image processing models provided in this specification.

[0022] Fourthly, embodiments of this application also provide a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the training method for the image processing model as provided in any of the embodiments of this application.

[0023] This application provides a training method, apparatus, terminal, and storage medium for an image processing model. The training method includes obtaining target training data, which at least includes a first image and a second image, wherein the second image is an image obtained by filtering the first image; performing image filtering on the first image using an initial image processing model to obtain a third image; performing image segmentation on the second image to obtain a first foreground image and a first background image of the second image, and performing image segmentation on the third image to obtain a second foreground image and a second background image of the third image; and obtaining a second image and a third image based on the first foreground image, the first background image, the second foreground image, and the second background image. The contour segmentation error between images is calculated; the pixel difference between pixels at the same position in the second and third images is calculated to obtain the pixel error between the second and third images; a loss function is constructed based on the contour segmentation error and the pixel error; the initial image processing model is iteratively updated based on the target training data and the loss function to obtain the target image processing model. The obtained target image processing model can then be used to filter patient images or medical images. Since the image data and scenes are diverse during model training, the target image processing model can be used for multiple visual scenes and medical tasks, thereby improving the versatility of the image processing model and effectively solving the cumbersome problem of filter selection in image filtering. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a training method for an image processing model provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of the network structure of the image recognition model vgg16 provided in the embodiments of this application;

[0027] Figure 3 This is a schematic diagram of the network structure of an image processing model provided in an embodiment of this application;

[0028] Figure 4 A schematic diagram of the module structure of a training device for an image processing model provided in an embodiment of this application;

[0029] Figure 5 This is a schematic block diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0032] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0033] In recent decades, the rapid development of algorithm research in computer architecture has led to the widespread application of digital image processing technology in fields such as medicine, investigation, and industry. Image filtering is a crucial aspect of digital image processing and a vital step in image preprocessing. Compared to other preprocessing steps in computer vision tasks such as classification, object detection, and object segmentation, its primary purpose is to improve image quality and reduce noise. For example, in the financial sector, when recognizing invoices, preprocessing the invoice image is necessary to obtain a clearer image, facilitating subsequent object detection and segmentation. Reducing image noise is crucial for invoice recognition. Similarly, in the medical field, when classifying medical images, reducing noise during image data preprocessing is essential to ensure accurate classification results and improve classification accuracy.

[0034] There are many existing image filtering methods, including median filtering, mean filtering, bilateral filtering, Gaussian filtering, etc. However, there is a lack of unified evaluation criteria for selecting filtering methods. Usually, many related experiments are required to determine the filter. When carrying out new tasks or using new datasets, similar experiments need to be repeated, which is extremely cumbersome.

[0035] This application provides a training method, apparatus, terminal, and storage medium for an image processing model. The training method for the image processing model can be applied to a terminal device, which can be a tablet computer, laptop computer, personal digital assistant, wearable device, or server. The server can be a standalone server or a server cluster.

[0036] This application provides a training method, apparatus, terminal, and storage medium for an image processing model. The training method includes obtaining target training data, which includes at least a first image and a second image, wherein the second image is an image obtained by filtering the first image; performing image filtering on the first image using an initial image processing model to obtain a third image; performing image segmentation on the second image to obtain a first foreground image and a first background image of the second image, and performing image segmentation on the third image to obtain a second foreground image and a second background image of the third image; and obtaining the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image. The contour segmentation error between images is calculated; the pixel difference between the corresponding pixels at the same position in the second and third images is calculated to obtain the pixel error between the second and third images; a loss function is constructed based on the contour segmentation error and the pixel error; the initial image processing model is iteratively updated based on the target training data and the loss function to obtain the target image processing model. The obtained target image processing model can then be used to filter patient images or medical images. Since the image data and scenes are diverse during model training, the target image processing model can be used for multiple visual scenes and medical tasks, thereby improving the versatility of the image processing model and effectively solving the cumbersome problem of filter selection in image filtering.

[0037] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0038] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a training method for an image processing model provided in an embodiment of this application.

[0039] like Figure 1As shown, the training method of this image processing model includes steps S1 to S7.

[0040] Step S1: Obtain target training data, wherein the target training data includes at least a first image and a second image, and the second image is an image obtained by filtering the first image.

[0041] For example, image filtering, which suppresses noise in a target image while preserving as much detail as possible, is an indispensable operation in image preprocessing. Its effectiveness directly impacts the efficiency and reliability of subsequent image processing and analysis. Therefore, in computer vision tasks such as classification, object detection, and object segmentation, image filtering is often incorporated into data augmentation when training related models. However, due to the variety of image filtering methods, including median filtering, mean filtering, bilateral filtering, Gaussian filtering, etc., and the lack of a unified evaluation standard for selecting filtering methods, numerous experiments are usually required to determine the filter. Repeating similar experiments is necessary when working on new tasks or using new datasets, making the process extremely cumbersome.

[0042] For example, when classifying patient images, it is necessary to extract features from the patient images and then classify the images based on the extracted features. To improve the accuracy of patient image classification, the patient images need to be filtered. At this time, it is necessary to select an appropriate filtering method from the filtering methods according to the patient images. The selection of filtering methods is required for different patient images, which is cumbersome, increases the overall time consumption of patient image classification, and affects the overall user experience.

[0043] To address the issues of cumbersome operation and lack of versatility in selecting image filters, this application adopts a scheme of training an image filter model. First, the target training data required by the image processing model is obtained. The target training data includes a first image and a second image, where the second image is the first image obtained after filtering.

[0044] For example, the first image can be processed by an image filter to obtain the second image. The image filter can be a filter constructed with different parameters using different methods such as median filtering, mean filtering, bilateral filtering, and Gaussian filtering. The filtered images are then subjected to feature extraction. When the feature result of the filtered image after feature extraction is consistent with the feature extraction result of the first image, the filtered image is considered to meet the requirements for being used as a training sample. That is, the filtered image retains the features of the first image while also reducing the noise of the first image.

[0045] For example, when image processing models are used to support prescription recognition in the field of digital healthcare, they need to collect relevant images of prescriptions and then pass these images through image filters to obtain filtered images to support subsequent training. Similarly, when image processing models are used to support case image classification in the field of digital healthcare, they need to collect images of relevant user cases and then pass these images through image filters to obtain filtered images to meet the requirements of subsequent image classification.

[0046] In some embodiments, obtaining the target training data includes: obtaining initial training data, wherein the initial training data includes at least an initial training image and first feature information, the first feature information being feature information of a target object in the initial training image; performing image preprocessing on the initial training image to obtain an initial processed image of the initial training image; and determining the target training data based on the first feature information, the initial processed image, and the image recognition model.

[0047] For example, the initial training model includes initial training images and first feature information containing feature information of the target object in the initial training images. The initial training images can be custom-labeled according to actual needs or can use open-source image data. The initial training data is filtered through an image filter to obtain an initial processed image. The image filter can be constructed using different methods and parameters, such as median filtering, mean filtering, bilateral filtering, and Gaussian filtering. The initial processed image is then processed by an image recognition model to obtain feature information from the initial processed image. The feature information in the initial processed image is compared with the first feature information. If they are the same, the training pair consisting of the initial processed image and the initial training image is determined to be usable as target training data.

[0048] For example, the initial training model includes the initial training image and the target object category and location as the first feature information. A median filter is selected as the image filter. The initial training image is passed through the median filter to obtain the initial processed image. The initial processed image is then processed by an image recognition model to extract features, obtaining the predicted target object category and predicted target object location. The target object category is compared with the predicted target object category, and the target object location is compared with the predicted target object location. If both comparison results are consistent, the image pair consisting of the initial training image and the initial processed image meets the requirements for use as target training data; otherwise, the image pair does not meet the requirements for use as target training data.

[0049] For example, when the initial training images are prescription recognition data in the field of digital healthcare, the target object category can be drug type, drug quantity, drug administration time, etc. If the initial processed image obtained after the initial training images are processed by an image filter can accurately identify the information corresponding to the target object category, then it is considered that the filtered image can still clearly identify the target object in the image while reducing image noise.

[0050] In some embodiments, the step of preprocessing the initial training image to obtain the initial processed image of the initial training image includes: obtaining basic information of the initial training image in the initial training data, wherein the basic information includes at least the number of channels; and filtering the initial training image according to the initial training image and the number of channels to obtain the initial processed image corresponding to the initial training image.

[0051] For example, the different data sources used to acquire the initial training images result in variations in image data quality, type, size, and other representations. Image data may be color or grayscale, and different images require different data processing methods. Therefore, it is necessary to first obtain the image data format, which can also be expressed as the number of image channels, and then perform image filtering processing based on the number of channels for different images.

[0052] For example, when an image has 3 channels, the data from each of the 3 channels can be filtered separately. Then, the results of the filtering of the 3 channels can be summed and averaged to obtain the filtered image, i.e., the initial processed image. Alternatively, a condition can be added: when the number of channels in the image is less than 3, no filtering is performed, and the corresponding initial training image is filtered instead, i.e., image filtering is only performed on color images.

[0053] In some embodiments, the image recognition model includes a feature extraction network and an image classification network. The step of determining the target training data based on the first feature information, the initial processed image, and the image recognition model includes: using the feature extraction network to extract features from the initial processed image to obtain a feature extraction result; using the image classification network to classify the feature extraction result to obtain second feature information of the initial processed image; and matching the first feature information and the second feature information to determine the target training data.

[0054] For example, the image recognition model includes a feature extraction network and an image classification network. The feature extraction network is mainly used to extract image features, and the image classification network is mainly used to determine the category based on the results of image feature extraction.

[0055] For example, image recognition models can be selected from open-source models such as ResNet50, VGG16, and AlexNet. A schematic diagram of the network structure of the VGG16 model is shown below. Figure 2 As shown, the initial processed image is passed through any image recognition model to obtain the second feature information corresponding to the initial processed image. Alternatively, the initial processed image can be passed through ResNet50, VGG16, and AlexNet sequentially to obtain the first second feature information, the second second feature information, and the third second feature information. The first second feature information, the second second feature information, and the third second feature information are then matched with the first feature information. When the matching results all meet the preset conditions, the image pair consisting of the initial training image and the initial processed image is determined to be the target training data.

[0056] For example, the initial processed image obtained from the initial training image after image preprocessing is transmitted to the image processing model, and the second feature information corresponding to the initial processed image is extracted. The first feature information and the second feature information are matched. When the matching results of the first feature information and the second feature information are the same or the error meets the preset condition, the image pair composed of the initial training image and the initial processed image is determined to be the target training data. When the matching results of the first feature information and the second feature information are not the same and the error is too large, the image pair composed of the initial training image and the initial processed image cannot be used as the target training data.

[0057] For example, the first feature information corresponding to the initial training image is the target object position: [(10, 20), (10, 40), (30, 40), (30, 20)], and the target object category is: cat. The second feature information corresponding to the initial processed image is the target object position: [(12, 22), (12, 42), (32, 42), (32, 22)], and the target object category is: cat. When the first and second features are matched, the error difference is two pixels. If the preset matching condition is an error within 5 pixels, then the image pair consisting of the initial training image and the initial processed image can be used as target training data under this condition. If the preset matching condition is an error within 1 pixel, then the image pair consisting of the initial training image and the initial processed image cannot be used as target training data under this condition.

[0058] Step S2: Use the initial image processing model to perform image filtering on the first image to obtain the third image.

[0059] For example, the first image in the target training data is processed by an initial image processing model to obtain a third image. This third image is then used to establish a loss function based on the second and third images, thereby optimizing the parameters of the initial image processing model and making the image processed by the model more suitable for practical needs. The structure of the initial image processing model can be designed according to requirements, as long as it can learn how to process images from the target training data.

[0060] For example, an initial image processing model can consist of multiple layers of convolutional networks, extracting different image features at different convolutional network layers to achieve the purpose of image processing.

[0061] In some embodiments, the image processing model includes a first convolutional network, a second convolutional network, and a third convolutional network. The first image is processed using the initial image processing model to obtain a third image, including: extracting local feature information from the first image using the first convolutional network; extracting global feature information from the first image using the second convolutional network; and processing the global feature information using the third convolutional network to obtain the third image.

[0062] For example, the image processing model includes a three-layer convolutional network: a first convolutional network, a second convolutional network, and a third convolutional network. First, the first convolutional network extracts local feature information from the first image. At this stage, the features extracted by the first convolutional network are closer to the input and contain more pixel-level information, primarily fine-grained information such as color, texture, edges, and corners. The first convolutional network captures more detailed information. Second, the local feature information is then processed by the second convolutional network to extract global feature information from the first image. The features extracted by the second convolutional network are closer to the output and contain more abstract information. The receptive field increases, the overlap between receptive fields increases, and image information is compressed, aiming to obtain overall image information. Finally, the global feature information is processed by the third convolutional network to obtain the third image.

[0063] For example, the structure of an image processing model can be designed such that the network structure contains three convolutional layers, as shown in the figure. Figure 3 As shown, all convolutional kernel sizes are set to 3x3. The first convolutional network is designed with 3 input channels and 32 output channels. The second convolutional network has 32 input channels and 32 output channels. The third convolutional network has 32 input channels and 3 output channels. The parameters are continuously updated through model training to obtain the image processing model. The number of channels in the convolutional networks can be adjusted according to the amount of training data or the model training environment.

[0064] Step S3: Perform image segmentation on the second image to obtain a first foreground image and a first background image of the second image, and perform image segmentation on the third image to obtain a second foreground image and a second background image of the third image.

[0065] For example, the second image in the target training data needs to be segmented according to the target object. The second image is the image obtained by filtering the first image. After image segmentation, the second image yields a first foreground image and a first background image. After image segmentation, the contour information of the second image can be extracted. The processing of the third image is the same as that of the second image.

[0066] For example, if the second image contains prescription text and other data from the prescription, the prescription text is the target object. It's necessary to segment the prescription text and other data, meaning the first foreground image needs to contain the prescription text, and the first background image needs to contain the other data. This can be achieved using a grayscale histogram of the second image. A threshold for segmentation based on the grayscale histogram is calculated, and the foreground and background are separated using this threshold. The foreground and background images are then represented by 0 and 1 respectively in the resulting image matrix. This yields the foreground and background images.

[0067] In some embodiments, the step of segmenting the second image to obtain a first foreground image and a first background image of the second image, and segmenting the third image to obtain a second foreground image and a second background image of the third image, includes: performing grayscale processing on the second image to obtain a first grayscale image corresponding to the second image; performing binarization processing on the first grayscale image to obtain a first segmented image and determining the first foreground image and the first background image in the first segmented image according to the threshold information of the binarization processing; performing grayscale processing on the third image to obtain a second grayscale image corresponding to the third image; performing binarization processing on the second grayscale image to obtain a second segmented image and determining the second foreground image and the second background image in the second segmented image according to the threshold information of the binarization processing.

[0068] For example, the second image is processed into grayscale to obtain a corresponding first grayscale image. A grayscale image is an image without color. For instance, the original color image is composed of RGB colors (meaning each pixel contains three values). Now, by processing the image into grayscale, each pixel has only one value (0-255) representing the color depth. Grayscale processing can be done by averaging the pixel values ​​under the RGB colors, summing the values ​​of the three colors with different weights, or taking the value of a specific channel as the grayscale value. The processing method for processing the third image into grayscale to obtain a corresponding second grayscale image is the same as that for the second image.

[0069] For example, if the pixel values ​​corresponding to the RGB colors in a color image are (100, 150, 50), then the corresponding grayscale value after grayscale processing is (100+150+50) / 3 = 100, or (100*0.3+150*0.3+50*0.4) = 95.

[0070] For example, the first grayscale image is binarized to obtain a first segmented image, and the first foreground image and the first background image in the first segmented image are determined based on the threshold information of the binarization process. In the above grayscale processing, the image color is transformed from three dimensions to one dimension (value range 0-255). When this value can only be 0 or 1, it is image binarization. The binarized value is not limited to 0 and 1; it can also be any other two values. Commonly used image binarization methods include simple binarization, averaging, and bimodal binarization.

[0071] For example, simple binarization uses half the range of grayscale values ​​in the grayscale image as the threshold; that is, all pixels with a value greater than 127 are set to 1, and all pixels with a value less than 127 are set to 0. The averaging method addresses the issue of varying grayscale values ​​in each image by using the average of the summation of all pixels in the image as the threshold for binarization. The bimodal method obtains the threshold based on the histogram. The histogram is an important image characteristic that analyzes grayscale variations. If there is a significant contrast between the foreground and background grayscale values, the histogram will contain two peaks, representing the foreground and background. The trough between these peaks represents the relatively few target pixels near the edge. Generally, this minimum value is the optimal binarization boundary, and the value corresponding to this point can be set as the threshold, effectively separating the foreground and background of the image.

[0072] Step S4: Obtain the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image.

[0073] For example, contour segmentation error is used to represent the similarity of the contours of the second image and the third image. Image segmentation is performed on the second image to obtain a first foreground image and a first background image. Image segmentation is performed on the third image to obtain a second foreground image and a second background image. The contour error between the segmented image contours of the first foreground image and the first background image and the segmented image contours of the second foreground image and the second background image is calculated. The smaller the contour error, the better the processing result of the image processing model. The contour error can be calculated based on whether the pixel values ​​of the first foreground image and the second foreground image are the same at the same position, or based on whether the pixel values ​​of the first background image and the second background image are the same at the same position, or by checking whether the pixel values ​​of the first foreground image and the second foreground image, or the first background image and the second background image, are the same at the same position.

[0074] In some embodiments, obtaining the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image includes: calculating the pixel value difference of pixels at the same position in the first foreground image and the second foreground image to obtain the foreground image error; calculating the pixel value difference of pixels at the same position in the first background image and the second background image to obtain the background image error; and obtaining the contour segmentation error between the second image and the third image based on the foreground image error and the background image error.

[0075] For example, the first foreground image and the first background image are obtained by image binarization. The pixel values ​​of the first foreground image and the first background image are different, but the pixel values ​​within each of the first foreground image and the first background image are the same. The contour segmentation error between the second image and the third image is mainly due to the contour segmentation error formed by the pixel values ​​of the background image that should have been displayed and the pixel values ​​of the foreground image that should have been displayed.

[0076] For example, the pixel value inside the first foreground image is 1, the pixel value inside the first background image is 0, the pixel value of the second foreground image is set to be the same as that of the first foreground image, and the pixel value of the second background image is set to be the same as that of the second background image.

[0077] The difference in pixel values ​​at the same position in the first and second foreground images is calculated. When the pixel values ​​at the same position in the first and second foreground images are the same, the difference is 0, meaning the error at that position is 0. When the pixel values ​​at the same position in the first and second foreground images are different, the difference is 1, meaning the error at that position is 1. The accumulated errors of the first and second foreground images constitute the foreground image error.

[0078] Calculate the pixel value difference between pixels at the same position in the first and second background images. When the pixel values ​​at the same position are the same in both images, the difference is 0, meaning the error at that position is 0. When the pixel values ​​at the same position are different in both images, the difference is -1, meaning the error at that position is 1. In this case, the absolute value of the difference is taken. The accumulated errors of the first and second background images constitute the background image error.

[0079] For example, the contour segmentation error between the second image and the third image is obtained based on the foreground image error and the background image error. The contour segmentation error is then obtained by summing the foreground image error and the background image error and dividing by the number of image pixels.

[0080] Step S5: Calculate the pixel difference between the pixels at the same position in the second image and the third image to obtain the pixel error between the second image and the third image.

[0081] For example, pixel error is used to represent the similarity in detail between the second and third images. The pixel error between the second and third images is obtained by calculating the pixel difference between pixels at the same location in the second and third images, squared the pixel differences, summed them, and then averaged.

[0082] For example, to calculate that the second and third images are color images, i.e., three channels, we can first calculate the pixel difference based on the pixel values ​​in each channel, calculate the square of the pixel difference, then sum the squares of the pixel differences, and then calculate the average of the sum of the squares of the pixel differences to obtain the pixel error of each channel. Finally, we can average the pixel errors of the three channels to obtain the pixel error between the second and third images.

[0083] Alternatively, after converting the color image to a grayscale image, the pixel difference is calculated using the pixel values ​​at the same position in the grayscale image. The square of the pixel difference is then calculated, and the sum of the squares of the pixel differences is calculated. Finally, the average of the sum of the squares of the pixel differences is calculated to obtain the pixel error between the second and third images.

[0084] Step S6: Construct a loss function based on the contour segmentation error and the pixel error.

[0085] For example, contour segmentation error and pixel error can be used as loss functions for model training. These loss functions evaluate the quality of the current model parameters, allowing for continuous adjustment and updating of the model's parameters. The smaller the sum of the contour segmentation error and pixel error, the closer the image processing result of the image processing model is to the target effect. Different weights can be designed for the contour segmentation error and pixel error to make the optimized model parameters more aligned with actual needs. The specific weights for the segmentation parameters can be set according to actual requirements.

[0086] For example, the loss function can be composed of 0.6 * contour segmentation error and 0.4 * pixel error, or it can be composed of 0.7 * contour segmentation error and 0.3 * pixel error, or it can be composed of the sum of contour segmentation error and pixel error.

[0087] Step S7: Iteratively update the initial image processing model based on the target training data and the loss function to obtain the target image processing model.

[0088] For example, the parameters of the initial image processing model are continuously updated based on the target training data. Iterative updates stop when the model has completed a preset number of training iterations or when the loss function meets a set threshold. The obtained image processing model is then evaluated. If the evaluation result meets the requirements, the resulting image processing model is the target processing model. If the evaluation result does not meet the requirements, the reasons need to be analyzed, and incremental training or retraining should be performed based on the reasons, such as increasing the target training data, increasing the training data, or lowering the threshold of the loss function.

[0089] For example, if the model evaluation results indicate that the model lacks generalization ability, then it is necessary to increase the target training data or increase the number of training epochs; if the model evaluation results indicate that the model is overfitting, then it is necessary to reduce the number of training epochs.

[0090] Once the target image processing model is trained, it can provide excellent data support for prescription recognition and medical image segmentation in the field of digital healthcare. The images processed by the target image processing model can effectively suppress noise in prescription or medical image while preserving the detailed features of the images. The target image processing model can also support tasks such as medical image classification in the medical field, effectively suppressing noise in medical images while preserving the detailed features of the images, thereby improving the accuracy of medical image classification.

[0091] Please see Figure 4 , Figure 4An image processing model training device 200 is provided in this application embodiment. The image processing model training device 200 includes a data acquisition module 201, a data recognition module 202, a data segmentation module 203, a first data processing module 204, a second data processing module 205, a data analysis module 206, and a data update module 207. The data acquisition module 201 is used to acquire target training data, wherein the target training data includes at least a first image and a second image, the second image being an image obtained by filtering the first image. The data recognition module 202 is used to perform image filtering on the first image using an initial image processing model to obtain a third image. The data segmentation module 203 is used to segment the second image to obtain a first foreground image and a first background image of the second image, and to segment the third image to obtain a second foreground image and a second background image of the third image. The first data processing module 204 is used to obtain the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image. The second data processing module 205 is used to calculate the pixel difference between pixels at the same position in the second image and the third image, thereby obtaining the pixel error between the second image and the third image. The data analysis module 206 is used to construct a loss function based on the contour segmentation error and the pixel error. The data update module 207 is used to iteratively update the initial image processing model based on the target training data and the loss function, thereby obtaining the target image processing model.

[0092] In some implementations, the data acquisition module 201 performs the following during the process of acquiring the target training data:

[0093] Obtain initial training data, wherein the initial training data includes at least initial training images and first feature information, wherein the first feature information is feature information of the target object in the initial training images;

[0094] The initial training images are preprocessed to obtain the initial processed images of the initial training images;

[0095] The target training data is determined based on the first feature information, the initial processed image, and the image recognition model.

[0096] In some implementations, during the process of preprocessing the initial training images to obtain the initial processed images of the initial training images, the data acquisition module 201 performs the following:

[0097] Obtain basic information about the initial training images in the initial training data, wherein the basic information includes at least the number of channels;

[0098] The initial training image is filtered based on the initial training image and the number of channels to obtain the initial processed image corresponding to the initial training image.

[0099] In some implementations, the image recognition model includes a feature extraction network and an image classification network. During the process of determining the target training data based on the first feature information, the initial processed image, and the image recognition model, the data acquisition module 201 performs the following:

[0100] The feature extraction network is used to extract features from the initial processed image to obtain the feature extraction result of the initial processed image;

[0101] The image classification network is used to classify the feature extraction results to obtain the second feature information of the initial processed image;

[0102] The target training data is determined by matching the first feature information and the second feature information.

[0103] In some implementations, the image processing model includes a first convolutional network, a second convolutional network, and a third convolutional network. During the process of obtaining the third image by processing the first image using the initial image processing model, the data recognition module 202 performs the following:

[0104] The first convolutional network is used to extract local feature information from the first image;

[0105] The second convolutional network is used to extract global feature information from the first image based on the local feature information;

[0106] The third image is obtained by processing the global feature information using the third convolutional network.

[0107] In some implementations, the data segmentation module 203 performs the following steps during the process of segmenting the second image to obtain a first foreground image and a first background image of the second image, and segmenting the third image to obtain a second foreground image and a second background image of the third image:

[0108] The second image is processed into grayscale to obtain the first grayscale image corresponding to the second image;

[0109] The first grayscale image is binarized to obtain a first segmented image, and the first foreground image and the first background image in the first segmented image are determined according to the threshold information of the binarization process.

[0110] The third image is processed into grayscale to obtain a second grayscale image corresponding to the third image;

[0111] The second grayscale image is binarized to obtain a second segmented image, and the second foreground image and the second background image in the second segmented image are determined according to the threshold information of the binarization process.

[0112] In some implementations, the first data processing module 204, during the process of obtaining the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image, performs the following:

[0113] Calculate the pixel value difference of pixels at the same position in the first foreground image and the second foreground image to obtain the foreground image error;

[0114] Calculate the difference in pixel values ​​of pixels at the same position in the first background image and the second background image to obtain the background image error;

[0115] The contour segmentation error between the second image and the third image is obtained based on the foreground image error and the background image error.

[0116] In some implementations, the image processing model training device 200 is applied to a terminal device.

[0117] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the aforementioned image processing model training method embodiment, and will not be repeated here.

[0118] Please see Figure 5 , Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of this application.

[0119] like Figure 5 As shown, the terminal device 300 includes a processor 301 and a memory 302, which are connected by a bus 303, such as an I2C (Inter-integrated Circuit) bus.

[0120] Specifically, processor 301 provides computing and control capabilities to support the operation of the entire server. Processor 301 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0121] Specifically, the memory 302 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0122] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the embodiments of this application, and does not constitute a limitation on the terminal device to which the embodiments of this application are applied. A specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0123] The processor 301 is used to run a computer program stored in the memory, and to implement the training method of the image processing model provided in any embodiment of this application when executing the computer program.

[0124] In some implementations, the processor 301 is used to run a computer program stored in memory, applied to a terminal device, and performs the following steps when executing the computer program:

[0125] Obtain target training data, wherein the target training data includes at least a first image and a second image, and the second image is an image obtained by filtering the first image;

[0126] The first image is filtered using the initial image processing model to obtain the third image.

[0127] The second image is segmented to obtain a first foreground image and a first background image of the second image, and the third image is segmented to obtain a second foreground image and a second background image of the third image;

[0128] The contour segmentation error between the second image and the third image is obtained based on the first foreground image, the first background image, the second foreground image, and the second background image;

[0129] Calculate the pixel difference between the pixels at the same position in the second image and the third image to obtain the pixel error between the second image and the third image;

[0130] A loss function is constructed based on the contour segmentation error and the pixel error;

[0131] The initial image processing model is iteratively updated based on the target training data and the loss function to obtain the target image processing model.

[0132] In some implementations, during the process of obtaining the target training data, the processor 301 executes:

[0133] Obtain initial training data, wherein the initial training data includes at least initial training images and first feature information, wherein the first feature information is feature information of the target object in the initial training images;

[0134] The initial training images are preprocessed to obtain the initial processed images of the initial training images;

[0135] The target training data is determined based on the first feature information, the initial processed image, and the image recognition model.

[0136] In some embodiments, during the process of preprocessing the initial training images to obtain the initial processed images of the initial training images, the processor 301 performs the following:

[0137] Obtain basic information about the initial training images in the initial training data, wherein the basic information includes at least the number of channels;

[0138] The initial training image is filtered based on the initial training image and the number of channels to obtain the initial processed image corresponding to the initial training image.

[0139] In some implementations, the image recognition model includes a feature extraction network and an image classification network. During the process of determining the target training data based on the first feature information, the initial processed image, and the image recognition model, the processor 301 executes:

[0140] The feature extraction network is used to extract features from the initial processed image to obtain the feature extraction result of the initial processed image;

[0141] The image classification network is used to classify the feature extraction results to obtain the second feature information of the initial processed image;

[0142] The target training data is determined by matching the first feature information and the second feature information.

[0143] In some implementations, the image processing model includes a first convolutional network, a second convolutional network, and a third convolutional network. During the process of obtaining a third image by performing image filtering on the first image using the initial image processing model, the processor 301 executes:

[0144] The first convolutional network is used to extract local feature information from the first image;

[0145] The second convolutional network is used to extract global feature information from the first image based on the local feature information;

[0146] The third image is obtained by processing the global feature information using the third convolutional network.

[0147] In some embodiments, during the process of segmenting the second image to obtain a first foreground image and a first background image of the second image, and segmenting the third image to obtain a second foreground image and a second background image of the third image, the processor 301 performs the following:

[0148] The second image is processed into grayscale to obtain the first grayscale image corresponding to the second image;

[0149] The first grayscale image is binarized to obtain a first segmented image, and the first foreground image and the first background image in the first segmented image are determined according to the threshold information of the binarization process.

[0150] The third image is processed into grayscale to obtain a second grayscale image corresponding to the third image;

[0151] The second grayscale image is binarized to obtain a second segmented image, and the second foreground image and the second background image in the second segmented image are determined according to the threshold information of the binarization process.

[0152] In some embodiments, during the process of obtaining the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image, the processor 301 performs the following:

[0153] Calculate the pixel value difference of pixels at the same position in the first foreground image and the second foreground image to obtain the foreground image error;

[0154] Calculate the difference in pixel values ​​of pixels at the same position in the first background image and the second background image to obtain the background image error;

[0155] The contour segmentation error between the second image and the third image is obtained based on the foreground image error and the background image error.

[0156] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the terminal device described above can be referred to the corresponding process in the aforementioned image processing model training method embodiment, and will not be repeated here.

[0157] This application also provides a storage medium for computer-readable storage, which stores one or more programs that can be executed by one or more processors to implement the steps of a training method for any image processing model provided in the embodiments of this application.

[0158] The storage medium can be an internal storage unit of the terminal device described in the aforementioned embodiments, such as the terminal device's memory. Alternatively, the storage medium can be an external storage device of the terminal device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card.

[0159] It will be understood by those skilled in the art that all or some of the steps in the methods disclosed above, and the functional modules / units in the apparatus, can be implemented as software, firmware, hardware, and suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0160] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0161] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for an image processing model, characterized in that, The training method includes: Obtain target training data, wherein the target training data includes at least a first image and a second image, and the second image is an image obtained by filtering the first image; The first image is filtered using the initial image processing model to obtain the third image. The second image is segmented to obtain a first foreground image and a first background image of the second image, and the third image is segmented to obtain a second foreground image and a second background image of the third image; The contour segmentation error between the second image and the third image is obtained based on the first foreground image, the first background image, the second foreground image, and the second background image. The contour segmentation error is used to represent the similarity of the contours of the second image and the third image. The smaller the contour segmentation error, the greater the similarity of the contours of the second image and the third image. Calculate the pixel difference between the pixels at the same position in the second image and the third image to obtain the pixel error between the second image and the third image; A loss function is constructed based on the contour segmentation error and the pixel error; The initial image processing model is iteratively updated based on the target training data and the loss function to obtain the target image processing model.

2. The method according to claim 1, characterized in that, The obtained target training data includes: Obtain initial training data, wherein the initial training data includes at least initial training images and first feature information, wherein the first feature information is feature information of the target object in the initial training images; The initial training images are preprocessed to obtain the initial processed images of the initial training images; The target training data is determined based on the first feature information, the initial processed image, and the image recognition model.

3. The method according to claim 2, characterized in that, The step of preprocessing the initial training images to obtain the initial processed images of the initial training images includes: Obtain basic information about the initial training images in the initial training data, wherein the basic information includes at least the number of channels; The initial training image is filtered based on the initial training image and the number of channels to obtain the initial processed image corresponding to the initial training image.

4. The method according to claim 2, characterized in that, The image recognition model includes a feature extraction network and an image classification network. The step of determining the target training data based on the first feature information, the initially processed image, and the image recognition model includes: The feature extraction network is used to extract features from the initial processed image to obtain the feature extraction result of the initial processed image; The image classification network is used to classify the feature extraction results to obtain the second feature information of the initial processed image; The target training data is determined by matching the first feature information and the second feature information.

5. The method according to claim 1, characterized in that, The image processing model includes a first convolutional network, a second convolutional network, and a third convolutional network. The step of using the initial image processing model to perform image filtering on the first image to obtain the third image includes: The first convolutional network is used to extract local feature information from the first image; The second convolutional network is used to extract global feature information from the first image based on the local feature information; The third image is obtained by processing the global feature information using the third convolutional network.

6. The method according to claim 1, characterized in that, The step of performing image segmentation on the second image to obtain a first foreground image and a first background image of the second image, and performing image segmentation on the third image to obtain a second foreground image and a second background image of the third image, includes: The second image is processed into grayscale to obtain the first grayscale image corresponding to the second image; The first grayscale image is binarized to obtain a first segmented image, and the first foreground image and the first background image in the first segmented image are determined according to the threshold information of the binarization process. The third image is processed into grayscale to obtain a second grayscale image corresponding to the third image; The second grayscale image is binarized to obtain a second segmented image, and the second foreground image and the second background image in the second segmented image are determined according to the threshold information of the binarization process.

7. The method according to claim 1, characterized in that, The step of obtaining the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image includes: Calculate the pixel value difference of pixels at the same position in the first foreground image and the second foreground image to obtain the foreground image error; Calculate the difference in pixel values ​​of pixels at the same position in the first background image and the second background image to obtain the background image error; The contour segmentation error between the second image and the third image is obtained based on the foreground image error and the background image error.

8. An image processing model training device, characterized in that, include: A data acquisition module is used to acquire target training data, wherein the target training data includes at least a first image and a second image, and the second image is an image obtained by filtering the first image; The data recognition module is used to perform image filtering on the first image using the initial image processing model to obtain the third image; The data segmentation module is used to perform image segmentation on the second image to obtain a first foreground image and a first background image of the second image, and to perform image segmentation on the third image to obtain a second foreground image and a second background image of the third image; The first data processing module is used to obtain the contour segmentation error between the second image and the third image based on the first foreground image, the first background image, the second foreground image, and the second background image. The contour segmentation error is used to represent the similarity of the contours of the second image and the third image, and the smaller the contour segmentation error, the greater the similarity of the contours of the second image and the third image. The second data processing module is used to calculate the pixel difference between pixels at the same position in the second image and the third image, so as to obtain the pixel error between the second image and the third image. The data analysis module is used to construct a loss function based on the contour segmentation error and the pixel error; The data update module is used to iteratively update the initial image processing model based on the target training data and the loss function to obtain the target image processing model.

9. A terminal device, characterized in that, The terminal includes a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program and, in executing the computer program, implement the training method of the image processing model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the computer-readable storage medium is executed by one or more processors, the one or more processors perform the steps of the training method for the image processing model as described in any one of claims 1 to 7.

Images