Image data processing method, device and computer readable storage medium

Abstract

The specification provides an image data processing method, device and computer readable storage medium. Based on the method, an image block generated by a front-end network module of a target model when processing a target image can be acquired first; then a quality evaluation result of the image block is determined; and according to the quality evaluation result of the image block and a first processing result, invalidation processing is performed on the image block with poor quality in a targeted manner to obtain a second processing result; wherein the first processing result is result data output by the front-end network module of the target model after processing the target image; and a corresponding target data processing is performed by a back-end network module of the target model according to the second processing result. Thus, the error interference of image information of an invalid region in the target image on the target model can be effectively reduced, and then the target model can accurately and efficiently perform specific target data processing according to the input target image.

Description

Technical Field

[0001] This specification belongs to the field of image processing technology, and in particular relates to image data processing methods, apparatus and computer-readable storage media. Background Technology

[0002] With the development of artificial intelligence technology, models are being trained and used in more and more application scenarios to perform image data processing such as image recognition and image segmentation.

[0003] However, existing methods are often easily affected and interfered with by invalid regions (e.g., noisy regions, solid color background regions, etc.) in the image during training and image processing, resulting in poor accuracy of the trained model or large errors in the processing results obtained by using the model to process images.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This specification provides an image data processing method, apparatus, and computer-readable storage medium that can effectively reduce the error interference of image information from invalid regions in an image on the target model, thereby enabling the target model to accurately and efficiently perform specific target data processing based on the input target image.

[0006] This specification provides an image data processing method, comprising: acquiring image patches generated by the front-end network module of a target model when processing a target image; wherein the target model includes at least a front-end network module and a back-end network module; determining the quality evaluation result of the image patches; obtaining a second processing result based on the quality evaluation result of the image patches and a first processing result; wherein the first processing result is the result data output by the front-end network module of the target model after processing the target image; and performing corresponding target data processing using the back-end network module of the target model based on the second processing result.

[0007] This specification also provides an image data processing apparatus, comprising: an acquisition module for acquiring image patches generated by a front-end network module of a target model when processing a target image; wherein the target model includes at least a front-end network module and a back-end network module; a determination module for determining a quality evaluation result of the image patches; a first processing module for obtaining a second processing result based on the quality evaluation result of the image patches and the first processing result; wherein the first processing result is the result data output by the front-end network module of the target model after processing the target image; and a second processing module for performing corresponding target data processing using the back-end network module of the target model based on the second processing result.

[0008] This specification also provides a computer device including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement relevant steps of an image data processing method.

[0009] This specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, perform the following steps: obtaining image patches generated by the front-end network module of the target model when processing a target image; wherein the target model includes at least a front-end network module and a back-end network module; determining the quality evaluation result of the image patches; obtaining a second processing result based on the quality evaluation result of the image patches and a first processing result; wherein the first processing result is the result data output by the front-end network module of the target model after processing the target image; and performing corresponding target data processing using the back-end network module of the target model based on the second processing result.

[0010] Based on the image data processing method, apparatus, and computer-readable storage medium provided in this specification, image patches generated by the front-end network module of the target model when processing the target image can be obtained first; then, the quality evaluation result of the image patches can be determined; and based on the quality evaluation result and the first processing result, image patches with poor quality can be selectively invalidated to obtain a second processing result; wherein, the first processing result is the output data of the front-end network module of the target model after processing the target image; and the back-end network module of the target model performs corresponding target data processing based on the second processing result. This effectively reduces the error interference of invalid regions in the target image on the target model, and allows the target model to accurately and efficiently perform specific target data processing based on the input target image. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of an image data processing method provided in one embodiment of this specification;

[0013] Figure 2 This is a schematic diagram of the structural composition of a target model with a quality evaluation module constructed using the image data processing method provided in the embodiments of this specification in a scenario example;

[0014] Figure 3This is a schematic diagram of an embodiment in which the target model with a quality assessment module provided in the embodiments of this specification is applied to process a target image in a scenario example;

[0015] Figure 4 This is a schematic diagram of an embodiment in which the target model without a quality assessment module provided in this specification is applied to process a target image in a scenario example.

[0016] Figure 5 This is a schematic diagram of an embodiment of processing a target image using a target model that includes at least a convolutional neural network, as provided in the embodiments of this specification, in a scenario example.

[0017] Figure 6 This is a schematic diagram of an embodiment of processing a target image using a target model that includes at least an attention mechanism neural network, as provided in the embodiments of this specification, in a scenario example.

[0018] Figure 7 This is a flowchart illustrating the process of determining the quality evaluation result of an image patch using the image data processing method provided in the embodiments of this specification in a scenario example.

[0019] Figure 8 This is a schematic diagram illustrating the process of obtaining a preset image quality evaluation model through separate training using the image data processing method provided in the embodiments of this specification in a scenario example.

[0020] Figure 9 This is a schematic diagram illustrating the process of obtaining a preset image quality evaluation model through collaborative training using the image data processing method provided in the embodiments of this specification in a scenario example.

[0021] Figure 10 This is a schematic diagram illustrating an embodiment of training a target model using the image data processing method provided in the embodiments of this specification in a scenario example.

[0022] Figure 11 This is a schematic diagram illustrating an embodiment of applying the image data processing method provided in this specification to a target model in a scenario example;

[0023] Figure 12 This is a schematic diagram of an embodiment in which the image data processing method provided in the embodiments of this specification is applied to perform image segmentation on a high-quality image frame in a specific scenario example;

[0024] Figure 13 This is a schematic diagram of an embodiment of applying the image data processing method provided in the embodiments of this specification to perform image segmentation on a poor-quality image frame in a specific scenario example;

[0025] Figure 14This is a schematic diagram of an embodiment of detecting and recognizing feature points of surgical instruments in an image frame using the image data processing method provided in the embodiments of this specification in another specific scenario example;

[0026] Figure 15 This is a schematic diagram illustrating an embodiment of processing CT images using the image data processing method provided in this specification, in a specific scenario example.

[0027] Figure 16 This is a schematic diagram of the structural composition of a computer device provided in one embodiment of this specification;

[0028] Figure 17 This is a schematic diagram of the structural composition of an image data processing apparatus provided in one embodiment of this specification. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0030] See Figure 1 As shown in the embodiments of this specification, an image data processing method is provided. In specific implementation, this method may include the following:

[0031] S101: Obtain image patches generated by the front-end network module of the target model when processing the target image; wherein, the target model includes at least a front-end network module and a back-end network module;

[0032] S102: Determine the quality assessment results of the image patch;

[0033] S103: Based on the quality evaluation results of the image patch and the first processing result, a second processing result is obtained; wherein, the first processing result is the result data output by the front-end network module of the target model after processing the target image;

[0034] S104: The back-end network module of the target model performs corresponding target data processing based on the second processing result.

[0035] In some embodiments, the target image may be an image containing different content depending on the application scenario. For example, in a medical scenario, the target image may be a medical image (e.g., an image containing organs and / or surgical instruments); in a traffic monitoring scenario, the target image may be a traffic monitoring image (e.g., a screenshot of a highway monitoring system); in a weather forecasting scenario, the target image may be a weather image, and so on.

[0036] The target image mentioned above can be an image captured by a camera or sensor, or it can be a frame extracted from video data.

[0037] The aforementioned target model can be understood as a neural network model used for image data processing such as image recognition, image segmentation, and scene classification.

[0038] Specifically, for example, the target model described above could be a model for identifying tissues and organs in an image. Alternatively, it could be a model for image segmentation, separating the background region containing the tissues and organs from the foreground region containing the surgical instruments. It should be noted that the target models listed above are merely illustrative. In actual implementation, depending on the specific application scenario and processing requirements, the target model may include other types of models.

[0039] In some embodiments, the target model includes at least a front-end network module and a back-end network module. Specifically, the front-end network module selects image patches (e.g., patches) from the input image, processes these image patches, and obtains and outputs a first processing result. The first processing result is then further processed to obtain a corresponding second processing result. The back-end network module performs further processing based on the input second processing result to obtain and output the final processed target image. The front-end and back-end network modules can be collectively referred to as the main neural network of the target model.

[0040] It should be noted that the processing results of the target image can be different types of results depending on the type of target model. For example, when the target model is an image recognition model, the processing result of the target image can be the recognition result of data objects (e.g., surgical instruments, tissues, organs, plants, animals, etc.) in the target image. As another example, when the target model is an image segmentation model, the processing result of the target image can be the segmentation result of the foreground and background regions in the target image, and so on.

[0041] In some embodiments, see Figure 2As shown, the target model may further include a quality evaluation module; wherein the quality evaluation module is connected to the front-end network module and the back-end network module respectively; the quality evaluation module is used to determine the quality evaluation result of the image patch. Specifically, the quality evaluation module may be configured with algorithm rules for determining the quality evaluation result of the image patch, and / or a preset image quality evaluation model.

[0042] For details, please refer to Figure 3 As shown, after the front-end network module of the target model processes the target image to generate image patches, it can copy the image patches to obtain two sets of image patches. One set of image patches can be input into the quality evaluation module through the connection between the front-end network module and the quality evaluation module. The front-end network module continues to process the remaining set of image patches, obtaining and outputting the corresponding first processing result. Simultaneously, the quality evaluation module determines and outputs the quality evaluation result (e.g., the quality score of the image patch) based on the input image patches. The target model combines the quality evaluation result of the image patches and the first processing result, and obtains a second processing result by selectively reducing the influence of image information from image patches with poor quality (corresponding to invalid regions in the target image); this second processing result is then input into the back-end network module. Furthermore, the back-end network module can, based on the input second processing result, further process the image data to obtain and output a processing result of the target image less affected by the image patches with poor quality. Alternatively, the second processing result and the quality evaluation result can be input together into the back-end network module. Correspondingly, the backend network module can simultaneously process the image data more accurately based on the second processing result and the quality evaluation result, taking into account the influence of poor-quality image patches. This results in a target image with relatively smaller errors and higher accuracy.

[0043] Specifically, the aforementioned image blocks with poor image quality can be understood as image blocks corresponding to invalid regions in the target image. These invalid regions can include at least one of the following: image regions with poor clarity, image regions containing image noise, image regions with solid color backgrounds, etc.

[0044] In some embodiments, see Figure 4As shown, the target model described above may also exclude the quality evaluation module. Before implementation, the inputs and outputs of the front-end and back-end network modules in the target model can be adjusted accordingly. This allows the front-end network module to copy image blocks generated from the target image after processing them, and transmit a set of these image blocks outside the target model. The computer can then determine the quality evaluation result of the image blocks outside the target model based on the image blocks and return the determined quality evaluation result to the target model. The target model can then obtain a second processing result based on the image block quality evaluation result and the first processing result, and input this second processing result to the back-end network module. The back-end network module then further processes the input second processing result to obtain and output a processed result of the target image less affected by the poor-quality image blocks.

[0045] In some embodiments, obtaining the second processing result based on the image patch quality evaluation result and the first processing result may specifically include: identifying image patches whose quality does not meet the requirements (e.g., image patches with poor quality whose evaluation result is less than a preset scoring threshold) based on the image patch quality evaluation result; further distinguishing between image patches whose quality does not meet the requirements and image patches whose quality meets the requirements; and, based on the first processing result, performing invalidation processing on the image patches whose quality does not meet the requirements, so as to weaken or even eliminate the error interference caused by invalid regions in the target image before generating the processing result of the target image, thereby obtaining a second processing result with higher accuracy and smaller error. Subsequently, the processing result of the target image can be generated more accurately based on the above-mentioned second processing result for corresponding target data processing.

[0046] In some embodiments, the target model may specifically include a model of a neural network that includes a convolutional neural network and / or an attention mechanism. Specifically, the model that includes at least an attention mechanism may be a ViT neural network model.

[0047] Specifically, the aforementioned Convolutional Neural Networks (CNNs) can refer to a type of feedforward neural network that includes convolutional computation and has a deep structure; it is an algorithm based on deep learning.

[0048] The aforementioned attention mechanism in neural networks can specifically refer to the introduction of the attention mechanism from computer vision into neural networks, allowing the system to focus its attention on areas of interest. Typically, neural networks with an attention mechanism can learn the attention mechanism autonomously.

[0049] The ViT (Vision Transformer) neural network mentioned above can be understood as a type of neural network with an attention mechanism, applying the Transformer structure to image classification in the field of computer vision.

[0050] Specifically, the target model mentioned above can be a model composed solely of a convolutional neural network, a model composed solely of a neural network with an attention mechanism, or a model composed of a hybrid of convolutional neural networks and neural networks with attention mechanisms.

[0051] In some embodiments, when the target model includes a model that at least contains a convolutional neural network, the first processing result includes a first convolution value.

[0052] Accordingly, see Figure 5 As shown, the front-end network module may include one or more convolutional layers. Each convolutional layer also has a convolutional kernel. The front-end network module selects corresponding image blocks from the input target image through the convolutional layers, performs convolution operations on the image blocks, obtains and outputs initial convolutional values, which are used as the first convolutional values.

[0053] The backend network module may include one or more of the following network layers: convolutional layer, pooling layer, fully connected layer, multi-head attention layer, etc.

[0054] Furthermore, the backend network module can also embed a quality evaluation module. Accordingly, when processing the second processing result using the backend network model of the target model, the relevant image patches of interest can be evaluated again to obtain the corresponding quality evaluation results; and combined with the above-mentioned corresponding quality evaluation results, the second processing result can be used more accurately and effectively for image data processing of the target image.

[0055] In some embodiments, see Figure 5 As shown, the second processing result is obtained based on the quality evaluation result of the image patch and the first processing result. In specific implementation, it may include the following:

[0056] S1: Determine the matching weight coefficients based on the quality evaluation results of the image patches;

[0057] S2: Calculate the corresponding second convolution value based on the first convolution value and the weight coefficients;

[0058] S3: Generate a corresponding feature map based on the second convolution value, which serves as the result of the second processing.

[0059] The first convolution value mentioned above is generated by the convolution kernel in the front-end network module of the target model performing convolution operations on the image block.

[0060] In some embodiments, the determination of matching weight coefficients based on the quality evaluation results of image patches may include the following:

[0061] S1: Detect whether the quality evaluation result of the image patch is greater than or equal to the preset scoring threshold;

[0062] S2: If the quality evaluation result of the image patch is greater than or equal to the preset scoring threshold, the matching weight coefficient is set to 1.

[0063] S3: If the quality evaluation result of the image patch is less than the preset scoring threshold, the matching weight coefficient is set to 0.

[0064] The aforementioned preset scoring thresholds can be flexibly set according to specific circumstances and accuracy requirements.

[0065] Generally, the higher the quality evaluation result, the better the quality of the corresponding image patch.

[0066] In practice, image blocks with quality evaluation results below a preset scoring threshold can be identified as low-quality, non-compliant image blocks corresponding to invalid regions. By setting the matching weight coefficient to 0, the error impact caused by these low-quality image blocks can be weakened or even eliminated. At the same time, image blocks with quality evaluation results greater than or equal to the preset scoring threshold can be identified as high-quality, compliant image blocks corresponding to valid regions containing image information to be identified and processed. By setting the matching weight coefficient to 1, the image information of these high-quality image blocks can be completely preserved for subsequent image data processing.

[0067] Furthermore, in some cases, based on certain quality assessment mechanisms, a smaller quality assessment result value can indicate a better quality for the corresponding image patch. In response to this, the above-mentioned determination of a matching weight coefficient based on the image patch's quality assessment result can, in specific implementation, include: detecting whether the image patch's quality assessment result is less than or equal to a preset scoring threshold; if the image patch's quality assessment result is determined to be less than or equal to the preset scoring threshold, setting the matching weight coefficient to 1; if the image patch's quality assessment result is determined to be greater than the preset scoring threshold, setting the matching weight coefficient to 0.

[0068] See Figure 5As shown, the target model can calculate a second convolutional value, which weakens or even eliminates the error caused by poor-quality image patches, based on the first convolutional value and weight coefficients. This second convolutional value has higher reference value. Then, based on the second convolutional value, a corresponding feature map can be generated as the second processing result. This second processing result is then transmitted to the backend network module, so that the backend network module can generate a processing result with higher accuracy for the target image based on the second processing result.

[0069] In some embodiments, when the target model includes a model containing at least an attention mechanism of a neural network, the first processing result includes a query matrix, a key matrix, and a value matrix.

[0070] Accordingly, see Figure 6 As shown, the front-end network module may include one or more attention layers. The front-end network module first selects corresponding image patches from the input target image, and performs processing such as flattening, linear transformation, and embedding of positional and classification information based on the image patches to obtain and output the corresponding Query matrix, Key matrix, and Value matrix as the first processing result.

[0071] The backend network module may include one or more of the following network layers: convolutional layer, pooling layer, fully connected layer, multi-head attention layer, etc.

[0072] In some embodiments, see Figure 6 As shown, the second processing result is obtained based on the quality evaluation result of the image patch and the first processing result. In specific implementation, it may include the following:

[0073] S1: Determine the matching mask matrix based on the quality evaluation results of the image patch;

[0074] S2: The encoder processes the Query matrix, Key matrix, Value matrix, and mask matrix to obtain the second processing result.

[0075] The Query matrix, Key matrix, and Value matrix mentioned above are obtained by processing the network layers in the front-end network module of the target model through flattening, linear transformation, embedding of positional and classification information based on image patches.

[0076] The encoder mentioned above can specifically be a Transformer encoder.

[0077] In some embodiments, the determination of matching weight coefficients based on the quality evaluation results of image patches may include the following:

[0078] S1: Detect whether the quality evaluation result of the image patch is greater than or equal to the preset scoring threshold;

[0079] S2: If the quality evaluation result of the image patch is greater than or equal to the preset scoring threshold, set the values ​​of the elements in the row and column corresponding to the image patch in the mask matrix to 0.

[0080] S3: If the quality evaluation result of the image patch is less than the preset scoring threshold, set the values ​​of the elements in the row and column corresponding to the image patch in the mask matrix to negative infinity.

[0081] The aforementioned preset scoring thresholds can be flexibly set according to specific circumstances and accuracy requirements.

[0082] In specific implementation, if the quality evaluation result of an image patch is greater than or equal to a preset scoring threshold, the values ​​of the elements in the row and column (e.g., row i and column i) of the mask matrix (e.g., mask) corresponding to the image patch's number (e.g., i) can be set to 0. Conversely, if the quality evaluation result of an image patch is less than the preset scoring threshold, the values ​​of the elements in the row and column of the mask matrix corresponding to the image patch's number can be set to negative infinity. This yields a mask matrix that meets the requirements. The mask matrix obtained above can effectively weaken or even eliminate the error caused by poor-quality image patches when used to generate the second processing result; simultaneously, it can relatively completely preserve the image information of high-quality image patches for subsequent image data processing.

[0083] Furthermore, in some cases, based on certain quality assessment mechanisms, a smaller quality assessment result value can indicate a better quality for the corresponding image patch. In response to this, the above-mentioned determination of matching weight coefficients based on the image patch's quality assessment result can, in specific implementation, include: detecting whether the image patch's quality assessment result is less than or equal to a preset scoring threshold; if the image patch's quality assessment result is determined to be less than or equal to the preset scoring threshold, setting the values ​​of the elements in the mask matrix corresponding to the image patch's row and column to 0; if the image patch's quality assessment result is determined to be greater than the preset scoring threshold, setting the values ​​of the elements in the mask matrix corresponding to the image patch's row and column to negative infinity.

[0084] See Figure 6As shown, the target model can use the encoder to perform a function operation based on the Softmax function according to the Query matrix, Key matrix, Value matrix, and Mask matrix to obtain the attention relationship between image patches; then, according to the attention relationship between image patches, the corresponding second processing result can be obtained.

[0085] In practice, the following formula can be used to perform function operations based on the Query matrix, Key matrix, Value matrix, and mask matrix:

[0086] in, is the scaling factor, mask is the mask matrix, Query is the Query matrix, Key is the Key matrix, and Value is the Value matrix.

[0087] It should be noted that, through the aforementioned Softmax function operation, the product of the intermediate Query matrix and Key matrix, where the element in the i-th row and j-th column represents the association between image patch i and image patch j. Furthermore, by adding a mask matrix to the product of the Query and Key matrices, the elements with negative infinity in the mask matrix can be used to shield against the errors of lower-quality image patches, while the elements with a value of 0 in the mask matrix have no effect. This allows for the more complete preservation of image information from higher-quality image patches for subsequent image data processing.

[0088] In some embodiments, the quality evaluation result of the image block can be determined based on dimensions such as sharpness dimension and / or noise dimension.

[0089] In some embodiments, see Figure 7 As shown, the quality evaluation results for determining the image patches described above can, in specific implementation, include the following:

[0090] S1: Convert the image patch into a grayscale image, and perform a downsampling transformation on the grayscale image to obtain the transformed grayscale image;

[0091] S2: Perform Gaussian blurring on the grayscale image and the transformed grayscale image respectively to obtain the corresponding first blurred grayscale image and second blurred grayscale image.

[0092] S3: Using preset operators, process the grayscale image, the transformed grayscale image, the grayscale image after the first fuzzing process, and the grayscale image after the second fuzzing process respectively to obtain the corresponding first data value, second data value, third data value, and fourth data value;

[0093] S4: Determine the quality evaluation result of the image patch based on the first data value, the second data value, the third data value, and the fourth data value.

[0094] The aforementioned preset operator can be the Brenner operator.

[0095] For specific implementation, please refer to Figure 7 As shown, the image block can be converted to grayscale first to obtain the corresponding grayscale image; then the grayscale image can be downsampled (for example, a 2x2 average pooling operation with a step size of 2) to obtain the transformed grayscale image.

[0096] Furthermore, Gaussian blurring can be applied to the grayscale image and the transformed grayscale image respectively (e.g., performing a 3x3 Gaussian blurring operation) to obtain the corresponding first blurred grayscale image and second blurred grayscale image.

[0097] Next, the grayscale image, the converted grayscale image, the grayscale image after the first blurring, and the grayscale image after the second blurring can be input into the Brenner operator for processing to obtain the corresponding four Brenner scores as the first data value, the second data value, the third data value, and the fourth data value.

[0098] Then, a first ratio value (e.g., r) can be calculated based on the first data value and the second data value. fh Simultaneously, a second ratio value (e.g., r) is calculated based on the third and fourth data values. gj Then, perform corresponding function calculations based on the first and second proportional values ​​(for example, Perform the corresponding function operations (for example, The first and second parameter values ​​are then input into the Sigmoid function for calculation, yielding the first result. This first result is then input into the ReLU function for calculation, yielding the second result. Finally, the product of the second result and the first proportional value is used as the quality evaluation result for the image patch.

[0099] See Figure 7 As shown, image patches of higher quality typically receive higher quality scores, while image patches of lower quality receive lower quality scores.

[0100] Based on the above embodiments, the quality evaluation results of image blocks can be accurately determined according to the corresponding algorithm rules.

[0101] In some embodiments, the determination of the quality evaluation result of the image block described above may further include the following: calling a preset image quality evaluation model to process the image block in order to determine the quality evaluation result of the image block.

[0102] Specifically, the preset image quality evaluation model can be understood as a pre-trained neural network model that can determine and output the quality evaluation results of image patches based on dimensions such as sharpness and / or noise.

[0103] In practice, image blocks can be used as model inputs to a preset image quality model; the preset image quality evaluation model can be run to obtain the corresponding model outputs, which serve as the quality evaluation results for the image blocks.

[0104] In some embodiments, the aforementioned preset image quality evaluation model may be a model trained independently, or it may be a model trained in conjunction with a target model (e.g., a target model that at least contains a neural network with an attention mechanism).

[0105] In some embodiments, when the aforementioned preset image quality assessment model is a model obtained through separate training, see [reference needed]. Figure 8 As shown, before implementation, a preset image quality assessment model can be trained as follows: acquire sample image blocks; perform manual quality assessment on the sample image blocks to obtain corresponding reference assessment results; simultaneously, process the sample image blocks using the preset image quality assessment model to obtain corresponding assessment scores; calculate the loss function value based on the assessment scores and reference assessment results; and adjust and update the model parameters of the preset image quality assessment model based on the loss function value. After repeatedly adjusting and updating the model parameters as described above, a preset image quality assessment model with satisfactory accuracy can be obtained.

[0106] In some embodiments, when the aforementioned preset image quality evaluation model is a model obtained through co-training with a target model, see [reference needed]. Figure 9As shown, before implementation, a preset image quality assessment model can be trained as follows: acquire sample images; process the sample images using a target model (e.g., a ViT-based main neural network model) to obtain processing results, and acquire image patches generated by the target model during sample image processing; perform manual quality assessment on the image patches to obtain corresponding reference assessment results; process the image patch using the preset image quality assessment model to obtain corresponding assessment scores; calculate the loss function value based on the image annotations of the sample images, the reference assessment results, the processing results, and the assessment scores; then adjust and update the model parameters of the preset image quality assessment model and the target model based on the loss function value. After adjusting and updating the model parameters multiple times as described above, a preset image quality assessment model with satisfactory model accuracy and a target model can be obtained.

[0107] In some embodiments, the backend network module of the target model performs corresponding target data processing based on the second processing result. In a specific implementation, it may also include the following: the backend network module of the target model performs corresponding target data processing based on the second processing result and the quality evaluation result.

[0108] For specific examples, please refer to Figure 6 As shown, in some cases, the second processing result and the quality evaluation result can be input together into the backend network module of the target model. Accordingly, the backend network module can accurately perform specific image data processing on the target image based on the second processing result and the quality evaluation result, taking into account the error impact caused by poor-quality image patches, thereby effectively reducing processing errors in subsequent target data processing.

[0109] In some embodiments, the target model includes an image data processing model to be trained, or a pre-trained image data processing model.

[0110] In some embodiments, see Figure 10 As shown, when the target model includes an image data processing model to be trained, the back-end network module of the target model performs corresponding target data processing based on the second processing result. Specifically, this may include the following:

[0111] S1: The back-end network module of the target model determines the processing result of the target image based on the second processing result;

[0112] S2: Calculate the loss function value based on the processing results of the target image and the annotation data of the target image;

[0113] S3: Update the network parameters of the target model based on the loss function value.

[0114] Specifically, when training the target model, the aforementioned target image can be a sample image used to train the target model.

[0115] For specific implementation, please refer to Figure 10 As shown, the target image can be annotated first to obtain the labeled data. Then, the front-end network module of the target model processes the target image to obtain image patches, and the front-end network module further processes these image patches to obtain the corresponding first processing result. Simultaneously, a quality evaluation module determines the quality evaluation result of the image patches. Based on the quality evaluation result and the first processing result, invalidation processing is used to mask the influence of poor-quality image patches, resulting in a better second processing result. The back-end network module processes the second processing result and outputs the processed target image. Finally, the loss function value is calculated based on the processed target image and the labeled data.

[0116] Based on the loss function value, it is determined whether the current target model meets the accuracy requirements. If the current target model is determined to meet the accuracy requirements, it can be selected as the required target model, and model training can be terminated.

[0117] Conversely, if it is determined that the current target model does not meet the accuracy requirements, the network parameters of the target model can be updated based on the loss function value. This process is repeated multiple times until the updated target model meets the accuracy requirements.

[0118] Based on the above embodiments, during the training process, the quality evaluation results of image patches of the target image can be introduced and utilized to reduce the interference and influence of invalid regions in the target image on model training, thereby obtaining a target model with higher accuracy and better performance.

[0119] In some embodiments, see Figure 11 As shown, when the target model includes an image data processing model to be trained, the backend network module of the target model performs corresponding target data processing based on the second processing result. Specifically, this may further include: processing the target image using the target model to obtain the processing result of the target image; displaying or analyzing the processing result of the target image to the user so that the user can perform further data processing based on the processing result. For example, the user can use the processing result of the target image to detect abnormalities in organs or tissues, or assist in related surgical procedures. The user can also use the processing result of the target image to determine liability in traffic accidents. The user can also use the processing result of the target image to predict weather changes, etc.

[0120] Specifically, when applying the target model, the target image can be an image data to be processed, such as image recognition or image segmentation.

[0121] For specific implementation, please refer to Figure 11 As shown, the user can input the target image to be processed into the target model. The target model first processes the target image through a front-end network module to obtain image patches, and then provides these image patches to a quality evaluation module. The front-end network module continues to process the image patches, obtaining the corresponding first processing result; simultaneously, the quality evaluation module determines the quality evaluation result of the image patches. The target model then, based on the quality evaluation result and the first processing result, performs invalidation processing to mask the influence of poor-quality image patches, obtaining a better second processing result. Finally, the target model processes the second processing result through a back-end network module and outputs the processed target image.

[0122] Based on the above embodiments, the quality evaluation results of image patches of the target image can be introduced and utilized during the processing of the target image to reduce the interference and influence of invalid regions in the target image, and the target model can be used to process the target image more accurately, resulting in a processing result with higher accuracy and fewer errors.

[0123] As can be seen from the above, the image data processing method provided in the embodiments of this specification can first obtain image blocks generated by the front-end network module of the target model when processing the target image; then determine the quality evaluation result of the image blocks; and based on the quality evaluation result of the image blocks and the first processing result, selectively invalidate image blocks with poor quality to obtain a second processing result; wherein, the first processing result is the result data output by the front-end network module of the target model after processing the target image; and the back-end network module of the target model performs corresponding target data processing based on the second processing result. This effectively reduces the error interference of image information from invalid regions in the image on the target model, and thus allows the target model to accurately and efficiently perform specific target data processing based on the input target image.

[0124] In a specific scenario example, see Figure 12 As shown, intraoperative video of the surgical procedure can be acquired first. Multiple image frames (equivalent to target images) are extracted from the intraoperative video at preset intervals. These image frames can then be input into a target model with a quality evaluation module based on the image data processing method provided in this specification to perform image segmentation of surgical instruments in the surgical scenario. Specifically, the target model may include a front-end network module and a back-end network module.

[0125] When processing image frames, the target model can apply the image data processing methods provided in this specification, and use the quality evaluation module to distinguish between high-quality effective regions and low-quality invalid regions in the image frame, and perform differentiated processing.

[0126] Specifically, when the image frame is determined to be a high-quality effective region through the quality evaluation module, refer to... Figure 12 As shown, the target model can process the image frame normally and obtain the corresponding feature vector; then, based on the feature vector, the target image is segmented to obtain the corresponding segmentation result.

[0127] Conversely, see Figure 13 As shown, when an image frame is determined to be an invalid region with poor quality by the quality evaluation module (e.g., the image is blurred due to defocus, or the image is pure black due to an unexpected power outage, or white noise due to signal transmission errors), the target model can invalidate the image frame and obtain the corresponding feature vector as a 0 vector, thus avoiding wasting processing resources and processing time on segmenting the image frame.

[0128] See another specific scenario example. Figure 14 As shown, a target model with a quality assessment module can also be used to detect feature points of surgical instruments (or other operating instruments) in image frames extracted from intraoperative videos.

[0129] Specifically, after the image frame is input into the target model, the target model first divides it into blocks, obtaining multiple image blocks. These multiple image blocks are then input into a quality evaluation module to determine the quality evaluation result for each block. Based on the quality evaluation results, the target model invalidates the image blocks with poor quality, thus masking them and obtaining the remaining image blocks. The target model then extracts feature points from the remaining image blocks to obtain the feature points of the required surgical instruments. Subsequently, based on these feature points, the surgical instruments used during the operation can be identified or located.

[0130] For another specific scenario example, please refer to Figure 15 As shown, the image data processing method provided in this specification can be applied to perform image segmentation on CT images containing organs using a target model with a quality assessment module, resulting in image segmentation results containing only the regions of the organs of interest.

[0131] For specific implementation, please refer to Figure 15As shown, after the CT image is input into the target model, the target model first divides the CT image into blocks, obtaining multiple image blocks. These multiple image blocks are then input into the quality assessment module to determine the quality assessment result for each image block. Based on the quality assessment results, the target model invalidates the image blocks with poor quality (e.g., image blocks corresponding to solid colors in the background region) to mask these poor-quality image blocks, obtaining the remaining image blocks. The target model then performs image recognition and image segmentation on the remaining image blocks, ultimately obtaining and outputting image segmentation results containing only the organ regions of interest.

[0132] Through the above scenario examples, it is verified that the image data processing method provided in this specification can effectively reduce the error interference of image information from invalid regions in the image on the target model, and thus the target model can be used to accurately and efficiently perform specific target data processing based on the input target image.

[0133] This specification also provides a computer device, including a processor and a memory for storing processor-executable instructions. Specifically, the processor can execute the following steps according to the instructions to obtain image blocks generated by the front-end network module of a target model when processing a target image; wherein the target model includes at least a front-end network module and a back-end network module; determine the quality evaluation result of the image blocks; obtain a second processing result based on the quality evaluation result of the image blocks and a first processing result; wherein the first processing result is the result data output by the front-end network module of the target model after processing the target image; and perform corresponding target data processing using the back-end network module of the target model based on the second processing result.

[0134] To execute the above instructions more accurately, please refer to... Figure 16 As shown in the embodiments of this specification, another specific server is also provided, wherein the server includes a network communication port 1601, a processor 1602 and a memory 1603, and the above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0135] Specifically, the network communication port 1601 can be used to acquire image blocks generated by the front-end network module of the target model when processing the target image; wherein the target model includes at least a front-end network module and a back-end network module.

[0136] The processor 1602 can be specifically used to determine the quality evaluation result of the image patch; obtain a second processing result based on the quality evaluation result of the image patch and the first processing result; wherein, the first processing result is the result data output by the front-end network module of the target model after processing the target image; and perform corresponding target data processing using the back-end network module of the target model based on the second processing result.

[0137] The memory 1603 can be used to store the corresponding instruction program.

[0138] In this embodiment, the network communication port 1601 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0139] In this embodiment, the processor 1602 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0140] In this embodiment, the memory 1603 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0141] This specification also provides a computer-readable storage medium based on the above-described image data processing method. The computer-readable storage medium stores computer program instructions that, when executed, implement the following steps: obtaining image blocks generated by the front-end network module of the target model when processing a target image; wherein the target model includes at least a front-end network module and a back-end network module; determining the quality evaluation result of the image blocks; obtaining a second processing result based on the quality evaluation result of the image blocks and a first processing result; wherein the first processing result is the result data output by the front-end network module of the target model after processing the target image; and performing corresponding target data processing using the back-end network module of the target model based on the second processing result.

[0142] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0143] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer-readable storage medium can be explained in comparison with other embodiments, and will not be repeated here.

[0144] See Figure 17 As shown, at the software level, this specification also provides an image data processing apparatus, which may specifically include the following structural modules:

[0145] The acquisition module 1701 is specifically used to acquire image blocks generated by the front-end network module of the target model when processing the target image; wherein, the target model includes at least a front-end network module and a back-end network module;

[0146] The determination module 1702 can be used to determine the quality evaluation results of image patches;

[0147] The first processing module 1703 is specifically used to obtain a second processing result based on the quality evaluation result of the image patch and the first processing result; wherein, the first processing result is the result data output by the front-end network module of the target model after processing the target image;

[0148] The second processing module 1704 can be used to perform corresponding target data processing based on the second processing result using the backend network module of the target model.

[0149] In some embodiments, the target model may further include a quality evaluation module; wherein the quality evaluation module is connected to the front-end network module and the back-end network module respectively; the quality evaluation module may be used to determine the quality evaluation result of the image patch.

[0150] In some embodiments, the target model may specifically include: a model containing a neural network and / or an attention mechanism, etc.

[0151] In some embodiments, when the target model includes a model that at least contains a convolutional neural network, the first processing result may specifically include a first convolution value, etc.

[0152] In some embodiments, when the first processing module 1703 is specifically implemented, the second processing result can be obtained according to the quality evaluation result of the image block and the first processing result in the following manner: determining the matching weight coefficients according to the quality evaluation result of the image block; calculating the corresponding second convolution value according to the first convolution value and the weight coefficients; generating the corresponding feature map according to the second convolution value as the second processing result.

[0153] In some embodiments, when the first processing module 1703 is specifically implemented, the matching weight coefficient can be determined according to the quality evaluation result of the image block in the following manner: detecting whether the quality evaluation result of the image block is greater than or equal to a preset scoring threshold; if it is determined that the quality evaluation result of the image block is greater than or equal to the preset scoring threshold, the matching weight coefficient is determined to be 1; if it is determined that the quality evaluation result of the image block is less than the preset scoring threshold, the matching weight coefficient is determined to be 0.

[0154] In some embodiments, when the target model includes a model containing at least an attention mechanism neural network, the first processing result may specifically include a query matrix, a key matrix, and a value matrix, etc.

[0155] In some embodiments, when the first processing module 1703 is specifically implemented, the second processing result can be obtained according to the quality evaluation result of the image block and the first processing result in the following manner: determine the matching mask matrix according to the quality evaluation result of the image block; and obtain the second processing result by processing the Query matrix, Key matrix, Value matrix and mask matrix using the encoder.

[0156] In some embodiments, when the first processing module 1703 is specifically implemented, it can determine the matching mask matrix based on the quality evaluation result of the image block in the following manner: detect whether the quality evaluation result of the image block is greater than or equal to a preset scoring threshold; if it is determined that the quality evaluation result of the image block is greater than or equal to the preset scoring threshold, set the values ​​of the elements in the mask matrix corresponding to the image block in the corresponding row and column to 0; if it is determined that the quality evaluation result of the image block is less than the preset scoring threshold, set the values ​​of the elements in the mask matrix corresponding to the image block in the corresponding row and column to negative infinity.

[0157] In some embodiments, when the determining module 1702 is specifically implemented, the quality evaluation result of the image block can be determined in the following manner: the image block is converted into a grayscale image, and the grayscale image is downsampled to obtain a transformed grayscale image; Gaussian blurring is performed on the grayscale image and the transformed grayscale image respectively to obtain corresponding first blurred grayscale images and second blurred grayscale images; the grayscale image, the transformed grayscale image, the first blurred grayscale image, and the second blurred grayscale image are processed using preset operators respectively to obtain corresponding first data values, second data values, third data values, and fourth data values; the quality evaluation result of the image block is determined based on the first data values, second data values, third data values, and fourth data values.

[0158] In some embodiments, when the determination module 1702 is specifically implemented, the quality evaluation result of the image block can also be determined in the following way: the image block is processed by calling a preset image quality evaluation model to determine the quality evaluation result of the image block.

[0159] In some embodiments, the target model may specifically include an image data processing model to be trained, or a pre-trained image data processing model, etc.

[0160] In some embodiments, when the second processing module 1704 is specifically implemented, if the target model includes an image data processing model to be trained, the target data processing can be performed by the back-end network module of the target model according to the second processing result in the following manner: the back-end network module of the target model determines the processing result of the target image according to the second processing result; the loss function value is calculated according to the processing result of the target image and the labeled data of the target image; and the network parameters of the target model are updated according to the loss function value.

[0161] In some embodiments, when the second processing module 1704 is specifically implemented, the back-end network module of the target model can also be used to perform corresponding target data processing based on the second processing result and the quality evaluation result.

[0162] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0163] As can be seen from the above, the image data processing apparatus provided in the embodiments of this specification can first obtain image patches generated by the front-end network module of the target model when processing the target image; then determine the quality evaluation result of the image patches; and obtain a second processing result based on the quality evaluation result of the image patches and the first processing result; wherein, the first processing result is the result data output by the front-end network module of the target model after processing the target image; and the back-end network module of the target model performs corresponding target data processing based on the second processing result. This effectively reduces the error interference of image information from invalid regions in the image on the target model, and thus allows the target model to accurately and efficiently perform specific target data processing based on the input target image.

[0164] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0165] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0166] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer-readable storage media, including storage devices.

[0167] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0168] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0169] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. An image data processing method, characterized in that, include: Obtain image patches generated by the front-end network module of the target model when processing the target image; wherein, the target model includes at least a front-end network module and a back-end network module; The quality assessment results of image patches are determined based on the sharpness dimension and / or noise dimension; Based on the quality evaluation results of the image patches and the first processing result, a second processing result is obtained; including: determining image patches whose quality does not meet the requirements based on the quality evaluation results of the image patches; the image patches whose quality does not meet the requirements are image patches corresponding to invalid regions in the target image; distinguishing between image patches whose quality does not meet the requirements and image patches whose quality meets the requirements, and performing invalidation processing on the image patches whose quality does not meet the requirements in the first processing result to reduce the error interference caused by invalid regions in the target image, thereby obtaining the second processing result; wherein, the first processing result is the result data output by the front-end network module of the target model after processing the target image; the invalid region includes at least one of the following: image region with poor clarity, image region with image noise, and image region with solid color background; The target data is processed accordingly using the backend network module of the target model based on the second processing result.

2. The method according to claim 1, characterized in that, The target model also includes a quality evaluation module; wherein the quality evaluation module is connected to the front-end network module and the back-end network module respectively; the quality evaluation module is used to determine the quality evaluation result of the image patch.

3. The method according to claim 1, characterized in that, The target model includes: a model containing a neural network and / or an attention mechanism.

4. The method according to claim 3, characterized in that, When the target model includes a model that contains at least a convolutional neural network, the first processing result includes a first convolution value.

5. The method according to claim 4, characterized in that, Based on the image patch quality assessment results and the first processing result, the second processing result is obtained, including: Based on the quality evaluation results of the image patches, determine the matching weight coefficients; Calculate the corresponding second convolution value based on the first convolution value and the weight coefficients; Based on the second convolution value, a corresponding feature map is generated as the second processing result.

6. The method according to claim 5, characterized in that, Based on the quality assessment results of the image patches, matching weight coefficients are determined, including: Check whether the quality evaluation result of the detected image patch is greater than or equal to the preset scoring threshold; If the quality evaluation result of the image patch is greater than or equal to the preset scoring threshold, the matching weight coefficient is set to 1. If the quality evaluation result of an image patch is less than a preset scoring threshold, the matching weight coefficient will be set to 0.

7. The method according to claim 3, characterized in that, When the target model includes a model containing at least an attention mechanism neural network, the first processing result includes a query matrix, a key matrix, and a value matrix.

8. The method according to claim 7, characterized in that, Based on the image patch quality assessment results and the first processing result, the second processing result is obtained, including: Based on the quality evaluation results of the image patches, a matching mask matrix is ​​determined; The encoder processes the Query matrix, Key matrix, Value matrix, and mask matrix to obtain the second processing result.

9. The method according to claim 8, characterized in that, Based on the quality assessment results of the image patches, a matching mask matrix is ​​determined, including: Check whether the quality evaluation result of the detected image patch is greater than or equal to the preset scoring threshold; If the quality evaluation result of an image patch is determined to be greater than or equal to a preset scoring threshold, the values ​​of the elements in the row and column corresponding to the image patch in the mask matrix are set to 0. If the quality evaluation result of an image patch is determined to be less than a preset scoring threshold, the values ​​of the elements in the corresponding row and column of the mask matrix corresponding to that image patch are set to negative infinity.

10. The method according to claim 1, characterized in that, Determine the quality assessment results of the image patches, including: The image patch is converted into a grayscale image, and then the grayscale image is downsampled to obtain the transformed grayscale image. Gaussian blurring is applied to the grayscale image and the transformed grayscale image respectively to obtain the corresponding first blurred grayscale image and second blurred grayscale image. The grayscale image, the transformed grayscale image, the grayscale image after the first blurring, and the grayscale image after the second blurring are processed by preset operators respectively to obtain the corresponding first data value, second data value, third data value, and fourth data value. The quality evaluation result of the image patch is determined based on the first data value, the second data value, the third data value, and the fourth data value.

11. The method according to claim 1, characterized in that, Determining the quality assessment results of image patches also includes: The image block is processed by a preset image quality evaluation model to determine the quality evaluation result of the image block.

12. The method according to claim 1, characterized in that, The target model includes an image data processing model to be trained, or an image data processing model that has already been trained.

13. The method according to claim 12, characterized in that, When the target model includes an image data processing model to be trained, the back-end network module of the target model performs corresponding target data processing based on the second processing result, including: The processing result of the target image is determined by the back-end network module of the target model based on the second processing result; The loss function value is calculated based on the processing results of the target image and the annotation data of the target image; Update the network parameters of the target model based on the loss function value.

14. The method according to claim 1, characterized in that, The backend network module of the target model performs corresponding target data processing based on the second processing result, and also includes: The target data is processed accordingly using the backend network module of the target model based on the second processing result and the quality evaluation result.

15. An image data processing apparatus, characterized in that, include: An acquisition module is used to acquire image patches generated by the front-end network module of the target model when processing the target image; wherein, the target model includes at least a front-end network module and a back-end network module; The determination module is used to determine the quality assessment results of image patches based on the sharpness dimension and / or the noise dimension; The first processing module is used to obtain a second processing result based on the quality evaluation result of the image patch and the first processing result; wherein, the first processing result is the result data output by the front-end network module of the target model after processing the target image; The second processing module is used to perform corresponding target data processing based on the second processing result using the backend network module of the target model; Specifically, the first processing module is used to: determine image blocks that do not meet the quality requirements based on the quality evaluation results of the image blocks; the image blocks that do not meet the quality requirements are image blocks corresponding to invalid regions in the target image; distinguish between image blocks that do not meet the quality requirements and image blocks that meet the quality requirements, and perform invalidation processing on the image blocks that do not meet the quality requirements in the first processing result to reduce the error interference caused by invalid regions in the target image, thereby obtaining a second processing result; the invalid regions include at least one of the following: image regions with poor clarity, image regions with image noise, and image regions with solid color backgrounds.

16. A computer device, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 14.

Images