Model training method, image processing method, device, equipment and storage medium

Abstract

The present application discloses a model training method, an image processing method, an apparatus, a device and a storage medium, which belong to the field of artificial intelligence technology. The model training method includes: obtaining a first sample image and a first standard area ratio corresponding to a first object in the first sample image; based on the first standard area ratio, obtaining a standard area segmentation result corresponding to the first sample image; training a first initial segmentation model based on the first sample image and the standard area segmentation result to obtain a first target segmentation model. In this process, a standard area segmentation result is obtained based on the standard area ratio, and the accuracy of the standard area segmentation result is high. The trained first target segmentation model has a good area segmentation function, so that the first target segmentation model can be called to realize the process of automatically obtaining the measurement index corresponding to the image to be processed, which is conducive to improving the stability and efficiency of image processing and the accuracy of the obtained measurement index.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a model training method, image processing method, apparatus, device, and storage medium. Background Art

[0002] As image acquisition methods increase, the types of images increase, and the application scenarios for image processing also increase. One application scenario involves processing an image containing numerous small objects of interest to obtain a corresponding metric. This metric indicates the percentage of the area in the image corresponding to objects that meet certain conditions. For example, a tumor image containing numerous small stained cells can be processed to obtain a metric that indicates the percentage of the area corresponding to stained cells outside the cancer cell area.

[0003] In related technologies, the image to be processed is manually compared with a pre-set template image, and the pre-set metric corresponding to the template image that best matches the image to be processed is used as the metric for the image to be processed. This manual image processing process is significantly affected by human subjectivity, resulting in poor stability and efficiency, and the resulting metric is not very accurate. Summary of the Invention

[0004] The embodiments of the present application provide a model training method, image processing method, apparatus, device, and storage medium that can be used to improve the stability and efficiency of image processing and the accuracy of obtained measurement indicators. The technical solution is as follows:

[0005] In one aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Acquire an image to be processed, where the image to be processed includes a first object;

[0007] calling a first target segmentation model to perform region segmentation on the image to be processed to obtain a first region segmentation result corresponding to the image to be processed, where the first region segmentation result is used to indicate a region in the image to be processed where the first object is located, the first target segmentation model being trained based on a first sample image and a standard region segmentation result corresponding to the first sample image, where the standard region segmentation result is obtained based on a first standard region ratio corresponding to the first object in the first sample image;

[0008] Based on the first region segmentation result, a measurement index corresponding to the image to be processed is obtained, where the measurement index is used to indicate a proportion of an area corresponding to a first object that meets a region restriction condition in the image to be processed.

[0009] In a possible implementation, obtaining a measurement indicator corresponding to the image to be processed based on the first region segmentation result includes:

[0010] Determining a target area in the image to be processed;

[0011] Using the target area to filter the area where the first object in the image to be processed is located, to obtain the area where the first object in the image to be processed is located that meets the area restriction condition;

[0012] The area where the first object meeting the area restriction condition is located is compared with the total area of ​​the image to be processed to obtain a measurement index corresponding to the image to be processed.

[0013] In a possible implementation, calling the first target segmentation model to perform region segmentation on the image to be processed to obtain a first region segmentation result corresponding to the image to be processed includes:

[0014] Performing attribute transformation on the image to be processed so that the image to be processed after the attribute transformation has target visual attributes;

[0015] The first target segmentation model is called to perform region segmentation on the image to be processed after the attribute transformation, to obtain a first region segmentation result corresponding to the image to be processed.

[0016] In a possible implementation, the image to be processed corresponds to at least one color channel, and performing attribute transformation on the image to be processed includes:

[0017] Determining the pixel mean values ​​corresponding to the respective color channels based on the pixel values ​​of the respective pixel points in the image to be processed under the respective color channels;

[0018] Performing numerical transformation on the pixel means corresponding to each color channel using the reference values ​​corresponding to each color channel to obtain pixel value transformation benchmarks corresponding to each color channel, wherein the reference values ​​corresponding to each color channel are determined based on the target visual attribute;

[0019] Based on the pixel value transformation benchmarks corresponding to the respective color channels, the pixel values ​​of the respective pixel points in the image to be processed under the respective color channels are transformed.

[0020] In a possible implementation, performing attribute transformation on the image to be processed includes:

[0021] Acquire an image transformation model, wherein the image transformation model is used to output an image having the target visual attribute;

[0022] The image transformation model is called to perform attribute transformation on the image to be processed.

[0023] A model training method is also provided, comprising:

[0024] Obtaining a first sample image and a first standard area ratio corresponding to the first object in the first sample image;

[0025] Based on the first standard area proportion, obtaining a standard area segmentation result corresponding to the first sample image, where the standard area segmentation result is used to indicate the area where the first object is located in the first sample image;

[0026] A first initial segmentation model is trained based on the first sample image and the standard region segmentation result to obtain a first target segmentation model.

[0027] In one possible implementation, the training of the first initial segmentation model based on the first sample image and the standard region segmentation result to obtain the first target segmentation model includes:

[0028] performing attribute transformation on the first sample image so that the attribute-transformed first sample image has target visual attributes;

[0029] The first initial segmentation model is trained based on the first sample image after the attribute transformation and the standard region segmentation result to obtain the first target segmentation model.

[0030] An image processing method is also provided, the method comprising:

[0031] Acquire an image to be processed, where the image to be processed includes a first object;

[0032] Performing attribute transformation on the image to be processed so that the image to be processed after the attribute transformation has target visual attributes;

[0033] Calling a first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation, to obtain a first region segmentation result corresponding to the image to be processed, where the first region segmentation result is used to indicate a region in the image to be processed where the first object is located;

[0034] Based on the first region segmentation result, a measurement index corresponding to the image to be processed is obtained, where the measurement index is used to indicate a proportion of an area corresponding to a first object that meets a region restriction condition in the image to be processed.

[0035] In one possible implementation, obtaining a measurement indicator corresponding to the image to be processed based on the first region segmentation result includes:

[0036] Determining a target area in the image to be processed;

[0037] Using the target area to filter the area where the first object in the image to be processed is located, to obtain the area where the first object in the image to be processed is located that meets the area restriction condition;

[0038] The area where the first object meeting the area restriction condition is located is compared with the total area of ​​the image to be processed to obtain a measurement index corresponding to the image to be processed.

[0039] A model training method is also provided, comprising:

[0040] Acquire a first sample image and a standard region segmentation result corresponding to the first sample image, wherein the standard region segmentation result is used to indicate a region in the first sample image where the first object is located;

[0041] performing attribute transformation on the first sample image so that the attribute-transformed first sample image has target visual attributes;

[0042] A first initial segmentation model is trained based on the first sample image after attribute transformation and the standard region segmentation result to obtain a first target segmentation model.

[0043] In a possible implementation, obtaining a standard area segmentation result corresponding to the first sample image based on the first standard area proportion includes:

[0044] calling at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result, wherein the at least one second target segmentation model is trained based on the second sample image and a standard region division image corresponding to the second sample image, and the standard region division image corresponding to the second sample image is obtained based on a second standard region ratio corresponding to the first object in the second sample image;

[0045] The candidate region segmentation result that satisfies a first matching condition with the first standard region proportion in the at least one candidate region segmentation result is used as the standard region segmentation result corresponding to the first sample image.

[0046] In one possible implementation, the at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models; and before calling the at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result, the method further includes:

[0047] Based on the second standard area ratio, obtaining a standard area division image corresponding to the second sample image;

[0048] Based on the second sample image and the standard region division image, the reference number of second initial segmentation models are trained respectively to obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models and obtained through different rounds of training.

[0049] In a possible implementation, obtaining a standard area division image corresponding to the second sample image based on the second standard area ratio includes:

[0050] Obtaining a threshold interval corresponding to a reference color channel and a sample channel image corresponding to the second sample image under the reference color channel;

[0051] Performing binarization processing on the sample channel image using the threshold interval to obtain a binarized image;

[0052] Performing morphological processing on the binary image to obtain at least one candidate region division image;

[0053] The candidate area division image that satisfies a second matching condition with the second standard area ratio in the at least one candidate area division image is used as the standard area division image corresponding to the second sample image.

[0054] In one possible implementation, obtaining a threshold range corresponding to a reference color channel includes:

[0055] Acquire a first channel image corresponding to a first reference image in the reference color channel and a second channel image corresponding to a second reference image in the reference color channel, wherein the first reference image does not include the first object, and the second reference image includes the first object;

[0056] Obtaining a first pixel value distribution of the first channel image;

[0057] Obtaining a second pixel value distribution of the second channel image;

[0058] A threshold interval corresponding to the reference color channel is determined based on the first pixel value distribution and the second pixel value distribution.

[0059] In a possible implementation, performing morphological processing on the binary image to obtain at least one candidate region segmentation image includes:

[0060] Determining, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image;

[0061] In response to the predicted area ratio being smaller than the second standard area ratio, the binary image is dilated using a reference dilation kernel to obtain the at least one candidate area division image.

[0062] In a possible implementation, performing morphological processing on the binary image to obtain at least one candidate region segmentation image includes:

[0063] Determining, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image;

[0064] In response to the predicted area ratio being greater than the second standard area ratio, the binarized image is reduced using a reference reduction kernel to obtain the at least one candidate area division image.

[0065] In another aspect, an image processing apparatus is provided, the apparatus comprising:

[0066] A first acquiring unit, configured to acquire an image to be processed, where the image to be processed includes a first object;

[0067] a segmentation unit, configured to call a first target segmentation model to perform region segmentation on the image to be processed, and obtain a first region segmentation result corresponding to the image to be processed, wherein the first region segmentation result is used to indicate a region in the image to be processed where the first object is located, wherein the first target segmentation model is trained based on a first sample image and a standard region segmentation result corresponding to the first sample image, and wherein the standard region segmentation result is obtained based on a first standard region ratio corresponding to the first object in the first sample image;

[0068] The second acquisition unit is used to acquire a measurement index corresponding to the image to be processed based on the first region segmentation result, where the measurement index is used to indicate the area ratio corresponding to the first object that meets the area restriction condition in the image to be processed.

[0069] In one possible implementation, the second acquisition unit is used to determine a target area in the image to be processed; use the target area to filter the area where the first object in the image to be processed is located to obtain the area where the first object that meets the area restriction condition is located in the image to be processed; compare the area where the first object that meets the area restriction condition is located with the total area of ​​the image to be processed to obtain a measurement indicator corresponding to the image to be processed.

[0070] In one possible implementation, the segmentation unit is used to perform attribute transformation on the image to be processed so that the image to be processed after the attribute transformation has target visual attributes; and call the first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation to obtain a first region segmentation result corresponding to the image to be processed.

[0071] In one possible implementation, the image to be processed corresponds to at least one color channel, and the segmentation unit is further used to determine the pixel mean values ​​corresponding to each color channel based on the pixel values ​​of each pixel point in the image to be processed under each color channel; use the reference values ​​corresponding to each color channel to perform numerical transformation on the pixel mean values ​​corresponding to each color channel to obtain the pixel value transformation benchmarks corresponding to each color channel, and the reference values ​​corresponding to each color channel are determined based on the target visual attributes; based on the pixel value transformation benchmarks corresponding to each color channel, the pixel values ​​of each pixel point in the image to be processed under each color channel are transformed.

[0072] In a possible implementation, the segmentation unit is further configured to obtain an image transformation model, which is configured to output an image having the target visual attributes; and to call the image transformation model to perform attribute transformation on the image to be processed.

[0073] A model training device is also provided, comprising:

[0074] A first acquiring unit, configured to acquire a first sample image and a first standard area ratio corresponding to the first object in the first sample image;

[0075] a second acquiring unit, configured to acquire a standard region segmentation result corresponding to the first sample image based on the first standard region proportion, wherein the standard region segmentation result is used to indicate a region in the first sample image where the first object is located;

[0076] A training unit is used to train a first initial segmentation model based on the first sample image and the standard region segmentation result to obtain a first target segmentation model.

[0077] In one possible implementation, the second acquisition unit is used to call at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result, wherein the at least one second target segmentation model is trained based on the second sample image and the standard region division image corresponding to the second sample image, and the standard region division image corresponding to the second sample image is obtained based on the second standard region proportion corresponding to the first object in the second sample image; and the candidate region segmentation result in the at least one candidate region segmentation result that meets the first matching condition with the first standard region proportion is used as the standard region segmentation result corresponding to the first sample image.

[0078] In one possible implementation, the at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models respectively; the apparatus further includes:

[0079] a third acquiring unit, configured to acquire a standard area division image corresponding to the second sample image based on the second standard area ratio;

[0080] The training unit is also used to train the reference number of second initial segmentation models based on the second sample image and the standard area division image, and obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models through different rounds of training.

[0081] In one possible implementation, the third acquisition unit is used to obtain a threshold interval corresponding to a reference color channel, and a sample channel image corresponding to the second sample image under the reference color channel; binarize the sample channel image using the threshold interval to obtain a binarized image; perform morphological processing on the binarized image to obtain at least one candidate area division image; and use the candidate area division image in the at least one candidate area division image that meets a second matching condition with the second standard area ratio as the standard area division image corresponding to the second sample image.

[0082] In one possible implementation, the third acquisition unit is further used to acquire a first channel image corresponding to a first reference image under the reference color channel and a second channel image corresponding to a second reference image under the reference color channel, wherein the first reference image does not include the first object, and the second reference image includes the first object; acquire a first pixel value distribution of the first channel image; acquire a second pixel value distribution of the second channel image; and determine a threshold interval corresponding to the reference color channel based on the first pixel value distribution and the second pixel value distribution.

[0083] In one possible implementation, the third acquisition unit is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being less than the second standard area ratio, the binarized image is expanded using a reference expansion kernel to obtain the at least one candidate area division image.

[0084] In one possible implementation, the third acquisition unit is further used to determine, based on the binarized image, the predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being greater than the second standard area ratio, the binarized image is reduced using a reference reduction kernel to obtain the at least one candidate area division image.

[0085] In one possible implementation, the training unit is used to perform attribute transformation on the first sample image so that the first sample image after attribute transformation has target visual attributes; and the first initial segmentation model is trained based on the first sample image after attribute transformation and the standard region segmentation result to obtain the first target segmentation model.

[0086] An image processing device is also provided, comprising:

[0087] A first acquiring unit, configured to acquire an image to be processed, where the image to be processed includes a first object;

[0088] a transforming unit, configured to perform attribute transformation on the image to be processed so that the image to be processed after attribute transformation has target visual attributes;

[0089] a segmentation unit, configured to call a first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation, to obtain a first region segmentation result corresponding to the image to be processed, wherein the first region segmentation result is used to indicate a region in the image to be processed where the first object is located;

[0090] The second acquisition unit is used to acquire a measurement index corresponding to the image to be processed based on the first region segmentation result, where the measurement index is used to indicate the area ratio corresponding to the first object that meets the area restriction condition in the image to be processed.

[0091] In one possible implementation, the image to be processed corresponds to at least one color channel, and the transformation unit is used to determine the pixel mean values ​​corresponding to each color channel based on the pixel values ​​of each pixel point in the image to be processed under each color channel; use the reference values ​​corresponding to each color channel to perform numerical transformation on the pixel mean values ​​corresponding to each color channel to obtain the pixel value transformation benchmarks corresponding to each color channel, and the reference values ​​corresponding to each color channel are determined based on the target visual attributes; based on the pixel value transformation benchmarks corresponding to each color channel, the pixel values ​​of each pixel point in the image to be processed under each color channel are transformed.

[0092] In a possible implementation, the transformation unit is configured to obtain an image transformation model, where the image transformation model is configured to output an image having the target visual attributes; and the image transformation model is called to perform attribute transformation on the image to be processed.

[0093] In one possible implementation, the second acquisition unit is used to determine a target area in the image to be processed; use the target area to filter the area where the first object in the image to be processed is located to obtain the area where the first object that meets the area restriction condition is located in the image to be processed; compare the area where the first object that meets the area restriction condition is located with the total area of ​​the image to be processed to obtain a measurement indicator corresponding to the image to be processed.

[0094] A model training device is also provided, comprising:

[0095] a first acquiring unit, configured to acquire a first sample image and a standard region segmentation result corresponding to the first sample image, wherein the standard region segmentation result is used to indicate a region in the first sample image where the first object is located;

[0096] a transforming unit, configured to perform attribute transformation on the first sample image so that the first sample image after attribute transformation has target visual attributes;

[0097] A training unit is used to train a first initial segmentation model based on the first sample image after attribute transformation and the standard region segmentation result to obtain a first target segmentation model.

[0098] In a possible implementation, the first acquisition unit is configured to acquire a first standard area ratio corresponding to the first object in the first sample image; and acquire a standard area segmentation result corresponding to the first sample image based on the first standard area ratio.

[0099] In one possible implementation, the first acquisition unit is further used to call at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result, wherein the at least one second target segmentation model is trained based on the second sample image and the standard region division image corresponding to the second sample image, and the standard region division image corresponding to the second sample image is obtained based on the second standard region proportion corresponding to the first object in the second sample image; and the candidate region segmentation result in the at least one candidate region segmentation result that meets the first matching condition with the first standard region proportion is used as the standard region segmentation result corresponding to the first sample image.

[0100] In one possible implementation, the at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models respectively; the apparatus further includes:

[0101] A second acquiring unit, configured to acquire a standard area division image corresponding to the second sample image based on the second standard area ratio;

[0102] The training unit is also used to train the reference number of second initial segmentation models based on the second sample image and the standard area division image, and obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models through different rounds of training.

[0103] In one possible implementation, the second acquisition unit is used to obtain a threshold interval corresponding to a reference color channel, and a sample channel image corresponding to the second sample image under the reference color channel; binarize the sample channel image using the threshold interval to obtain a binarized image; perform morphological processing on the binarized image to obtain at least one candidate area division image; and use the candidate area division image in the at least one candidate area division image that meets a second matching condition with the second standard area ratio as the standard area division image corresponding to the second sample image.

[0104] In one possible implementation, the second acquisition unit is further used to acquire a first channel image corresponding to a first reference image under the reference color channel and a second channel image corresponding to a second reference image under the reference color channel, wherein the first reference image does not include the first object, and the second reference image includes the first object; acquire a first pixel value distribution of the first channel image; acquire a second pixel value distribution of the second channel image; and determine a threshold interval corresponding to the reference color channel based on the first pixel value distribution and the second pixel value distribution.

[0105] In one possible implementation, the second acquisition unit is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being less than the second standard area ratio, the binarized image is expanded using a reference expansion kernel to obtain the at least one candidate area division image.

[0106] In one possible implementation, the second acquisition unit is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being greater than the second standard area ratio, the binarized image is reduced using a reference reduction kernel to obtain the at least one candidate area division image.

[0107] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above-mentioned image processing methods or any of the above-mentioned model training methods.

[0108] On the other hand, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor so that the computer implements any of the above-mentioned image processing methods or any of the above-mentioned model training methods.

[0109] In another aspect, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the aforementioned image processing methods or implement any of the aforementioned model training methods.

[0110] The technical solutions provided by the embodiments of the present application bring at least the following beneficial effects:

[0111] In an embodiment of the present application, a standard area segmentation result is obtained based on the standard area ratio, and the accuracy of the standard area segmentation result is relatively high. The first target segmentation model trained according to the standard area segmentation result has a better area segmentation function, so that the first target segmentation model can be called to realize the process of automatically obtaining the measurement indicators corresponding to the image to be processed, which is conducive to improving the stability and efficiency of image processing and the accuracy of the measurement indicators corresponding to the image to be processed.

[0112] Furthermore, calling the first target segmentation model automates the process of obtaining the corresponding metrics for the image to be processed. This image processing process requires no human intervention and offers improved stability and efficiency. Before calling the first target segmentation model for region segmentation, the image to be processed is transformed into an image with the target visual attributes. This reduces the interference of the visual attributes of the image to be processed on the first target segmentation model, resulting in a more accurate first region segmentation result obtained by calling the first target segmentation model, thereby improving the accuracy of the obtained metrics. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0114] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0115] Figure 2 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0116] Figure 3 This is a schematic diagram of obtaining an image to be processed provided by an embodiment of the present application;

[0117] Figure 4 This is a flow chart of a model training method provided in an embodiment of the present application;

[0118] Figure 5 is a schematic diagram of a process of obtaining a first sample image provided by an embodiment of the present application;

[0119] Figure 6 This is a flowchart of a process for obtaining a standard area division image corresponding to a second sample image based on a second standard area ratio, provided by an embodiment of the present application;

[0120] Figure 7 1 is a schematic diagram of a first reference image and a first channel image corresponding to the first reference image in a DAB channel provided by an embodiment of the present application;

[0121] Figure 8 is a schematic diagram of a first pixel value distribution and a second pixel value distribution provided in an embodiment of the present application;

[0122] Figure 9 is a schematic diagram of a binarized image provided in an embodiment of the present application;

[0123] Figure 10 1 is a schematic diagram of five candidate region partition images obtained after five iterations of dilation processing provided in an embodiment of the present application;

[0124] Figure 11 is a schematic diagram of a first sample image and a standard region segmentation result corresponding to the first sample image represented in image form, provided in an embodiment of the present application;

[0125] Figure 12 is a schematic diagram of a process for obtaining a standard region segmentation result corresponding to a first sample image provided by an embodiment of the present application;

[0126] Figure 13 This is a schematic diagram of a process for obtaining a measurement indicator corresponding to an image to be processed, provided in an embodiment of the present application;

[0127] Figure 14 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0128] Figure 15 This is a flow chart of a model training method provided in an embodiment of the present application;

[0129] Figure 16 This is a flow chart of a process for determining an IC value corresponding to a tumor image provided by an embodiment of the present application;

[0130] Figure 17 is a schematic diagram of a process for determining an IC value corresponding to a tumor image provided by an embodiment of the present application;

[0131] Figure 18 is a schematic diagram of test results of the predicted performance of an image processing method provided by an embodiment of the present application on two different image acquisition devices;

[0132] Figure 19 1 is a schematic diagram of a test result of performing an IC value test on a tumor image collected by a microscope, provided in an embodiment of the present application;

[0133] Figure 20 is a schematic diagram of an image processing device provided in an embodiment of the present application;

[0134] Figure 21 is a schematic diagram of a model training device provided in an embodiment of the present application;

[0135] Figure 22 is a schematic diagram of a model training device provided in an embodiment of the present application;

[0136] Figure 23 is a schematic diagram of an image processing device provided in an embodiment of the present application;

[0137] Figure 24 is a schematic diagram of a model training device provided in an embodiment of the present application;

[0138] Figure 25 is a schematic diagram of a model training device provided in an embodiment of the present application;

[0139] Figure 26 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0140] Figure 27 This is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0141] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0142] In order to facilitate understanding of the technical process of the embodiments of the present application, the terms involved in the embodiments of the present application are explained below.

[0143] PD-1 (Programmed Death 1): A receptor protein on the surface of immune cells T cells.

[0144] PD-L1 (Programmed Cell Death-Ligand 1): ligand of PD-1.

[0145] IC score (IC ratio value or IC proportion value, referred to as IC value): the percentage of immune cells occupying the tumor area.

[0146] In exemplary embodiments, the image processing method and model training method provided in the embodiments of the present application can be applied to the field of artificial intelligence technology. Next, artificial intelligence technology is introduced.

[0147] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and create new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0148] Artificial intelligence technology is a comprehensive discipline covering a wide range of fields, encompassing both hardware and software technologies. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technologies, operating / interactive systems, and mechatronics. Artificial intelligence software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning. The image processing methods and model training methods provided in the embodiments of this application involve computer vision and machine learning technologies.

[0149] Computer vision (CV) technology is the study of how machines can "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Furthermore, it performs image processing to transform the computer-generated images into images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional object reconstruction, 3D (three-dimensional) technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0150] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0151] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0152] Figure 1 The schematic diagram of the implementation environment provided by the embodiment of the present application is shown. The implementation environment may include: a terminal 11 and a server 12.

[0153] The image processing method provided in the embodiment of the present application can be executed by the terminal 11, can be executed by the server 12, or can be executed jointly by the terminal 11 and the server 12, and the embodiment of the present application does not limit this. In the case where the image processing method provided in the embodiment of the present application is jointly executed by the terminal 11 and the server 12, the server 12 performs the primary computing work and the terminal 11 performs the secondary computing work; or, the server 12 performs the secondary computing work and the terminal 11 performs the primary computing work; or, the server 12 and the terminal 11 use a distributed computing architecture to perform collaborative computing.

[0154] The model training method provided in the embodiment of the present application can be executed by the terminal 11, can be executed by the server 12, or can be executed jointly by the terminal 11 and the server 12, and the embodiment of the present application does not limit this. In the case where the model training method provided in the embodiment of the present application is jointly executed by the terminal 11 and the server 12, the server 12 undertakes the main computing work and the terminal 11 undertakes the secondary computing work; or, the server 12 undertakes the secondary computing work and the terminal 11 undertakes the main computing work; or, the server 12 and the terminal 11 adopt a distributed computing architecture to perform collaborative computing.

[0155] The image processing method and model training method provided in the embodiments of the present application can be executed by the same device or by different devices, and the embodiments of the present application are not limited to this.

[0156] In one possible implementation, the terminal 11 may be any electronic product that can interact with a user through one or more methods such as a keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device, such as a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC), a tablet computer, a smart car computer, a smart TV, a smart speaker, etc. The server 12 may be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 via a wired or wireless network.

[0157] Those skilled in the art should understand that the above-mentioned terminal 11 and server 12 are only examples. Other existing or future terminals or servers that are applicable to this application should also be included in the scope of protection of this application and are included here by reference.

[0158] Based on the above Figure 1 In the implementation environment shown, the embodiment of the present application provides an image processing method, which is executed by a computer device. The computer device can be a server 12 or a terminal 11, and the embodiment of the present application does not limit this. Figure 2 As shown, the image processing method provided in the embodiment of the present application includes the following steps 201 to 203:

[0159] In step 201, an image to be processed is acquired, where the image to be processed includes a first object.

[0160] The image to be processed refers to an image that requires processing, and the first object is an object of interest included in the image to be processed. In an exemplary embodiment, the first objects in the image to be processed are small and numerous, making it difficult to manually accurately label the areas where the first objects are located. Exemplarily, the image to be processed is a pathology image obtained by capturing diseased tissue on a pathology slide, and the first object is a stained cell in the pathology image. Exemplarily, the image to be processed is a tissue image obtained by capturing a tissue slide, and the first object is a blood vessel in the tissue image.

[0161] A pathology slide may have specific characteristics. For example, the pathology slide may be a pathology slide of a biological tissue with a certain pathology, such as a pathology slide of an animal or plant tissue with a specific pathology, or a pathology slide of a tumor tissue in a certain part of the human body. The pathology slide may be a stained slide. In the stained pathology slide, certain cells will be stained. The stained cells are called stained cells and can be visually distinguished from other unstained cells and non-cells. For example, the pathology slide is a slide stained based on the PD-L1 method. By capturing an image of the tumor area field in the pathology slide stained based on the PD-L1 method, a pathology image such as a tumor image can be obtained.

[0162] Methods for the computer device to obtain images to be processed include, but are not limited to: the computer device extracting images to be processed from an image library; an image acquisition device that has established a communication connection with the computer device transmitting captured images to be processed to the computer device; the computer device obtaining images to be processed that have been manually uploaded. For example, the images in the image library and the manually uploaded images are both images captured by the image acquisition device.

[0163] The type of image acquisition device used to acquire the image to be processed is related to the type of the image to be processed, and the embodiments of the present application do not limit this. For example, the image acquisition device that acquires the image to be processed includes but is not limited to a microscope, an imported scanner, a domestic scanner, etc. For example, the visual attributes of the images to be processed acquired by different image acquisition devices may be the same or different. The visual attributes of an image are used to indicate the visual attributes of the image, such as the brightness of the image, the color richness of the image, the focus accuracy of the image, the sharpness of the image, etc.

[0164] For example, Figure 3 As shown, after a pathology slide 301 is obtained by staining based on the PD-L1 method, a microscope 302, an imported scanner 303, and a domestic scanner 304 can be used to respectively capture images of the pathology slide 301 to obtain images to be processed.

[0165] Exemplarily, an image acquisition device is used to capture multiple fields of view on a pathology slide, thereby obtaining multiple field of view images. These multiple field of view images can constitute a WSI (Whole Slide Image) image. Exemplarily, the image to be processed is an image of a field of view of a tumor region (referred to as a tumor image), or an ROI (Region of Interest) image obtained from the WSI image. Exemplarily, the ROI image is a tumor image. Exemplarily, the ROI image obtained from the WSI image can also be referred to as a patch image.

[0166] The number of images to be processed may be one or more, and this is not limited in this embodiment of the present application. If there are multiple images to be processed, each image to be processed includes a first object. This embodiment of the present application uses one image to be processed as an example for illustration. If there are multiple images to be processed, it is necessary to obtain a corresponding measurement indicator for each image to be processed according to steps 202 and 203.

[0167] In step 202, the first target segmentation model is called to perform region segmentation on the image to be processed to obtain a first region segmentation result corresponding to the image to be processed. The first region segmentation result is used to indicate the region where the first object in the image to be processed is located. The first target segmentation model is trained based on the first sample image and the standard region segmentation result corresponding to the first sample image. The standard region segmentation result is obtained based on the first standard region proportion label corresponding to the first object in the first sample image.

[0168] The first target segmentation model is a trained model for performing regional segmentation on an image to segment out the region where the first object in the image is located. The embodiment of the present application does not limit the type of the first target segmentation model, as long as it can output a regional segmentation result for indicating the region where the first object in the image is located. Exemplarily, the first target segmentation model is a Link Net model. The Link Net model has a relatively lightweight network and parameters, has a better balance between accuracy and speed, and can run in mid- and low-end computer devices (such as smart microscopes). Of course, the first target segmentation model can also be other types of segmentation models.

[0169] The first target segmentation model is trained based on the first sample image and the standard region segmentation result corresponding to the first sample image. The standard region segmentation result is obtained based on the first standard region ratio corresponding to the first object in the first sample image. The standard region segmentation result corresponding to the first sample image is used to standardly indicate the region where the first object in the first sample image is located. The first standard region ratio corresponding to the first object in the first sample image is used to standardly indicate the ratio of the region where the first object in the first sample image is located to the first sample image. Before implementing step 202, it is necessary to first train the first target segmentation model. For the process of training the first target segmentation model, see Figure 4 The embodiments shown are not described in detail here.

[0170] In one possible implementation, calling the first target segmentation model to perform region segmentation on the image to be processed, and obtaining a first region segmentation result corresponding to the image to be processed includes but is not limited to the following two methods:

[0171] Method 1: Call the first target segmentation model to directly perform region segmentation on the image to be processed to obtain a first region segmentation result corresponding to the image to be processed.

[0172] The image to be processed is directly input into the first target segmentation model, and the first target segmentation model directly performs region segmentation on the image to be processed and outputs a first region segmentation result corresponding to the image to be processed. Based on this method 1, the efficiency of obtaining the first region segmentation result is relatively high.

[0173] The process of the first target segmentation model directly performing region segmentation on the image to be processed is an internal processing process of the first target segmentation model, which is related to the type of the first target segmentation model and is not limited in the embodiments of the present application. Exemplarily, the first target segmentation model is a model composed of an encoder and a decoder, and the process of the first target segmentation model directly performing region segmentation on the image to be processed is: the encoder in the first target segmentation model encodes the image to be processed to obtain image features of the image to be processed; the decoder in the first target segmentation model decodes the image features of the image to be processed to obtain a first region segmentation result corresponding to the image to be processed.

[0174] Method 2: Perform attribute transformation on the image to be processed so that the image to be processed after attribute transformation has target visual attributes; call the first target segmentation model to perform region segmentation on the image to be processed after attribute transformation to obtain a first region segmentation result corresponding to the image to be processed.

[0175] The target visual attribute is a pre-set visual attribute. For example, the target visual attribute is a visual attribute that provides a good visual experience to the user. The target visual attribute can be set based on experience or flexibly adjusted according to the application scenario, and the embodiments of the present application are not limited to this. For example, the target visual attribute is composed of one or more sub-attributes. For example, the target visual attribute is composed of at least one sub-attribute of sharpness, focus, color richness, and brightness.

[0176] In this second approach, the first target segmentation model is called to perform region segmentation on the image to be processed after the attribute transformation. That is, the image to be processed after the attribute transformation is input into the first target segmentation model, which then performs region segmentation on the image to be processed after the attribute transformation and outputs a first region segmentation result corresponding to the image to be processed.

[0177] Since the image to be processed after the attribute transformation has the target visual attributes, no matter what visual attributes the image to be processed itself has, the image to be processed with the target visual attributes is input into the first target segmentation model. This method 2 is conducive to reducing the interference of the visual attributes of the image to be processed itself on the first target segmentation model, and improving the accuracy of the first region segmentation result output by the first target segmentation model.

[0178] The process of the first target segmentation model performing regional segmentation on the image to be processed after the attribute transformation is an internal processing process of the first target segmentation model, which is related to the type of the first target segmentation model and is not limited in the embodiments of the present application. Exemplarily, the first target segmentation model is a model composed of an encoder and a decoder, and the process of the first target segmentation model performing regional segmentation on the image to be processed after the attribute transformation is as follows: the encoder in the first target segmentation model encodes the image to be processed after the attribute transformation to obtain the image features of the image to be processed after the attribute transformation; the decoder in the first target segmentation model decodes the image features of the image to be processed after the attribute transformation to obtain the first region segmentation result corresponding to the image to be processed.

[0179] Before calling the first target segmentation model to perform region segmentation on the image to be processed after attribute transformation, it is necessary to first perform attribute transformation on the image to be processed so that the image to be processed after attribute transformation has target visual attributes.

[0180] In one possible implementation, the image to be processed corresponds to at least one color channel. The color channel corresponding to the image to be processed is related to the color space corresponding to the image to be processed. Exemplarily, the color space corresponding to the image to be processed is the HED (Hematoxylin-Eosin-Diaminobenzidine, DAB for short) color space, and the number of color channels corresponding to the image to be processed is three, namely, the H channel, the E channel, and the DAB channel. Exemplarily, the color space corresponding to the image to be processed is the RGB (Red-Green-Blue) color space, and the number of color channels corresponding to the image to be processed is three, namely, the R channel, the G channel, and the B channel. Exemplarily, the color space corresponding to the image to be processed is the HSV (Hue-Saturation-Value) color space, and the number of color channels corresponding to the image to be processed is three, namely, the H channel, the S channel, and the V channel. Exemplarily, color spaces can be converted, and the image to be processed corresponding to a certain color space can be converted to another color space.

[0181] In an exemplary embodiment, when the image to be processed corresponds to at least one color channel, the process of performing attribute transformation on the image to be processed includes the following steps a to c:

[0182] Step a: Based on the pixel values ​​of each pixel point in the image to be processed under each color channel, determine the pixel mean corresponding to each color channel.

[0183] Each pixel in the image to be processed corresponds to a pixel value in each color channel, and the pixel value of a pixel in a color channel is used to indicate the value of the pixel in the color channel. The process of determining the pixel mean corresponding to each color channel based on the pixel values ​​of each pixel in the image to be processed in each color channel includes: for each color channel, based on the pixel values ​​of each pixel in the image to be processed in the color channel, determining the pixel mean corresponding to the color channel.

[0184] The pixel mean for a color channel is calculated by summing the pixel values ​​for each pixel in the image being processed for that color channel and dividing that sum by the total number of pixels in the image being processed. The pixel mean for a color channel represents the average value of each pixel in the image being processed for that color channel.

[0185] Step b: Use the reference values ​​corresponding to each color channel to perform numerical transformation on the pixel mean values ​​corresponding to each color channel to obtain the pixel value transformation benchmark corresponding to each color channel. The reference values ​​corresponding to each color channel are determined based on the target visual attributes.

[0186] The reference values ​​corresponding to each color channel are determined based on the target visual attributes. For example, the reference values ​​corresponding to each color channel are set manually based on experience with reference to the target visual attributes, or by a computer device with reference to the target visual attributes. The reference values ​​corresponding to each color channel can ensure that after the pixel values ​​of each pixel in the image to be processed in each color channel are transformed according to the pixel value transformation benchmarks obtained based on the reference values ​​corresponding to each color channel, the transformed pixel values ​​of each pixel in the image to be processed in each color channel all meet the target visual attributes.

[0187] It should be noted that the reference values ​​corresponding to different color channels can be the same or different, which depends on the actual situation and is not limited in this embodiment of the present application. For example, if the color space corresponding to the image to be processed is RGB space, the reference values ​​corresponding to the R channel, G channel, and B channel are all 180.

[0188] The process of numerically transforming the pixel mean values ​​corresponding to each color channel using the reference values ​​corresponding to each color channel to obtain the pixel value transformation benchmarks corresponding to each color channel includes: for each color channel, numerically transforming the pixel mean values ​​corresponding to the color channel using the reference values ​​corresponding to the color channel to obtain the pixel value transformation benchmark corresponding to the color channel.

[0189] In an exemplary embodiment, the process of numerically transforming the pixel mean corresponding to a color channel using the reference value corresponding to the color channel can be regarded as the process of normalizing the pixel mean corresponding to the color channel under the reference value corresponding to the color channel.

[0190] Exemplarily, a reference value corresponding to a color channel is used to perform a numerical transformation on the pixel mean corresponding to the color channel to obtain a pixel value transformation benchmark corresponding to the color channel: determine the ratio of the reference value corresponding to the color channel to the pixel mean corresponding to the color channel, and use the ratio as the pixel value transformation benchmark corresponding to the color channel.

[0191] Step c: Based on the pixel value transformation benchmark corresponding to each color channel, the pixel value of each pixel point in the image to be processed under each color channel is transformed.

[0192] After obtaining the pixel value transformation benchmarks corresponding to each color channel, the pixel values ​​of each pixel in the image to be processed are transformed in each color channel based on the pixel value transformation benchmarks corresponding to each color channel. The image composed of pixels with the transformed pixel values ​​serves as the image to be processed after attribute transformation. Because the pixel value transformation benchmark corresponding to each color channel is determined based on a reference value determined based on the target visual attributes, the image composed of pixels with the transformed pixel values ​​possesses the target visual attributes.

[0193] In the process of transforming the pixel values ​​of each pixel point in the image to be processed under each color channel based on the pixel value transformation benchmark corresponding to each color channel, the pixel value transformation benchmark corresponding to each color channel is used to transform the pixel value of each pixel point in the image to be processed under that color channel.

[0194] For example, if the color space corresponding to the image to be processed is an RGB color space, the pixel values ​​of each pixel in the image to be processed in the R channel are transformed based on the pixel value transformation benchmark corresponding to the R channel; the pixel values ​​of each pixel in the image to be processed in the G channel are transformed based on the pixel value transformation benchmark corresponding to the G channel; and the pixel values ​​of each pixel in the image to be processed in the B channel are transformed based on the pixel value transformation benchmark corresponding to the B channel. After completing the transformation of the pixel values ​​of each pixel in the image to be processed in the R channel, G channel, and B channel, the image composed of the pixels with the transformed pixel values ​​is used as the image to be processed after the attribute transformation.

[0195] For example, taking the RGB color space as an example, the process of performing attribute transformation on the image to be processed based on steps a to c is implemented based on the following pseudo code:

[0196] Input: img / / Input the image to be processed

[0197] Output: result / / Output the image to be processed after attribute conversion

[0198] bgr_mean = cv2.mean(img) / / Get the pixel mean corresponding to the B channel, G channel, and R channel based on the image to be processed

[0199] kb=k / bgr_mean[0] / / The ratio of the reference value k corresponding to the B channel to the pixel mean bgr_mean[0] corresponding to the B channel is used as the pixel value conversion benchmark kb corresponding to the B channel

[0200] kg=k / bgr_mean[1] / / The ratio of the reference value k corresponding to the G channel to the pixel mean bgr_mean[1] corresponding to the G channel is used as the pixel value conversion benchmark kg

[0201] kr=k / bgr_mean[2] / / The ratio of the reference value k corresponding to the R channel to the pixel mean bgr_mean[2] corresponding to the R channel is used as the pixel value conversion benchmark kr corresponding to the R channel

[0202] result=img*[kb,kg,kr] / / Based on the pixel value transformation benchmarks corresponding to the B channel, G channel and R channel respectively, the pixel values ​​of each pixel in the image to be processed are transformed under the B channel, G channel and R channel to obtain the image to be processed after attribute conversion

[0203] In another possible implementation, the process of performing attribute transformation on the image being processed includes: obtaining an image transformation model, which is used to output an image with target visual attributes; and invoking the image transformation model to perform attribute transformation on the image being processed. In this approach, invoking the image transformation model directly to perform attribute transformation on the image being processed improves the efficiency of attribute transformation on the image being processed.

[0204] The image transformation model is used to output an image with the target visual attributes. Acquiring the image transformation model can refer to extracting a pre-trained and stored image transformation model or to training the image transformation model in real time, which is not limited in the present embodiment. Regardless of the method, the image transformation model must be trained before calling it to perform attribute transformation on the image to be processed.

[0205] In an exemplary embodiment, the image transformation model is a generative model, such as a GAN (Generative Adversarial Network) model. The image transformation model is trained using a generative adversarial approach. For example, when training the image transformation model using the generative adversarial approach, the real image is an image having target visual attributes, so that the trained image transformation model can output an image having the target visual attributes.

[0206] In an exemplary embodiment, the image transformation model is a universal model capable of transforming the attributes of images from a variety of image acquisition devices. In this case, during the training process for the image transformation model, training images from a variety of image acquisition devices are utilized. This allows the image transformation model to transform all to-be-processed images from these various image acquisition devices into images with the target visual attributes, thereby improving the applicability of the image transformation model.

[0207] In an exemplary embodiment, the image transformation model is a dedicated image transformation model used to perform attribute transformation on an image from an image acquisition device that sources the image to be processed. In this case, in the process of training the image transformation model, training is performed using training images from an image acquisition device, and the efficiency of training the image transformation model is higher.

[0208] It should be noted that no matter which method is used to perform attribute transformation on the image to be processed, since the attribute transformation changes the visual attributes of the image to be processed, it will not change the first object in the image to be processed and the area where the first object is located. Therefore, the image to be processed after the attribute transformation also includes the first object, and the area where the first object in the image to be processed after the attribute transformation is located is the same as the area where the first object in the image to be processed is located.

[0209] After calling the first target segmentation model to perform region segmentation on the image to be processed, a first region segmentation result corresponding to the image to be processed is obtained. The first region segmentation result is used to indicate the region in the image to be processed where the first object is located. In other words, based on the first region segmentation result, it is possible to determine which regions in the image to be processed are regions where the first object is located. The representation of the first region segmentation result is related to the first target segmentation model, and the embodiments of the present application do not limit the representation of the first region segmentation result.

[0210] Exemplarily, the first region segmentation result is represented as an image, in which two different colors are used to distinguish the region where the first object is located from other regions, for example, the region where the first object is located is displayed in white, and other regions are displayed in black; or, the region where the first object is located is displayed in black, and other regions are displayed in white.

[0211] Exemplarily, the representation of the first region segmentation result is a matrix or a vector, which can indicate the category of each pixel in the image to be processed, and the category of any pixel is used to indicate whether the pixel is located in the region where the first object is located. Exemplarily, if any pixel is located in the region where the first object is located, the category of the pixel is represented by a first numerical value; if any pixel is not located in the region where the first object is located, the category of the pixel is represented by a second numerical value. The first numerical value is different from the second numerical value, and both the first numerical value and the second numerical value are set based on experience or flexibly adjusted according to the application scenario, which is not limited in the embodiments of the present application. Exemplarily, the first numerical value is 1 and the second numerical value is 0; or, the first numerical value is 0 and the second numerical value is 1.

[0212] In step 203, based on the first region segmentation result, a measurement index corresponding to the image to be processed is obtained, where the measurement index is used to indicate the area ratio corresponding to the first object that meets the region restriction condition in the image to be processed.

[0213] The first region segmentation result is used to indicate the region where the first object in the image to be processed is located in the image to be processed. After obtaining the first region segmentation result, a measurement index is obtained based on the first region segmentation result to indicate the proportion of the region corresponding to the first object that meets the region restriction condition in the image to be processed. The proportion of the region corresponding to the first object that meets the region restriction condition in the image to be processed refers to the proportion of the region where the first object that meets the region restriction condition is located in the image to be processed. The measurement index corresponding to the image to be processed can measure the properties of the image to be processed in a certain aspect. In an embodiment of the present application, the measurement index corresponding to the image to be processed can measure the proportion of the region corresponding to the first object that meets the region restriction condition in the image to be processed.

[0214] The regional restriction condition is set based on experience or flexibly adjusted according to the application scenario, and the embodiments of the present application do not limit this. Exemplarily, the first object that meets the regional restriction condition refers to the first object whose area is outside certain areas in the image to be processed. Whether the first object meets the regional restriction condition can be determined based on whether the area where the first object is located is outside certain areas in the image to be processed. Exemplarily, the first object that meets the regional restriction condition refers to the first object whose area is within certain areas in the image to be processed. Whether the first object meets the regional restriction condition can be determined based on whether the area where the first object is located is within certain areas in the image to be processed. Exemplarily, the first object that meets the regional restriction condition refers to the first object whose area is not less than the area threshold. Whether the first object meets the regional restriction condition can be determined based on whether the area of ​​the area where the first object is located is less than the area threshold.

[0215] In one possible implementation, based on the first region segmentation result, a process of obtaining a measurement index corresponding to the image to be processed includes the following steps 2031 to 2033:

[0216] Step 2031: Determine the target area in the image to be processed.

[0217] The target area is used to determine which are the first objects that meet the regional restriction conditions. Which areas in the image to be processed are the target areas are related to the image to be processed, the first object and the regional restriction conditions. For example, the image to be processed is a tumor image, and the area in the tumor image can be divided into a cancer cell area and a cancer cell interstitial area. The first object is a stained cell, and the stained cell that meets the regional restriction conditions refers to a stained cell whose area is within the cancer cell interstitial area (that is, outside the cancer cell area). In this case, the target area can refer to the cancer cell area or the cancer cell interstitial area.

[0218] In one possible implementation, determining the target region in the image to be processed involves calling a target region detection model to perform region detection on the image to be processed, obtaining a target region detection result, and determining the target region in the image to be processed based on the target region detection result. This embodiment of the present application does not limit the type of target region detection model; illustratively, the target region detection model is a Link Net model.

[0219] The target region detection model is used to detect target regions in an image. Before calling the target region detection model, it must be trained. In one possible implementation, the process of training the target region detection model is as follows: obtaining training images and corresponding target region annotations; and training the initial region detection model using the training images and corresponding target region annotations in a supervised manner to obtain the target region detection model.

[0220] The target area annotations corresponding to the training images are used to indicate which areas in the training images are target areas. Exemplarily, the target area annotations corresponding to the training images are obtained by manual annotation. Exemplarily, the process of training the initial area detection model in a supervised training manner based on the training images and the target area annotations corresponding to the training images is as follows: calling the initial area detection model to detect the training images to obtain target area prediction results; comparing the target area prediction results with the target area annotations to obtain a loss function; and using the loss function to update the model parameters of the initial area detection model. The embodiment of the present application does not limit the type of loss function. Exemplarily, the loss function is an MSE (Mean-Square Error) loss function, a cross entropy loss function, an L2 norm loss function, and the like.

[0221] In an exemplary embodiment, calling the target region detection model to perform region detection on the image to be processed may refer to calling the target region detection model to perform region detection directly on the image to be processed, or may refer to calling the target region detection model to perform region detection on the image to be processed after attribute transformation, and the embodiments of the present application do not limit this. In the case of calling the target region detection model to perform region detection on the image to be processed after attribute transformation, in the process of training the target region detection model, the attributes of the training image are first transformed, and then the training image after attribute transformation is input into the initial region detection model for region detection.

[0222] Step 2032: Filter the region where the first object in the image to be processed is located using the target region to obtain the region where the first object in the image to be processed is located that meets the region restriction condition.

[0223] The first object that satisfies the region restriction condition may be located within the target region, or may be located outside the target region. In different cases, the method for filtering the region where the first object in the image to be processed is located using the target region to obtain the region in the image to be processed that satisfies the region restriction condition may vary.

[0224] Exemplarily, when the first object that meets the area restriction condition refers to the first object whose area is within the target area, the target area is used to filter the area where the first object in the image to be processed is located, and the method for obtaining the area where the first object that meets the area restriction condition in the image to be processed is: if the area where a first object is located is within the target area, then the area where the first object is located is retained; if the area where a first object is located is outside the target area, then the area where the first object is located is eliminated; and the finally retained area is used as the area where the first object that meets the area restriction condition in the image to be processed is located.

[0225] Exemplarily, when the first object that meets the area restriction condition refers to the first object whose area is outside the target area, the target area is used to filter the area where the first object in the image to be processed is located, and the method for obtaining the area where the first object that meets the area restriction condition in the image to be processed is: if the area where a first object is located is outside the target area, then retain the area where the first object is located; if the area where a first object is located is within the target area, then eliminate the area where the first object is located; and use the finally retained area as the area where the first object that meets the area restriction condition in the image to be processed is located.

[0226] That is, the target area is used to eliminate the area where the first object that does not meet the area restriction condition is located, and retain the area where the first object that meets the area restriction condition is located.

[0227] Step 2033: Compare the area where the first object that meets the area restriction condition is located with the total area of ​​the image to be processed to obtain a measurement index corresponding to the image to be processed.

[0228] The metric corresponding to the image to be processed is used to indicate the proportion of the area corresponding to the first object that meets the area restriction condition in the image to be processed. The metric corresponding to the image to be processed can be obtained by comparing the area where the first object that meets the area restriction condition is located with the total area of ​​the image to be processed. In one possible implementation, the metric corresponding to the image to be processed is obtained by comparing the area where the first object that meets the area restriction condition is located with the total area of ​​the image to be processed, including but not limited to the following two methods:

[0229] Method 1: The ratio of the total area of ​​the region where the first object that meets the region restriction condition is located to the total area of ​​the image to be processed is used as the measurement index corresponding to the image to be processed.

[0230] Method 2: Count a first number of pixels located in the area where the first object that meets the area restriction condition is located, and a second number of pixels located in the total area of ​​the image to be processed; and use the ratio of the first number to the second number as a measurement indicator corresponding to the image to be processed.

[0231] Exemplarily, the pixel points located within the total area of ​​the image to be processed are all the pixel points in the image to be processed.

[0232] It should be noted that the above steps 2031 to 2033 are an exemplary description of obtaining the measurement index corresponding to the image to be processed, and the embodiments of the present application are not limited thereto. In an exemplary embodiment, all first objects are regarded as first objects that meet the area restriction condition. In this case, based on the first area segmentation result, the method for obtaining the measurement index corresponding to the image to be processed is as follows: based on the first area segmentation result, the area where each first object is located in the image to be processed is determined; and the area where each first object is located is compared with the total area of ​​the image to be processed to obtain the measurement index corresponding to the image to be processed.

[0233] In the embodiments of the present application, calling the first target segmentation model implements the process of automatically obtaining the corresponding measurement indicators for the image to be processed. This image processing process does not require human intervention, and the image processing is relatively stable and efficient. In addition, the standard region segmentation result based on which the first target segmentation model is trained is obtained based on the standard region proportion corresponding to the first object, and has high accuracy. This allows the first target segmentation model to be trained with better region segmentation capabilities, and the measurement indicators obtained based on the first region segmentation result segmented by the first target segmentation model are highly accurate.

[0234] Based on the above Figure 1 In the implementation environment shown in the figure, the embodiment of the present application provides a model training method, which can be used to train Figure 2 The first target segmentation model called in the embodiment shown in the figure is trained by a computer device, which may be a server 12 or a terminal 11, and is not limited in this embodiment of the present application.

[0235] like Figure 4 As shown, the model training method provided in the embodiment of the present application includes the following steps 401 to 403:

[0236] In step 401, a first sample image and a first standard area ratio corresponding to a first object in the first sample image are obtained.

[0237] For example, the first sample image is Figure 2 The images of the same type as the image to be processed in the embodiment shown are used to ensure that the trained first target segmentation model can perform relatively accurate regional segmentation on the image to be processed. The first standard area ratio corresponding to the first object in the first sample image is used to indicate the proportion of the area where the first object in the first sample image is located in the first sample image. Exemplarily, the first standard area ratio corresponding to the first object in the first sample image is obtained by manual annotation. It should be noted that in the embodiment of the present application, the number of first objects in the first sample image is large and the size is small, and it is difficult to manually mark out the area where the first object is located. Usually, the area ratio corresponding to the first object in the first sample image can only be manually estimated. In the embodiment of the present application, the area ratio corresponding to the first object in the first sample image is referred to as the first standard area ratio.

[0238] The number of first sample images is one or more, which is not limited in the embodiments of the present application. Typically, the number of first sample images is multiple to ensure the training effect of the first target segmentation model. In one possible implementation, the first sample image is extracted from an image library. In another possible implementation, the first sample image is obtained by having a doctor select a tumor area in the WSI image and then select a patch image from the tumor area as the first sample image.

[0239] For example, the process of obtaining the first sample image is as follows: Figure 5 As shown. Figure 5In the figure, the area within outline 501 is the tumor area, and multiple representative patch images (images within the black box) are selected from the tumor area as the first sample images. For example, the number of first sample images is 109. The WSI image size is 50,000 × 50,000 (pixels), with a resolution of 0.344 μm / pixel. The size of each patch image is 2,160 × 2,160 (pixels), with a resolution of 0.524 μm / pixel.

[0240] After obtaining the first sample image, the first standard area ratio corresponding to the first object in the first sample image is obtained. Exemplarily, the first standard area ratio corresponding to the first object in the first sample image is obtained by manually scoring the proportion of the area where the first object in the first sample image is located in the first sample image. For example, an online scoring platform is provided to enable experts to score the first sample images through the online scoring platform. Exemplarily, multiple experts are invited to score the same batch of images to avoid some accidental errors. The scores corresponding to each first sample image are recognized by all experts, and the scores recognized by all experts are used as the first standard area ratio corresponding to the first object in the first sample image. Exemplarily, the first standard area ratio corresponding to the first object in the first sample image is the gold standard corresponding to the first sample image.

[0241] In step 402, based on the first standard area ratio, a standard area segmentation result corresponding to the first sample image is obtained, where the standard area segmentation result is used to indicate the area where the first object in the first sample image is located.

[0242] The first standard area ratio is used to indicate the proportion of the area in the first sample image where the first object is located. The first standard area ratio cannot provide effective supervision for training a first target segmentation model with good segmentation results. Therefore, after obtaining the first standard area ratio corresponding to the first object in the first sample image, it is necessary to further obtain the standard area segmentation result corresponding to the first sample image based on the first standard area ratio, so as to use the standard area segmentation result corresponding to the first sample image to provide effective supervision for the process of training the first target segmentation model.

[0243] The standard region segmentation result corresponding to the first sample image is used to more accurately indicate the region in the first sample image where the first object is located. Exemplarily, the first standard region ratio corresponding to the first object in the first sample image is referred to as the text annotation corresponding to the first sample image, and the standard region segmentation result corresponding to the first sample image is referred to as the pixel-level annotation corresponding to the first sample image. That is, in the embodiment of the present application, the text annotation corresponding to the first sample image is converted into a pixel-level annotation, thereby utilizing the pixel-level annotation to provide effective supervision for the process of training the first target segmentation model.

[0244] In one possible implementation, the process of obtaining the standard region segmentation result corresponding to the first sample image based on the first standard region proportion includes the following steps 4021 and 4022:

[0245] Step 4021: Call at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result. The at least one second target segmentation model is trained based on the second sample image and the standard region division image corresponding to the second sample image. The standard region division image corresponding to the second sample image is obtained based on the second standard region ratio corresponding to the first object in the second sample image.

[0246] The second sample image is of the same type as the first sample image. The second standard area ratio corresponding to the first object in the second sample image is used to indicate the proportion of the area where the first object in the second sample image is located in the second sample image. The standard area division image corresponding to the second sample image is used to roughly indicate the area where the first object in the second sample image is located.

[0247] The at least one second object segmentation model trained based on the second sample image and the standard region division image corresponding to the second sample image has the function of roughly segmenting the region where the first object in the first sample image is located.

[0248] Before implementing step 4021, it is necessary to first train and obtain at least one second target segmentation model. In one possible implementation, the at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models. In this case, the process of training and obtaining the at least one second target segmentation model includes the following steps 4020A and 4020B:

[0249] Step 4020A: Based on the second standard area ratio, obtain a standard area division image corresponding to the second sample image.

[0250] The second standard region division image is used to roughly mark the region where the first object is located in the second sample image. In one possible implementation, see Figure 6 The process of obtaining the standard area division image corresponding to the second sample image based on the second standard area ratio includes the following steps 4020A-1 to 4020A-4:

[0251] Step 4020A-1: Obtain a threshold value interval corresponding to a reference color channel, and a sample channel image corresponding to the second sample image under the reference color channel.

[0252] The threshold interval corresponding to the reference color channel is used to perform binarization processing on the sample channel image corresponding to the second sample image under the reference color channel, so as to preliminarily divide the area where the first object is located in the second sample image.

[0253] The reference color channel is a color channel in a color space. The reference color channel is set based on experience or flexibly adjusted according to the application scenario, and this embodiment of the application is not limited to this. For example, the reference color channel is the H channel, E channel, or DAB channel in the HED color space; or the reference color channel is the R channel, G channel, or B channel in the RGB color space; or the reference color channel is the H channel, S channel, or V channel in the HSV color space.

[0254] In one possible implementation, the threshold interval corresponding to the reference color channel is obtained based on a first reference image that does not include the first object and a second reference image that includes the first object, to ensure that the second sample image can be more accurately binarized based on the threshold interval corresponding to the reference color channel, thereby obtaining a more accurate binarized image for preliminarily dividing the area where the first object is located in the second sample image. The process of obtaining the threshold interval corresponding to the reference color channel includes the following steps 1 to 4:

[0255] Step 1: Obtain a first channel image corresponding to the first reference image under a reference color channel and a second channel image corresponding to the second reference image under a reference color channel.

[0256] The first reference image does not include the first object, and the second reference image includes the first object.

[0257] The first reference image is an image that does not include the first object. Exemplarily, the first reference image is selected from an image in which the standard area corresponding to the first object accounts for 0%. The second reference image is an image that includes the first object. Exemplarily, the second reference image is selected from an image in which the standard area corresponding to the first object accounts for more than 0%.

[0258] In an exemplary embodiment, when selecting a second reference image, an image in which the standard area corresponding to the first object accounts for a greater than reference area is selected as the second reference image from among images in which the standard area corresponding to the first object accounts for a greater than 0% portion of the image. This increases the difference between the second reference image and the first reference image, thereby more accurately determining the threshold interval corresponding to the reference color channel. The reference area is a percentage greater than 0%, which is set based on experience or flexibly adjusted based on the application scenario, for example, 10%. In other words, the second reference image is selected from an image in which the standard area corresponding to the first object accounts for a greater than 10% portion of the image.

[0259] It should be noted that the number of the first reference images is one or more, and the number of the second reference images is one or more. The number of the first reference images and the number of the second reference images may be the same or different, and this embodiment of the application does not limit this.

[0260] After obtaining the first reference image, a first channel image corresponding to the first reference image in a reference color channel is obtained. In one possible implementation, the reference color channel may or may not be a color channel in the color space corresponding to the first reference image, and this embodiment of the present application is not limited thereto.

[0261] If the reference color channel is a color channel in the color space corresponding to the first reference image, an image of the reference color channel is directly decomposed from the first reference image, and the decomposed image is used as the first channel image. If the reference color channel is not a color channel in the color space corresponding to the first reference image, the first reference image is first subjected to color space conversion so that the color space corresponding to the converted first reference image is the color space of the source of the reference color channel. Then, an image of the reference color channel is decomposed from the converted first reference image, and the decomposed image is used as the first channel image.

[0262] For example, taking the reference color channel as the DAB channel in the HED color space, the first reference image and the first channel image corresponding to the first reference image in the DAB channel are as follows: Figure 7 shown. Figure 7 (1) in is the first reference image, Figure 7 (2) in the figure is the first channel image corresponding to the first reference image under the DAB channel.

[0263] It should be noted that the number of the first reference images is one or more, and a first channel image is obtained according to each first reference image, that is, the number of the first channel images is one or more.

[0264] The principle for obtaining the second channel image corresponding to the second reference image in the reference color channel is the same as the principle for obtaining the first channel image corresponding to the first reference image in the reference color channel, and is not further described here. There are one or more second reference images, and a second channel image is obtained based on each second reference image. In other words, there are one or more second channel images.

[0265] Step 2: Obtain the first pixel value distribution of the first channel image.

[0266] The first pixel value distribution is used to indicate a distribution of pixel values ​​in a first reference image that does not include the first object. In one possible implementation, the first pixel value distribution of the first channel image is obtained by counting the number of pixels corresponding to each pixel value in the first channel image based on the pixel values ​​corresponding to the pixels in the first channel image, and obtaining the first pixel value distribution based on the number of pixels corresponding to each pixel value.

[0267] The embodiment of the present application does not limit the representation format of the first pixel value distribution. Exemplarily, the first pixel value distribution is represented by a histogram, or the first pixel value distribution is represented by a curve. Exemplarily, when the first pixel value distribution is represented by a histogram or a curve, the abscissa of the histogram or curve is the pixel value, and the ordinate is the number of pixels.

[0268] It should be noted that, in the case where there are multiple first reference images, there are also multiple first channel images, and the first pixel value distribution is the pixel value distribution obtained based on all the first channel images.

[0269] Step 3: Obtain a second pixel value distribution of the second channel image.

[0270] The second pixel value distribution is used to indicate the distribution of pixel values ​​in the second reference image including the first object. The principle of obtaining the second pixel value distribution of the second channel image is the same as the principle of obtaining the first pixel value distribution of the first channel image in step 2, and will not be repeated here.

[0271] Step 4: Determine a threshold interval corresponding to a reference color channel based on the first pixel value distribution and the second pixel value distribution.

[0272] Since the first pixel value distribution is used to indicate the distribution of pixel values ​​in the first reference image that does not include the first object, and the second pixel value distribution is used to indicate the distribution of pixel values ​​in the second reference image that includes the first object, the threshold interval determined based on the first pixel value distribution and the second pixel value distribution can be used to binarize the sample channel image corresponding to the second sample image, so as to more accurately preliminarily divide the area where the first object is located in the second sample image.

[0273] In one possible implementation, determining the threshold interval corresponding to the reference color channel based on the first pixel value distribution and the second pixel value distribution includes determining a difference pixel value distribution between the first pixel value distribution and the second pixel value distribution, and using the pixel value interval corresponding to the difference pixel value distribution as the threshold interval corresponding to the reference color channel. The difference pixel value distribution between the first pixel value distribution and the second pixel value distribution can largely indicate the pixel values ​​of pixels located within the area where the first object is located.

[0274] For example, when the first object is a stained cell, the pixel value in the pixel value interval corresponding to the difference pixel value distribution is the pixel value of the pixel point in the area where the stained cell is located. The pixel value interval corresponding to the difference pixel value distribution is used as the threshold interval corresponding to the reference color channel. The threshold interval can be used to more accurately preliminarily divide the area where the stained cell is located.

[0275] For example, the first pixel value distribution and the second pixel value distribution are as follows: Figure 8 As shown. Figure 8 In the figure, curve 801 represents a first pixel value distribution, and curve 802 represents a second pixel value distribution. Based on curves 801 and 802, the difference pixel value distribution between the first pixel value distribution and the second pixel value distribution can be determined. Two vertical dashed lines are used to mark the difference pixel value distribution between the first pixel value distribution and the second pixel value distribution. The position between the two dashed lines is the position of the pixel value interval [165, 195]. Therefore, [165, 195] is used as the threshold interval corresponding to the reference color channel.

[0276] It should be noted that the above steps 1 to 4 are an exemplary description of obtaining the threshold interval corresponding to the reference color channel, and the embodiments of the present application are not limited thereto. Exemplarily, the threshold interval corresponding to the reference color channel is set based on experience.

[0277] Next, we will describe the process of obtaining the sample channel image corresponding to the second sample image under the reference color channel. If the reference color channel is the color channel in the color space corresponding to the first sample image, the image of the reference color channel is directly decomposed from the first sample image, and the decomposed image is used as the sample channel image. If the reference color channel is not the color channel in the color space corresponding to the first sample image, the first sample image is first color-space converted so that the color space corresponding to the converted first sample image is the color space of the reference color channel. Then, the image of the reference color channel is decomposed from the converted first sample image, and the decomposed image is used as the sample channel image.

[0278] Step 4020A-2: Binarize the sample channel image using the threshold interval to obtain a binary image.

[0279] After obtaining the threshold interval corresponding to the reference color channel, the sample channel image is binarized using the threshold interval, and the image obtained after the binarization is used as the binarized image. The binarized image can preliminarily demarcate the area where the first object in the second sample image is located.

[0280] In one possible implementation, the process of binarizing the sample channel image using a threshold interval is as follows: if the pixel value of a certain pixel point in the sample channel image is within the threshold interval, the pixel value of the pixel point is set to the first pixel value; if the pixel value of a certain pixel point in the sample channel image is not within the threshold interval, the pixel value of the pixel point is set to the second pixel value. After completing the adjustment of the pixel values ​​of all pixels in the sample channel image in this way, a binary image is obtained. The first pixel value and the second pixel value are set based on experience, or flexibly adjusted according to the application scenario. For example, the first pixel value is 255 and the second pixel value is 0. For example, in the binary image, white is used to represent the pixel point with the first pixel value, and black is used to represent the pixel point with the second pixel value.

[0281] For example, the binary image obtained after binarization of the sample channel image is as follows Figure 9 As shown. Figure 9 In FIG, a white area is used to represent the area where the first object in the second sample image is located after preliminary division.

[0282] Step 4020A-3: Perform morphological processing on the binary image to obtain at least one candidate region division image.

[0283] The accuracy of the region where the first object in the second sample image is located, initially delineated from the binarized image, may be poor. Therefore, morphological processing is required on the binarized image to obtain at least one candidate region delineation image. Each candidate region delineation image is capable of delineating the region where the first object in the second sample image is located. At least one candidate region delineation image may contain both accurately delineated candidate region delineation images and inaccurately delineated candidate region delineation images. One accurately delineated candidate region delineation image is selected from the at least one candidate region delineation image as the standard region delineation image corresponding to the second sample image.

[0284] In one possible implementation, after obtaining the binarized image, the predicted area ratio corresponding to the first object in the second sample image is determined based on the binarized image. The predicted area ratio corresponding to the first object in the second sample image refers to the proportion of the area where the first object is located in the second sample image, which is divided according to the binarized image. Exemplarily, the method for determining the predicted area ratio corresponding to the first object in the second sample image based on the binarized image is as follows: the area where the first object is located in the second sample image, which is divided according to the binarized image, is compared with the total area of ​​the binarized image to obtain the predicted area ratio corresponding to the first object in the second sample image. The area of ​​the total area of ​​the binarized image is the same as the area of ​​the total area of ​​the second sample image.

[0285] The predicted area ratio corresponding to the first object in the second sample image may be less than the second standard area ratio, or may be equal to the second standard area ratio, or may be greater than the second standard area ratio. This embodiment of the present application does not limit this. The embodiment of the present application is explained by taking the example that the predicted area ratio corresponding to the first object in the second sample image is less than or greater than the second standard area ratio. If the predicted area ratio corresponding to the first object in the second sample image is equal to the second standard area ratio, the binarized image is directly used as the standard area division image corresponding to the second sample image.

[0286] In one possible implementation, when the predicted region ratio is less than the second standard region ratio, it indicates that the region where the first object in the second sample image is located, as divided from the binary image, is smaller than the region where the first object actually resides in the second sample image. In this case, the process of performing morphological processing on the binary image to obtain at least one candidate region division image includes: in response to the predicted region ratio being less than the second standard region ratio, dilating the binary image using a reference dilation kernel to obtain the at least one candidate region division image.

[0287] The reference dilation kernel is used to dilate the binary image to expand the region within the second sample image delineated from the binary image. The size and type of the reference dilation kernel are set empirically or flexibly adjusted based on the application scenario, and are not limited in this embodiment of the present application. For example, the reference dilation kernel is a circle with a radius of 1 pixel; alternatively, the reference dilation kernel is a square with a side length of 3 pixels.

[0288] The process of dilating a binary image using a reference dilation kernel to obtain at least one candidate region division image is an iterative process: dilating the binary image using the reference dilation kernel to obtain the first candidate region division image; dilating the previous candidate region division image using the reference dilation kernel to obtain the next candidate region division image, and repeating this process until at least one required candidate region division image is obtained. The embodiment of the present application does not limit the number of candidate region division images. The dilation process is iteratively performed as many times as the number of candidate region division images required. For example, the process of dilating a binary image using a reference dilation kernel is the process of convolving the binary image using the reference dilation kernel to expand the area where the first object is located in the second sample image divided by the binary image.

[0289] Exemplarily, when the predicted area accounts for less than the second standard area proportion (for example, the predicted area accounts for 0.6%, and the second standard area accounts for 2.0%), the area where the first object in the second sample image is actually located can be regarded as the sum of the area where the first object in the second sample image divided from the binary image is located and a certain surrounding area. Therefore, the process of dilating the binary image can be regarded as a process of expanding the area where the first object in the second sample image divided from the binary image is located pixel by pixel to obtain a certain surrounding area.

[0290] In an exemplary embodiment, dilation processing using a reference dilation kernel can make the area where the first object in the second sample image divided from the binary image is located "larger", merge the background points that are in contact with the area where the first object in the second sample image divided from the binary image is located into the area where the first object in the second sample image is located, and expand the boundary of the area where the first object in the second sample image is located outward.

[0291] For example, the reference dilation kernel used in the dilation process is disk(1) (a circle with a radius of 1 pixel) to achieve the minimum expansion each time. The logic of the dilation process using the dilation kernel of disk(1) is as follows:

[0292] kernel = disk(1).astype(np.uint8) / / Convert the reference expansion kernel into an integer type

[0293] mask_ic = cv2.dilate(mask_init, kernel, iterations = ITERATION) / / Iterate to obtain at least one candidate region partition image

[0294] Wherein, ITERATION represents the number of iterations, that is, the number of candidate region partition images required. For example, the number of iterations is 5, and the number of iterations can be flexibly changed according to actual needs; mask_init represents the binary image; mask_ic represents the candidate region partition image obtained after each dilation process.

[0295] In one possible implementation, when the predicted area ratio is greater than the second standard area ratio, it indicates that the area where the first object in the second sample image is located, divided from the binary image, is larger than the area where the first object actually resides in the second sample image. In this case, the process of performing morphological processing on the binary image to obtain at least one candidate area division image includes: in response to the predicted area ratio being greater than the second standard area ratio, performing reduction processing on the binary image using a reference reduction kernel to obtain the at least one candidate area division image.

[0296] The reference reduction kernel is used to reduce the binarized image to reduce the area within the second sample image demarcated from the binarized image. The size and type of the reference reduction kernel are set empirically or flexibly adjusted based on the application scenario, and are not limited in this embodiment of the present application. For example, the reference reduction kernel is a circle with a radius of 1 pixel; alternatively, the reference reduction kernel is a square with a side length of 3 pixels, etc.

[0297] The process of reducing the binary image using the reference reduction kernel to obtain at least one candidate region division image is an iterative process: the binary image is reduced using the reference reduction kernel to obtain the first candidate region division image; the previous candidate region division image is reduced using the reference reduction kernel to obtain the next candidate region division image, and this cycle is repeated until at least one required candidate region division image is obtained. The reduction process is iteratively performed using the reduction and dilation kernel as many times as the number of required candidate region division images. Exemplarily, the process of reducing the binary image using the reference reduction kernel is the process of convolving the binary image using the reference reduction kernel to reduce the region where the first object is located in the second sample image divided by the binary image.

[0298] In an exemplary embodiment, the reduction processing using the reference reduction kernel can make the area where the first object in the second sample image divided from the binary image is located "smaller" and reduce the boundary with the area where the first object in the second sample image divided from the binary image is located inward.

[0299] After obtaining at least one candidate region image, a candidate region ratio corresponding to the first object in the second sample image can be obtained based on the region where the first object in the second sample image is located, which is divided in each candidate region image. The method for obtaining the candidate region ratio corresponding to the first object in the second sample image is similar to the method for obtaining the predicted region ratio corresponding to the first object in the second sample image, and is not further described here.

[0300] For example, taking the number of iterations as 5, Figure 9 Based on the binary image shown in the figure, after iteratively performing the expansion process 5 times, the 5 candidate region division images obtained are as follows: Figure 10 As shown in (1) to (5) in . Figure 10 The white area in the figure represents the area where the first object is located. Figure 10 It can be seen that as the number of iterations increases, the area where the first object is located is gradually expanded.

[0301] For example, based on Figure 10 The candidate area determined by the area where the first object is located in the candidate area division image shown in (1) is 1.1%; based on Figure 10 The candidate area determined by the area where the first object is located in the candidate area division image shown in (2) accounts for 1.8%; based on Figure 10 The candidate area determined by the area where the first object is located in the candidate area division image shown in (3) is 2.5%; based on Figure 10 The candidate area determined by the area where the first object is located in the candidate area division image shown in (4) is 3.4%; Figure 10 The candidate area determined by the area where the first object is located in the candidate area division image shown in (5) accounts for 4.3%.

[0302] Step 4020A-4: Using the candidate region division image that satisfies the second matching condition with the second standard region ratio in at least one candidate region division image as the standard region division image corresponding to the second sample image.

[0303] Based on each candidate region division image, a candidate region ratio corresponding to the first object in the second sample image can be determined. In one possible implementation, the implementation process of step 4020A-4 is as follows: based on at least one candidate region division image, determine at least one candidate region ratio corresponding to the first object in the second sample image; use the candidate region ratio with the smallest difference from the second standard region ratio as the first target region ratio; use the candidate region division image based on which the first target region ratio is determined as the candidate region division image that satisfies the second matching condition with the second standard region ratio; and use the candidate region division image that satisfies the second matching condition with the second standard region ratio as the standard region division image corresponding to the second sample image.

[0304] For example, for Figure 10 For the five candidate region partition images in the , the candidate region partition image obtained based on the second iteration (i.e. Figure 10 The candidate area division image shown in (2) in the figure) has a candidate area ratio (1.8%) that is closest to the second standard area ratio (2%), and the candidate area division image obtained in the second iteration is used as the standard area division image corresponding to the second sample image.

[0305] For example, the standard region division image corresponding to the second sample image can be regarded as a box-level annotation. In the standard region division image, each pixel is represented by a box, and the shape of the box is related to the type and size of the kernel used in the morphological processing process. Figure 10 It can be seen that by using a circular dilation kernel with a radius of 1 pixel for dilation processing, each pixel can be represented by a diamond box.

[0306] For example, the process of obtaining the standard region segmentation image corresponding to the second sample image based on steps 4020A-1 to 4020A-4 is summarized as follows: (1) Using prior knowledge and channel images, a statistical comparison is performed between the first pixel value distribution of the first reference image excluding the first object and the second pixel value distribution of the second reference image including the first object to determine a threshold interval. (2) Using the threshold interval, a binary image is obtained that can be used to preliminarily segment the region where the first object is located in the second sample image. Based on the binary image, morphological processing and the second standard region ratio are used for supervision to obtain box-level annotations (i.e., the standard region segmentation image).

[0307] It should be noted that steps 4020A-1 to 4020A-4 above describe the process of obtaining a standard region division image corresponding to a second sample image, using a single second sample image as an example. If there are multiple second sample images, steps 4020A-1 to 4020A-4 are performed for each second sample image to obtain a standard region division image corresponding to the second sample image.

[0308] Step 4020B: Based on the second sample image and the standard region division image, a reference number of second initial segmentation models are trained respectively to obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models through different numbers of training rounds.

[0309] The reference number is set based on experience or flexibly adjusted according to the application scenario, and is not limited in the embodiments of the present application. For example, the reference number is 2, or the reference number is 3, etc.

[0310] The second initial segmentation models used for training the reference number may refer to all second sample images and the standard area division images corresponding to all second sample images, or may refer to second sample images that meet the selection conditions and the standard area division images corresponding to the second sample images that meet the selection conditions. This embodiment of the present application does not limit this. For example, the second sample image that meets the selection conditions refers to a second sample image whose area ratio based on the corresponding standard area division image is not greater than the difference between the area ratio of the second standard area ratio and the difference threshold. The difference threshold is set based on experience or flexibly adjusted according to the application scenario. This embodiment of the present application does not limit this. For example, the difference threshold is 0.6%.

[0311] For example, in the actual processing process, the number of second sample images is 109, and the second sample images that meet the selection conditions account for about 76% (that is, 83 second sample images). The remaining second sample images cannot obtain more accurate standard area division images. In the process of training the reference number of second initial segmentation models, discarding this part of the second sample images that have not obtained more accurate standard area division images is beneficial to improving the processing performance of at least one second target segmentation model obtained by training.

[0312] The reference number of second initial segmentation models are different segmentation models so that the segmentation performance of different segmentation models complement each other. Exemplarily, the reference number is 2, and the two second initial segmentation models are FC-Dense Net (Fully Convolutional Dense Net) model and Mobile Net (mobile network) model. The network structure of the FC-Dense Net model is relatively complex but the processing speed is slow, and the network structure of the Mobile Net model is relatively simple but the processing speed is fast. Of course, the reference number can also be other values, and the reference number of second initial segmentation models can also be other segmentation models. The second sample images based on which different second initial segmentation models are trained can be the same or different, and the embodiments of the present application do not limit this.

[0313] By training each of the reference number of second initial segmentation models, at least one second target segmentation model corresponding to the second initial segmentation model can be obtained by training the second initial segmentation model for a different number of rounds. For example, one round of training can be referred to as an epoch, and the at least one second target segmentation model corresponding to the second initial segmentation model can be obtained by training the second initial segmentation model for a different number of epochs.

[0314] For example, assuming that the number of second sample images used to train a second initial segmentation model is 10, then when the model parameters of the second initial segmentation model are updated once using each of the 10 second sample images, one round of training for the second initial segmentation model is completed. In the process of updating the model parameters of the second initial segmentation model once using each of the 10 second sample images, one or more second sample images may be selected for updating each time until the model parameters of the second initial segmentation model are updated once using each of the 10 second sample images.

[0315] Exemplarily, based on the second sample image and the standard region division image, a second initial segmentation model is trained to obtain at least one second target segmentation model corresponding to the second initial segmentation model obtained through different rounds of training. The process is as follows: based on the second sample image and the standard region division image, the second initial segmentation model is trained for a first reference number of rounds to obtain a first reference number of candidate segmentation models corresponding to the second initial segmentation model; and the last second reference number of candidate segmentation models in the first reference number of candidate segmentation models corresponding to the second initial segmentation model are used as the second target segmentation model corresponding to the second initial segmentation model. In other words, the number of at least one second target segmentation model obtained through different rounds of training corresponding to the second initial segmentation model is the second reference number. The first reference number and the second reference number are set based on experience or flexibly adjusted according to the application scenario. For example, the first reference number is 300 and the second reference number is 100.

[0316] The process of training each second initial segmentation model based on the second sample image and the standard region division image is a supervised training process, and the standard region division image is used to provide a supervisory signal.

[0317] It should be noted that, for different second initial segmentation models, the first reference number may vary or remain unchanged; the second reference number may also vary or remain unchanged, which is not limited in the embodiment of the present application.

[0318] Exemplarily, in the process of training at least one second target segmentation model, the number of second sample images used is 83, the number of verification images is 26, and the size of each image is 1080*1080 (pixels), 1.048μm / pixel (micrometers per pixel).

[0319] It should be noted that the second sample image may be the same as the first sample image or may be different from the second sample image. In the case where the second sample image is different from the first sample image, before implementing step 4020A, it is necessary to first obtain the second sample image and the second standard area ratio corresponding to the first object in the second sample image. The method for obtaining the second sample image and the second standard area ratio corresponding to the first object in the second sample image is similar to the method for obtaining the first sample image and the first standard area ratio corresponding to the first object in the first sample image in step 401, which will not be repeated here. After obtaining the second standard area ratio corresponding to the first object in the second sample image and the second sample image, execute steps 4020A and 4020B to obtain at least one second target segmentation model, and then execute step 4021.

[0320] In the case where the second sample image is the same as the first sample image, after obtaining the first sample image and the first standard area ratio corresponding to the first object in the first sample image based on step 401, the first sample image is used as the second sample image, and the first standard area ratio corresponding to the first object in the first sample image is used as the second standard area ratio corresponding to the first object in the second sample image, and then steps 4020A and 4020B are directly executed to obtain at least one second target segmentation model, and then step 4021 is executed.

[0321] After at least one second target segmentation model is obtained through training, step 4021 may be executed, that is, calling at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result.

[0322] By calling each second target segmentation model in at least one target segmentation model to perform region segmentation on the first sample image, a candidate region segmentation result can be obtained. That is, the number of candidate region segmentation results is the same as the number of second target segmentation models. Exemplarily, at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models, so as to ensure the comprehensiveness of the candidate region segmentation results. Each candidate region segmentation result is used to indicate the region where the first object in the first sample image is located, and different candidate region segmentation results may indicate different regions where the first object in the first sample image is located.

[0323] For example, using a reference size of 2, the two second initial segmentation models are each trained for 300 epochs. The segmentation models obtained after the final 100 epochs of training for each second initial segmentation model serve as the 100 second target segmentation models corresponding to each second initial segmentation model. The segmentation models obtained after the final 100 epochs of training have good segmentation accuracy. Based on this, each second initial segmentation model corresponds to 100 second target segmentation models, and the two second initial segmentation models correspond to a total of 200 second target segmentation models. These 200 second target segmentation models are then used to perform region segmentation on the first sample image, resulting in 200 candidate region segmentation results.

[0324] Step 4022: taking the candidate region segmentation result whose proportion with the first standard region satisfies the first matching condition among the at least one candidate region segmentation result as the standard region segmentation result corresponding to the first sample image.

[0325] Each candidate region segmentation result is used to indicate the region where the first object in the first sample image is located. According to the region where the first object in the first sample image is located indicated by each candidate segmentation result, a candidate region proportion corresponding to the first object in the first sample image can be obtained.

[0326] In one possible implementation, the implementation process of step 4022 is: based on at least one candidate area division result, determine at least one candidate area proportion corresponding to the first object in the first sample image; use the candidate area proportion with the smallest difference from the first standard area proportion as the second target area proportion, use the candidate area segmentation result based on which the second target area proportion is determined as the candidate area segmentation result that satisfies the first matching condition with the first standard area proportion, and use the candidate area segmentation result that satisfies the first matching condition with the first standard area proportion as the standard area segmentation result corresponding to the first sample image.

[0327] For example, if the number of second target segmentation models is 200, 200 candidate region segmentation results can be obtained. Using the first standard region ratio corresponding to the first object in the first sample image, the candidate region segmentation result that can determine the candidate region ratio closest to the first standard region ratio is selected from the 200 candidate region segmentation results as the standard region segmentation result corresponding to the first sample image. Exemplarily, the standard region segmentation result corresponding to the first sample image can be regarded as the pixel-level annotation corresponding to the first sample image.

[0328] Exemplarily, the area where the first object is located indicated by the standard area segmentation result predicted by the second target segmentation model trained based on the box-level annotation (i.e., the standard area segmentation image) is more natural than the edge of the area where the first object is located indicated by the box-level annotation.

[0329] For example, Figure 11 As shown, the first sample image and the standard region segmentation results corresponding to the first sample image represented in image form are respectively as shown in FIG. Figure 11 As shown in (1) and (2) in .

[0330] For example, taking the second sample image as the same as the first sample image, the process of obtaining the standard region segmentation result corresponding to the first sample image is as follows: Figure 12 As shown. A first sample image 1201 and a text annotation corresponding to the first sample image (i.e., the first standard area ratio corresponding to the first object in the first sample image) are obtained; based on the text annotation corresponding to the first sample image, a box-level annotation corresponding to the first sample image (i.e., the standard area segmentation image corresponding to the first sample image) is obtained, and the box-level annotation displayed in image form is shown as 1202; based on the box-level annotation corresponding to the first sample image, a pixel-level annotation corresponding to the first sample image (i.e., the standard area segmentation result corresponding to the first sample image) is obtained, and the pixel-level annotation displayed in image form is shown as 1203.

[0331] In the process of obtaining the pixel-level annotation corresponding to the first sample image based on the box-level annotation corresponding to the first sample image, firstly, based on the first sample image and the box-level annotation corresponding to the first sample image, two second initial segmentation models (second initial segmentation model 1 and second initial segmentation model 2) are trained to obtain N (N is an integer not less than 1) second target segmentation models obtained by training different rounds corresponding to the two second initial segmentation models. The N second target segmentation models corresponding to the two second initial segmentation models are called to perform region segmentation on the first sample image to obtain the candidate region segmentation results predicted by each second target segmentation model. Based on the text annotation corresponding to the first sample image, the candidate region segmentation result that meets the first matching condition with the first standard region ratio is determined from each candidate region segmentation result as the pixel-level annotation corresponding to the first sample image.

[0332] It should be noted that what is described in the above steps 4021 and 4022 is an exemplary description of obtaining the standard area segmentation result corresponding to the first sample image based on the first standard area ratio, and the embodiments of the present application are not limited to this. Exemplarily, the method for obtaining the standard area segmentation result corresponding to the first sample image based on the first standard area ratio is: based on the first standard area ratio, obtain the standard area division image corresponding to the first sample image, and use the standard area division image corresponding to the first sample image as the standard area segmentation result corresponding to the first sample image. The efficiency of obtaining the standard area segmentation result corresponding to the first sample image based on this method is relatively high. For the implementation method of obtaining the standard area division image corresponding to the first sample image based on the first standard area ratio, please refer to the implementation method of obtaining the standard area division image corresponding to the second sample image based on the second standard area ratio in step 4020A, which will not be repeated here.

[0333] It should be noted that the embodiment of the present application takes the case where the number of first sample images is one as an example to introduce the process of obtaining the standard region segmentation result corresponding to the first sample image. If the number of first sample images is multiple, it is necessary to obtain the standard region segmentation result corresponding to each first sample image separately, and then execute step 403.

[0334] In step 403, a first initial segmentation model is trained based on the first sample image and the standard region segmentation result to obtain a first target segmentation model.

[0335] The process of training the first initial segmentation model based on the first sample image and the standard region segmentation result corresponding to the first sample image is a supervised training process, and the standard region segmentation result corresponding to the first sample image is used to provide a supervisory signal for the supervised training process.

[0336] In one possible implementation, the first initial segmentation model is trained based on the first sample image and the standard region segmentation result to obtain the first target segmentation model in the following two ways, but not limited to:

[0337] Method 1: directly train the first initial segmentation model based on the first sample image and the standard region segmentation result to obtain the first target segmentation model.

[0338] In this way, the first initial segmentation model is called to directly perform region segmentation on the first sample image to obtain the predicted region segmentation result; based on the predicted region segmentation result and the standard region segmentation result, the loss function is obtained; the first initial segmentation model is trained using the loss function to obtain the first target segmentation model.

[0339] Method 2: Perform attribute transformation on the first sample image so that the first sample image after attribute transformation has target visual attributes; train the first initial segmentation model based on the first sample image after attribute transformation and the standard region segmentation result to obtain a first target segmentation model.

[0340] In this second approach, before calling the first initial segmentation model for processing, the first sample image is first subjected to attribute transformation so that the first sample image after attribute transformation has the target visual attribute. The principle of performing attribute transformation on the first sample image so that the first sample image after attribute transformation has the target visual attribute is similar to that of Figure 2 In the embodiment shown, the principle of performing attribute transformation on the image to be processed so that the image to be processed after attribute transformation has the target visual attribute is the same and will not be repeated here.

[0341] After obtaining the attribute-transformed first sample image, the first initial segmentation model is invoked to perform region segmentation on the attribute-transformed first sample image to obtain a predicted region segmentation result. A loss function is obtained based on the predicted region segmentation result and the standard region segmentation result. The loss function is then used to train the first initial segmentation model to obtain a first target segmentation model. This second method is advantageous in reducing the number of first sample images required for training, allowing the training process to converge more quickly.

[0342] For example, the number of first sample images used to train the first target segmentation model is 83, and the number of validation images is 36. A 3-fold approach was used to train the first target segmentation model and test its segmentation accuracy. For example, segmentation accuracy was measured using the F1 score. The test results for segmentation accuracy are shown in Table 1. As can be seen from the test results in Table 1, the first target segmentation model achieved good segmentation accuracy and stability.

[0343] Table 1

[0344] Test dataset Segmentation accuracy F1 score Fold 1 0.843 Fold 2 0.856 Fold 3 0.837

[0345] After obtaining the trained first target segmentation model, the first target segmentation model has the function of performing relatively accurate regional segmentation on the first object in the image, so that the first target segmentation model can be called to perform regional segmentation on the image including the first object that needs to be processed, thereby obtaining the regional segmentation result corresponding to the image, and then obtaining the measurement index corresponding to the image based on the regional segmentation result corresponding to the image.

[0346] Exemplarily, after training the first target segmentation model, the process further includes: obtaining an image to be processed, the image to be processed including a first object; calling the first target segmentation model to perform region segmentation on the image to be processed, obtaining a first region segmentation result corresponding to the image to be processed, the first region segmentation result being used to indicate the region where the first object in the image to be processed is located; based on the first region segmentation result, obtaining a measurement index corresponding to the image to be processed, the measurement index being used to indicate the region ratio corresponding to the first object in the image to be processed that meets the region restriction condition. The implementation of this process is detailed in Figure 2 The embodiments shown are not described in detail here.

[0347] In an embodiment of the present application, a standard area segmentation result is obtained based on the standard area ratio, and the accuracy of the standard area segmentation result is high. The first target segmentation model trained according to the standard area segmentation result has a good area segmentation function, so that the first target segmentation model can be called to realize the process of automatically obtaining the measurement indicators corresponding to the image to be processed, which is conducive to improving the stability and efficiency of image processing and the accuracy of the measurement indicators corresponding to the image to be processed.

[0348] For example, the process of obtaining the measurement index corresponding to the image to be processed is as follows: Figure 13 Based on the text annotation corresponding to the first sample image, obtain the box-level annotation corresponding to the first sample image; based on the box-level annotation corresponding to the first sample image, obtain the pixel-level annotation corresponding to the first sample image. Figure 12The introduction of the method will not be repeated here. The first sample image is subjected to attribute transformation to obtain the first sample image 1301 after attribute transformation; based on the first sample image 1301 after attribute transformation and the pixel-level annotation 1302 corresponding to the first sample image, a first target segmentation model 1303 is trained to obtain the first target segmentation model, which is used to perform region segmentation on the image to obtain a region segmentation result indicating the region where the first object in the image is located. Based on the first sample image 1301 after attribute transformation and the target region annotation 1304 corresponding to the first sample image, a target region detection model 1305 is trained to detect the target region in the image (such as the cancer cell region or the cancer cell interstitial region).

[0349] When it is necessary to obtain a metric corresponding to the image to be processed, the first target segmentation model 1303 is called to perform region segmentation on the image to be processed after the attribute transformation, obtaining a first region segmentation result indicating the region where the first object in the image to be processed is located. The first region segmentation result, represented in image form, is shown in 1306. The target region detection model 1305 is called to perform region detection on the image to be processed after the attribute transformation, obtaining a target region detection result. The target region detection result, represented in image form, is shown in 1307. Based on the target region detection result and the first region segmentation result, the metric corresponding to the image to be processed is obtained.

[0350] Based on the above Figure 1 In the implementation environment shown, the embodiment of the present application provides an image processing method, which is executed by a computer device. The computer device can be a server 12 or a terminal 11, and the embodiment of the present application does not limit this. Figure 14 As shown, the image processing method provided in the embodiment of the present application includes the following steps 1401 to 1404:

[0351] In step 1401, an image to be processed is acquired, where the image to be processed includes a first object.

[0352] The implementation of step 1401 is shown in Figure 2 Step 201 in the illustrated embodiment will not be described in detail here.

[0353] In step 1402 , attribute transformation is performed on the image to be processed so that the image to be processed after attribute transformation has target visual attributes.

[0354] The target visual attribute is a pre-set visual attribute. For example, the target visual attribute is a visual attribute that provides a good visual experience to the user. The target visual attribute can be set based on experience or flexibly adjusted according to the application scenario, and the embodiments of the present application are not limited to this. For example, the target visual attribute is composed of one or more sub-attributes. For example, the target visual attribute is composed of at least one sub-attribute of sharpness, focus, color richness, and brightness.

[0355] In one possible implementation, the image to be processed corresponds to at least one color channel, and the process of performing attribute transformation on the image to be processed is: based on the pixel values ​​of each pixel point in the image to be processed under each color channel, determining the pixel mean values ​​corresponding to each color channel respectively; using the reference values ​​corresponding to each color channel respectively, performing numerical transformation on the pixel mean values ​​corresponding to each color channel respectively, to obtain the pixel value transformation benchmark corresponding to each color channel respectively, and the reference values ​​corresponding to each color channel respectively are determined based on the target visual attributes; based on the pixel value transformation benchmark corresponding to each color channel respectively, transforming the pixel values ​​of each pixel point in the image to be processed under each color channel.

[0356] In another possible implementation, the process of performing attribute transformation on the image being processed includes: obtaining an image transformation model, which is used to output an image with target visual attributes; and invoking the image transformation model to perform attribute transformation on the image being processed. In this approach, invoking the image transformation model directly to perform attribute transformation on the image being processed improves the efficiency of attribute transformation on the image being processed.

[0357] In step 1403, the first target segmentation model is called to perform region segmentation on the image to be processed after the attribute transformation, and a first region segmentation result corresponding to the image to be processed is obtained. The first region segmentation result is used to indicate the region where the first object in the image to be processed is located.

[0358] The first target segmentation model is a trained model for performing regional segmentation on an image with target visual attributes to segment the region where the first object in the image is located. The embodiment of the present application does not limit the type of the first target segmentation model, as long as it can output a regional segmentation result for indicating the region where the first object in the image is located. Exemplarily, the first target segmentation model is a Link Net model. The Link Net model has a relatively lightweight network and parameters, has a better balance between accuracy and speed, and can run in mid- and low-end computer devices (such as smart microscopes). Of course, the first target segmentation model can also be other types of segmentation models.

[0359] The first target segmentation model is trained based on the first sample image after attribute transformation and the standard region segmentation result corresponding to the first sample image. The first sample image after attribute transformation is obtained by performing attribute transformation on the first sample image. The first sample image after attribute transformation has the target visual attribute. Before implementing step 1403, it is necessary to first train the first target segmentation model. The process of training the first target segmentation model is described in detail in the following. Figure 15 The embodiments shown are not described in detail here.

[0360] For implementation of step 1402 and step 1403, see Figure 2 In the embodiment shown, method 2 of calling the first target segmentation model in step 202 to perform region segmentation on the image to be processed to obtain a first region segmentation result corresponding to the image to be processed will not be described in detail here.

[0361] In step 1404, based on the first region segmentation result, a measurement index corresponding to the image to be processed is obtained, where the measurement index is used to indicate the area ratio corresponding to the first object that meets the region restriction condition in the image to be processed.

[0362] In one possible implementation, based on the first region segmentation result, the process of obtaining the measurement index corresponding to the image to be processed is: determining the target region in the image to be processed; using the target region to filter the region where the first object in the image to be processed is located, and obtaining the region where the first object that meets the region restriction condition is located in the image to be processed; comparing the region where the first object that meets the region restriction condition is located with the total area of ​​the image to be processed, and obtaining the measurement index corresponding to the image to be processed.

[0363] The implementation of step 1404 is shown in Figure 2 Step 203 in the illustrated embodiment will not be described in detail here.

[0364] In the embodiments of the present application, calling the first target segmentation model implements the process of automatically obtaining the measurement indicators corresponding to the image to be processed. This image processing process does not require human intervention, and the image processing is stable and efficient. In addition, before calling the first target segmentation model to perform region segmentation, the image to be processed is first transformed into an image with the target visual attributes, thereby reducing the interference caused by the visual attributes of the image to be processed itself on the first target segmentation model. This makes the first region segmentation result obtained by calling the first target segmentation model more accurate, thereby improving the accuracy of the obtained measurement indicators.

[0365] Based on the above Figure 1 In the implementation environment shown in the figure, the embodiment of the present application provides a model training method, which can be used to train Figure 14The first target segmentation model called in the embodiment shown in the figure is trained by a computer device, which may be a server 12 or a terminal 11, and is not limited in this embodiment of the present application.

[0366] like Figure 15 As shown, the model training method provided in the embodiment of the present application includes the following steps 1501 to 1503:

[0367] In step 1501, a first sample image and a standard region segmentation result corresponding to the first sample image are obtained, where the standard region segmentation result is used to indicate a region where a first object in the first sample image is located.

[0368] For example, the first sample image is Figure 14 The images to be processed in the embodiment shown are of the same type, so as to ensure that the first target segmentation model obtained by training can perform relatively accurate region segmentation on the image to be processed after attribute transformation. Figure 4 The method of obtaining the first sample image in step 401 in the illustrated embodiment will not be described in detail here.

[0369] The standard region segmentation result corresponding to the first sample image is used to indicate the region in the first sample image where the first object is located. In other words, based on the standard region segmentation result, it is possible to determine which regions in the first sample image are regions where the first object is located. The embodiments of the present application do not limit the representation format of the standard region segmentation result. For example, the representation format of the standard region segmentation result is an image, or the representation format of the standard region segmentation result is a matrix or a vector.

[0370] The embodiments of the present application do not limit the method for obtaining the standard region segmentation result corresponding to the first sample image. In an exemplary embodiment, the method for obtaining the standard region segmentation result corresponding to the first sample image is: obtaining the standard region segmentation result corresponding to the first sample image based on the region where the first sample object is located in the manually annotated first sample image.

[0371] In an exemplary embodiment, the standard area segmentation result corresponding to the first sample image is obtained by: obtaining the first standard area ratio corresponding to the first object in the first sample image; and obtaining the standard area segmentation result corresponding to the first sample image based on the first standard area ratio.

[0372] In one possible implementation, based on the first standard area ratio, the method for obtaining the standard area segmentation result corresponding to the first sample image is: calling at least one second target segmentation model to perform area segmentation on the first sample image to obtain at least one candidate area segmentation result, the at least one second target segmentation model is trained based on the second sample image and the standard area division image corresponding to the second sample image, and the standard area division image corresponding to the second sample image is obtained based on the second standard area ratio corresponding to the first object in the second sample image; the candidate area segmentation result in at least one candidate area segmentation result that meets the first matching condition with the first standard area ratio is used as the standard area segmentation result corresponding to the first sample image.

[0373] Before calling at least one second target segmentation model to perform region segmentation on the first sample image and obtain at least one candidate region segmentation result, it is necessary to first train at least one second target segmentation model. In one possible implementation, the at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models; the process of training the at least one second target segmentation model is as follows: based on the second standard area ratio, obtaining a standard region division image corresponding to the second sample image; based on the second sample image and the standard region division image, respectively training the reference number of second initial segmentation models, and obtaining at least one second target segmentation model corresponding to the reference number of second initial segmentation models obtained through different rounds of training.

[0374] In one possible implementation, based on the second standard area ratio, a method for obtaining a standard area division image corresponding to the second sample image includes the following steps 1-1 to 1-4:

[0375] Step 1-1: Obtain a threshold range corresponding to a reference color channel and a sample channel image corresponding to the second sample image under the reference color channel.

[0376] In one possible implementation, a method for obtaining a threshold interval corresponding to a reference color channel is as follows: obtaining a first channel image corresponding to a first reference image under the reference color channel and a second channel image corresponding to a second reference image under the reference color channel, the first reference image does not include the first object, and the second reference image includes the first object; obtaining a first pixel value distribution of the first channel image; obtaining a second pixel value distribution of the second channel image; and determining the threshold interval corresponding to the reference color channel based on the first pixel value distribution and the second pixel value distribution.

[0377] Step 1-2: Use the threshold interval to binarize the sample channel image to obtain a binary image.

[0378] Step 1-3: Perform morphological processing on the binary image to obtain at least one candidate region segmentation image.

[0379] In a possible implementation, after obtaining the binarized image, the predicted area ratio corresponding to the first object in the second sample image is determined based on the binarized image.

[0380] In one possible implementation, morphological processing is performed on the binarized image to obtain at least one candidate region division image as follows: in response to the predicted region ratio being less than the second standard region ratio, the binarized image is dilated using a reference dilation kernel to obtain at least one candidate region division image; in response to the predicted region ratio being greater than the second standard region ratio, the binarized image is reduced using a reference reduction kernel to obtain at least one candidate region division image.

[0381] Step 1-4: taking a candidate region division image in at least one candidate region division image that satisfies a second matching condition with the second standard region ratio as a standard region division image corresponding to the second sample image.

[0382] The above method of obtaining the first standard area ratio corresponding to the first object in the first sample image; based on the first standard area ratio, obtaining the standard area segmentation result corresponding to the first sample image can be implemented by referring to Figure 4 Step 401 and step 402 in the illustrated embodiment will not be described in detail here.

[0383] In step 1502 , attribute transformation is performed on the first sample image so that the first sample image after attribute transformation has target visual attributes.

[0384] Performing attribute transformation on the first sample image can make the first sample image after attribute transformation have the target visual attributes, thereby reducing the interference of the visual attributes of the first sample image itself on the process of training the first target segmentation model and improving the training effect of the first target segmentation model. Figure 14 The principle of step 1402 in the illustrated embodiment is the same and will not be repeated here.

[0385] In step 1503, a first initial segmentation model is trained based on the first sample image after attribute transformation and the standard region segmentation result to obtain a first target segmentation model.

[0386] After obtaining the attribute-transformed first sample image, the first initial segmentation model is trained based on the attribute-transformed first sample image and the standard region segmentation result. The attribute-transformed first sample image is an image having the target visual attributes. Training the first initial segmentation model using sample images having the same visual attributes helps reduce interference caused by the visual attributes of the sample images themselves on the training process, allowing the training process to converge more quickly and improving the effectiveness of model training.

[0387] Exemplarily, the process of training the first initial segmentation model based on the attribute-transformed first sample image and the standard region segmentation result to obtain the first target segmentation model is as follows: calling the first initial segmentation model to perform region segmentation on the attribute-transformed first sample image to obtain a predicted region segmentation result; obtaining a loss function based on the predicted region segmentation result and the standard region segmentation result; and using the loss function to train the first initial segmentation model to obtain the first target segmentation model. This training process is a supervised training process, and the standard region segmentation result is used to provide a supervisory signal for the supervised training process.

[0388] After obtaining the trained first target segmentation model, the first target segmentation model has the function of more accurately segmenting the area where the first object is located in the image with target visual attributes, so that the first target segmentation model can be called to perform regional segmentation on the image with target visual attributes including the first object, thereby obtaining the regional segmentation result corresponding to the image, and then obtaining the measurement index corresponding to the image based on the regional segmentation result corresponding to the image.

[0389] Exemplarily, after the first target segmentation model is obtained through training, the method further includes: obtaining an image to be processed, the image to be processed including a first object; performing attribute transformation on the image to be processed so that the image to be processed after the attribute transformation has target visual attributes; calling the first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation, and obtaining a first region segmentation result corresponding to the image to be processed, the first region segmentation result being used to indicate the region where the first object in the image to be processed is located; obtaining a measurement index corresponding to the image to be processed based on the first region segmentation result, the measurement index being used to indicate the region proportion corresponding to the first object in the image to be processed that meets the region restriction condition. The implementation method of this process is detailed in Figure 14 The embodiments shown are not described in detail here.

[0390] In an embodiment of the present application, the first target segmentation model is trained based on the first sample image after attribute transformation and the standard region segmentation result. The first sample image after attribute transformation has target visual attributes, which is beneficial to reducing the interference caused by the visual attributes of the first sample image itself to the process of training the first target segmentation model, so that the trained first target segmentation model has better region segmentation function, so that the first target segmentation model can be called to realize the process of automatically obtaining the measurement indicators corresponding to the image to be processed, which is beneficial to improving the stability and efficiency of image processing and the accuracy of the measurement indicators corresponding to the image to be processed.

[0391] Next, an exemplary application of the embodiment of the present application in a practical application scenario is introduced.

[0392] In an exemplary embodiment, the image processing method provided in the embodiment of the present application can be applied to an application scenario of determining the IC value (immune cell ratio) corresponding to a tumor image. In this application scenario, the image to be processed is a tumor image, which is an image obtained by capturing an image of the tumor area field in a pathology slide obtained by staining based on the PD-L1 method. The actual physical meaning of the total area of ​​the tumor image refers to the tumor area, and the IC value corresponding to the tumor image refers to the ratio of stained IC cells in the tumor area in the tumor image. The IC value corresponding to the tumor image can be regarded as an expression of the PD-L1 method.

[0393] Currently, immunohistochemical assessment of PD-1 / PD-L immune responses has become a research focus and hotspot. PD-L1 is associated with tumor progression and poor prognosis, and is considered an effective biomarker for predicting prognosis. For example, breast cancer is the most common malignancy in women, and precision medicine offers opportunities for more refined and personalized treatment of breast cancer. A humanized monoclonal antibody (atezolizumab) combined with a protein-binding paclitaxel is used to treat unresectable, locally advanced, or metastatic triple-negative breast cancer (TNBC), with PD-L1 (SP142) as a companion diagnostic. In breast cancer treatment, the PD-L1 assay can stain immune cells in the tumor area, thereby determining the percentage of these cells within the tumor (PD-L1 expression). This assay helps physicians assess a patient's ability to defend against cancer. Specifically, physicians estimate the percentage of stained immune cells (ICs) within the tumor area to inform the selection of appropriate cancer treatments.

[0394] In this application scenario, the first object is a stained cell, and the first object that meets the region restriction condition is a stained IC cell (stained immune cell). The measurement indicator is the IC value corresponding to the tumor image. A stained IC cell is a stained cell whose area is not within the cancer cell area. Figure 16 Based on the image processing method provided in the embodiment of the present application, the process of determining the IC value corresponding to the tumor image includes the following steps 1601 to 1606:

[0395] In step 1601, a tumor image is obtained by capturing an image of a tumor region field in a pathology slide obtained by staining based on the PD-L1 method, where the tumor image includes stained cells.

[0396] In step 1602, attribute transformation is performed on the tumor image so that the transformed tumor image has target visual attributes.

[0397] In step 1603, the first target segmentation model is called to perform region segmentation on the tumor image after attribute transformation to obtain a first region segmentation result corresponding to the tumor image. The first region segmentation result is used to indicate the region where the stained cells in the tumor image are located.

[0398] The first target segmentation model is trained based on the first sample tumor image after attribute transformation and the standard region segmentation result corresponding to the first sample tumor image. The standard region segmentation result is obtained based on the first standard region ratio corresponding to the stained cells in the first sample tumor image. The first standard region ratio corresponding to the stained cells in the first sample tumor image is used to indicate the proportion of the region where the stained cells in the first sample tumor image are located in the first sample tumor image.

[0399] In step 1604, the target region detection model is called to perform region detection on the tumor image after attribute transformation to obtain the target region in the tumor image.

[0400] The target region is the cancer cell region or the cancer cell interstitial region. The cancer cell region and the cancer cell interstitial region are mutually exclusive regions. In the total area of ​​the tumor image, the area other than the cancer cell region is the cancer cell interstitial region.

[0401] For example, taking the target region as a cancer cell region, the target region detection model is a Link Net model. Since the cancer cell region is large, the cancer cell region is manually labeled during the training process of the target region detection model. However, due to the large number and small size of stained cells, it is difficult to manually label the region where the stained cells are located. Therefore, the standard region segmentation result during the training process of the first target segmentation model is obtained based on the manually labeled text annotation (i.e., the proportion of the standard region corresponding to the stained cells).

[0402] In step 1605, the region where the stained cells are located in the tumor image is filtered based on the target region in the tumor image to obtain the region where the stained IC cells are located in the tumor image.

[0403] If the target area is a cancer cell area, the cancer cell area is used to filter the areas where the stained cells in the tumor image are located, and the areas where the stained cells outside the cancer cell area are located are retained as the areas where the stained IC cells in the tumor image are located.

[0404] If the target area is the cancer cell interstitial area, the cancer cell interstitial area is used to filter the area where each stained cell in the tumor image is located in the tumor image, and the area where the stained cells in the cancer cell interstitial area are located is retained as the area where the stained IC cells in the tumor image are located.

[0405] Because the stained cells in the cancer cell area cannot be counted as stained IC cells, the cancer cell area or cancer cell interstitial area is used to filter out the area where invalid stained cells are located, and finally the area where stained IC cells are located in the tumor image is obtained.

[0406] In step 1606 , the ratio of the number of pixels located in the region where the stained IC cells are located to the number of pixels located in the total region of the tumor image is used as the IC value corresponding to the tumor image.

[0407] Exemplarily, the calculation formula of IC value is as follows:

[0408]

[0409] For example, the number of pixels within the total area of ​​a tumor image refers to the total number of pixels in the tumor image. The total area of ​​the tumor image is considered the tumor area, and the IC value calculated in this manner represents the proportion of stained IC cells within the tumor area.

[0410] For example, the process of determining the IC value corresponding to the tumor image is as follows: Figure 17 As shown. Attribute transformation is performed on the tumor image to obtain an attribute-transformed tumor image 1701. The first target segmentation model 1702 is called to perform region segmentation on the attribute-transformed tumor image 1701, obtaining a first region segmentation result 1703 represented in image form. In this first region segmentation result 1703, the region where the stained cells are located is represented by white, and other regions are represented by black. The first target segmentation model 1702 is trained based on the sample tumor image and the standard region segmentation result 1704 corresponding to the sample tumor image. The target region detection model 1705 is called to perform region detection on the attribute-transformed tumor image 1701, obtaining a target region detection result 1706 represented in image form. In this target region detection result 1706, the cancer cell region is represented by white, and the cancer cell interstitial region is represented by black.

[0411] Based on the target region detection result 1706, the region containing the stained cells (which can be considered stained cancer cells) within the cancer cell region, as indicated by the first region segmentation result 1703, is eliminated, resulting in an image 1707 indicating the region containing the stained IC cells. In this image 1707, the region containing the stained IC cells is represented by white, and other regions are represented by black. The ratio of the number of pixels in the region containing the stained IC cells in image 1707 to the total number of pixels in image 1707 is used as the IC value corresponding to the tumor image. For example, the IC value corresponding to the tumor image is annotated on image 1707, and image 1707 annotated with the IC value is presented to the doctor for review.

[0412] Illustratively, the method provided in the embodiment of the present application can be applied to tumor images acquired by a variety of image acquisition devices, that is, the IC values ​​corresponding to tumor images acquired by a variety of image acquisition devices can be obtained more accurately. The prediction performance of the image processing method provided in the embodiment of the present application on two different image acquisition devices was tested. Illustratively, the two different image acquisition devices are the domestic Unic scanner (Unic digital scanner precision600Series, Unic Technologies, INC.) and the Japanese Hamamatsu scanner (Nano Zoomer S210 Digitalslide scanner C13239-01, Hamamatsu Photonics KK). The domestic Unic scanner is commonly used in domestic hospitals, and the Japanese Hamamatsu scanner has a high usage rate internationally and is often used in international hospitals and professional institutions. Compared with the domestic Unic scanner, the image captured by the Hamamatsu scanner has high sharpness, accurate focus, rich colors, and is dark. During the test, the area where the stained cells are segmented by the first target segmentation model and the cancer cell area detected by the target area detection model are marked with different colors on the image after attribute transformation for easy comparison and viewing. The test results are as follows: Figure 18 shown.

[0413] exist Figure 18 middle, Figure 18 (1)(3)(5) in the figure are the test results of tumor images collected by the domestic Una scanner. Figure 18 (2)(4)(6) are the test results obtained by testing tumor images collected by Hamamatsu scanner in Japan. In each test result, the area where the stained cells are segmented by the first target segmentation model and the cancer cell area detected by the target area detection model are marked with different colors on the tumor image after attribute transformation. Figure 18(1) and (2) are the test results of tumor images collected by using the domestic Una scanner and the Japanese Hamamatsu scanner for the same tumor field. Figure 18 The standard IC value corresponding to (1) and (2) is 5%; Figure 18 (3) and (4) are the test results of tumor images collected by using the domestic Una scanner and the Japanese Hamamatsu scanner for the same tumor field. Figure 18 The standard IC value corresponding to (3) and (4) is 2%; Figure 18 (5) and (6) are the test results of tumor images collected by using the domestic Una scanner and the Japanese Hamamatsu scanner for the same tumor field. Figure 18 The standard IC values ​​corresponding to (5) and (6) are 0%.

[0414] based on Figure 18 The IC value determined by the test results shown in (1) is 6.0%, based on Figure 18 The IC value determined by the test results shown in (2) is 6.1%; based on Figure 18 The IC value determined by the test results shown in (3) is 2.1%, based on Figure 18 The IC value determined by the test results shown in (4) is 2.7%; based on Figure 18 The IC value determined by the test results shown in (5) is 0.1%, based on Figure 18 The IC value determined by the test results shown in (6) is 0.3%. Based on the test results, it can be obtained that the average error of IC value prediction for tumor images acquired by two different scanners is 0.5%, and the consistency of the prediction results ICC (Intraclass Correlation Coefficient) is 0.98. This shows that the image processing method provided by the embodiment of the present application can be applied to a variety of image acquisition devices. This is mainly because after the attribute transformation of the image, the overall visual experience is relatively close, which can reduce the adverse effects of the visual attributes of the image acquisition device itself on the model.

[0415] Based on the image processing method provided in the embodiment of the present application, the IC value test was performed on the tumor image collected by the microscope, and the test results included the IC value and the regional display result. Figure 19 As shown, Figure 19 (1) is the test result obtained by processing the tumor image after attribute transformation. Figure 19(2) is the test result obtained by directly processing the tumor image captured by the microscope. The test result is consistent with visual perception and has also been recognized by the cooperating doctors. The regional display results obtained by processing the tumor image after attribute transformation show clearer color segmentation. For example, in the regional display results, different colors are used to mark the areas where stained cells are located and the areas where cancer cells are located to facilitate differentiation.

[0416] Based on the image processing method provided in the embodiment of the present application, a universal interpretation method can be implemented under a variety of image acquisition devices (such as different WSI scanners, microscopes, etc.) for the IC value interpretation of breast pathology PD-L1. The interpretation error of the same field of view of different WSI scanners is within 0.5%, and the consistency of the interpretation results ICC = 0.98, showing the high robustness of the IC value interpretation method. The important factor of this highly universal interpretation method is to convert text annotations into pixel-level annotations and perform attribute transformation on the image, so that a segmentation model with high segmentation accuracy can be trained.

[0417] See also Figure 20 , an embodiment of the present application provides an image processing device, the device comprising:

[0418] A first acquiring unit 2001 is configured to acquire an image to be processed, where the image to be processed includes a first object;

[0419] Segmentation unit 2002 is configured to call a first target segmentation model to perform region segmentation on the image to be processed, and obtain a first region segmentation result corresponding to the image to be processed, where the first region segmentation result is used to indicate a region where a first object in the image to be processed is located. The first target segmentation model is trained based on a first sample image and a standard region segmentation result corresponding to the first sample image, where the standard region segmentation result is obtained based on a first standard region ratio corresponding to the first object in the first sample image.

[0420] The second acquisition unit 2003 is used to acquire a measurement index corresponding to the image to be processed based on the first region segmentation result, where the measurement index is used to indicate the area ratio corresponding to the first object that meets the region restriction condition in the image to be processed.

[0421] In one possible implementation, the second acquisition unit 2003 is used to determine a target area in the image to be processed; use the target area to filter the area where the first object in the image to be processed is located to obtain the area where the first object that meets the area restriction condition is located in the image to be processed; compare the area where the first object that meets the area restriction condition is located with the total area of ​​the image to be processed to obtain a measurement indicator corresponding to the image to be processed.

[0422] In one possible implementation, the segmentation unit 2002 is used to perform attribute transformation on the image to be processed so that the image to be processed after the attribute transformation has the target visual attributes; call the first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation to obtain a first region segmentation result corresponding to the image to be processed.

[0423] In one possible implementation, the image to be processed corresponds to at least one color channel, and the segmentation unit 2002 is further used to determine the pixel mean corresponding to each color channel based on the pixel value of each pixel point in the image to be processed under each color channel; use the reference numerical value corresponding to each color channel to perform numerical transformation on the pixel mean corresponding to each color channel to obtain the pixel value transformation benchmark corresponding to each color channel, and the reference numerical value corresponding to each color channel is determined based on the target visual attribute; based on the pixel value transformation benchmark corresponding to each color channel, the pixel value of each pixel point in the image to be processed under each color channel is transformed.

[0424] In a possible implementation, the segmentation unit 2002 is further configured to obtain an image transformation model, which is configured to output an image with target visual attributes; and to call the image transformation model to perform attribute transformation on the image to be processed.

[0425] In the embodiments of the present application, calling the first target segmentation model implements the process of automatically obtaining the corresponding measurement indicators for the image to be processed. This image processing process does not require human intervention, and the image processing is relatively stable and efficient. In addition, the standard region segmentation result based on which the first target segmentation model is trained is obtained based on the standard region proportion corresponding to the first object, and has high accuracy. This allows the first target segmentation model to be trained with better region segmentation capabilities, and the measurement indicators obtained based on the first region segmentation result segmented by the first target segmentation model are highly accurate.

[0426] See also Figure 21 , an embodiment of the present application provides a model training device, the device comprising:

[0427] A first acquiring unit 2101 is configured to acquire a first sample image and a first standard area ratio corresponding to a first object in the first sample image;

[0428] A second acquiring unit 2102 is configured to acquire a standard region segmentation result corresponding to the first sample image based on the first standard region proportion, where the standard region segmentation result is used to indicate a region where the first object in the first sample image is located;

[0429] The training unit 2103 is configured to train the first initial segmentation model based on the first sample image and the standard region segmentation result to obtain a first target segmentation model.

[0430] In one possible implementation, the second acquisition unit 2102 is used to call at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result. The at least one second target segmentation model is trained based on the second sample image and the standard region division image corresponding to the second sample image. The standard region division image corresponding to the second sample image is obtained based on the second standard region proportion corresponding to the first object in the second sample image; the candidate region segmentation result in the at least one candidate region segmentation result that meets the first matching condition with the first standard region proportion is used as the standard region segmentation result corresponding to the first sample image.

[0431] In one possible implementation, the at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models; see Figure 22 , the device further comprises:

[0432] The third acquiring unit 2104 is configured to acquire a standard region division image corresponding to the second sample image based on the second standard region ratio;

[0433] The training unit 2103 is also used to train a reference number of second initial segmentation models based on the second sample image and the standard area division image, and obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models through different rounds of training.

[0434] In one possible implementation, the third acquisition unit 2104 is used to obtain a threshold interval corresponding to a reference color channel, and a sample channel image corresponding to the second sample image under the reference color channel; binarize the sample channel image using the threshold interval to obtain a binarized image; perform morphological processing on the binarized image to obtain at least one candidate area division image; and use the candidate area division image in the at least one candidate area division image that satisfies a second matching condition with the second standard area ratio as the standard area division image corresponding to the second sample image.

[0435] In one possible implementation, the third acquisition unit 2104 is further used to obtain a first channel image corresponding to the first reference image under the reference color channel and a second channel image corresponding to the second reference image under the reference color channel, where the first reference image does not include the first object and the second reference image includes the first object; obtain a first pixel value distribution of the first channel image; obtain a second pixel value distribution of the second channel image; and determine a threshold interval corresponding to the reference color channel based on the first pixel value distribution and the second pixel value distribution.

[0436] In one possible implementation, the third acquisition unit 2104 is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being less than the second standard area ratio, the binarized image is expanded using a reference expansion kernel to obtain at least one candidate area division image.

[0437] In one possible implementation, the third acquisition unit 2104 is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being greater than the second standard area ratio, the binarized image is reduced using a reference reduction kernel to obtain at least one candidate area division image.

[0438] In one possible implementation, the training unit 2103 is used to perform attribute transformation on the first sample image so that the first sample image after attribute transformation has target visual attributes; and to train the first initial segmentation model based on the first sample image after attribute transformation and the standard region segmentation result to obtain a first target segmentation model.

[0439] In an embodiment of the present application, a standard area segmentation result is obtained based on the standard area ratio, and the accuracy of the standard area segmentation result is high. The first target segmentation model trained according to the standard area segmentation result has a good area segmentation function, so that the first target segmentation model can be called to realize the process of automatically obtaining the measurement indicators corresponding to the image to be processed, which is conducive to improving the stability and efficiency of image processing and the accuracy of the measurement indicators corresponding to the image to be processed.

[0440] See also Figure 23 , an embodiment of the present application provides an image processing device, the device comprising:

[0441] A first acquiring unit 2301 is configured to acquire an image to be processed, where the image to be processed includes a first object;

[0442] The transformation unit 2302 is used to perform attribute transformation on the image to be processed so that the image to be processed after attribute transformation has target visual attributes;

[0443] The segmentation unit 2303 is configured to call the first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation, and obtain a first region segmentation result corresponding to the image to be processed, where the first region segmentation result is used to indicate a region where the first object in the image to be processed is located.

[0444] The second acquisition unit 2304 is used to acquire a measurement index corresponding to the image to be processed based on the first region segmentation result, where the measurement index is used to indicate the area ratio corresponding to the first object that meets the region restriction condition in the image to be processed.

[0445] In one possible implementation, the image to be processed corresponds to at least one color channel, and the transformation unit 2302 is used to determine the pixel mean corresponding to each color channel based on the pixel value of each pixel point in the image to be processed under each color channel; use the reference numerical value corresponding to each color channel to perform numerical transformation on the pixel mean corresponding to each color channel to obtain the pixel value transformation benchmark corresponding to each color channel, and the reference numerical value corresponding to each color channel is determined based on the target visual attribute; based on the pixel value transformation benchmark corresponding to each color channel, the pixel value of each pixel point in the image to be processed under each color channel is transformed.

[0446] In a possible implementation, the transformation unit 2302 is configured to obtain an image transformation model, which is configured to output an image with target visual attributes; and to call the image transformation model to perform attribute transformation on the image to be processed.

[0447] In one possible implementation, the second acquisition unit 2304 is used to determine a target area in the image to be processed; use the target area to filter the area where the first object in the image to be processed is located to obtain the area where the first object that meets the area restriction condition is located in the image to be processed; compare the area where the first object that meets the area restriction condition is located with the total area of ​​the image to be processed to obtain a measurement indicator corresponding to the image to be processed.

[0448] In the embodiments of the present application, calling the first target segmentation model implements the process of automatically obtaining the measurement indicators corresponding to the image to be processed. This image processing process does not require human intervention, and the image processing is stable and efficient. In addition, before calling the first target segmentation model to perform region segmentation, the image to be processed is first transformed into an image with the target visual attributes, thereby reducing the interference caused by the visual attributes of the image to be processed itself on the first target segmentation model. This makes the first region segmentation result obtained by calling the first target segmentation model more accurate, thereby improving the accuracy of the obtained measurement indicators.

[0449] See also Figure 24 , an embodiment of the present application provides a model training device, the device comprising:

[0450] A first acquiring unit 2401 is configured to acquire a first sample image and a standard region segmentation result corresponding to the first sample image, where the standard region segmentation result indicates a region where a first object in the first sample image is located.

[0451] The transformation unit 2402 is configured to perform attribute transformation on the first sample image so that the first sample image after attribute transformation has target visual attributes;

[0452] The training unit 2403 is configured to train the first initial segmentation model based on the first sample image after attribute transformation and the standard region segmentation result to obtain a first target segmentation model.

[0453] In a possible implementation, the first acquisition unit 2401 is configured to acquire a first standard area ratio corresponding to a first object in a first sample image; and acquire a standard area segmentation result corresponding to the first sample image based on the first standard area ratio.

[0454] In one possible implementation, the first acquisition unit 2401 is also used to call at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result. The at least one second target segmentation model is trained based on the second sample image and the standard region division image corresponding to the second sample image. The standard region division image corresponding to the second sample image is obtained based on the second standard region proportion corresponding to the first object in the second sample image; the candidate region segmentation result in the at least one candidate region segmentation result that meets the first matching condition with the first standard region proportion is used as the standard region segmentation result corresponding to the first sample image.

[0455] In one possible implementation, the at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models; see Figure 25 , the device further comprises:

[0456] The second acquiring unit 2404 is configured to acquire a standard region division image corresponding to the second sample image based on the second standard region ratio;

[0457] The training unit 2403 is also used to train a reference number of second initial segmentation models based on the second sample image and the standard area division image, and obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models through different rounds of training.

[0458] In one possible implementation, the second acquisition unit 2404 is used to obtain a threshold interval corresponding to a reference color channel, and a sample channel image corresponding to the second sample image under the reference color channel; binarize the sample channel image using the threshold interval to obtain a binarized image; perform morphological processing on the binarized image to obtain at least one candidate area division image; and use the candidate area division image in the at least one candidate area division image that satisfies a second matching condition with the second standard area ratio as the standard area division image corresponding to the second sample image.

[0459] In one possible implementation, the second acquisition unit 2404 is further used to obtain a first channel image corresponding to the first reference image under the reference color channel and a second channel image corresponding to the second reference image under the reference color channel, where the first reference image does not include the first object and the second reference image includes the first object; obtain a first pixel value distribution of the first channel image; obtain a second pixel value distribution of the second channel image; and determine a threshold interval corresponding to the reference color channel based on the first pixel value distribution and the second pixel value distribution.

[0460] In one possible implementation, the second acquisition unit 2404 is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being less than the second standard area ratio, the binarized image is expanded using a reference expansion kernel to obtain at least one candidate area division image.

[0461] In one possible implementation, the second acquisition unit 2404 is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being greater than the second standard area ratio, the binarized image is reduced using a reference reduction kernel to obtain at least one candidate area division image.

[0462] In an embodiment of the present application, the first target segmentation model is trained based on the first sample image after attribute transformation and the standard region segmentation result. The first sample image after attribute transformation has target visual attributes, which is beneficial to reducing the interference caused by the visual attributes of the first sample image itself to the process of training the first target segmentation model, so that the trained first target segmentation model has better region segmentation function, so that the first target segmentation model can be called to realize the process of automatically obtaining the measurement indicators corresponding to the image to be processed, which is beneficial to improving the stability and efficiency of image processing and the accuracy of the measurement indicators corresponding to the image to be processed.

[0463] It should be noted that the apparatus provided in the above embodiments is merely illustrated by the division of the above functional units when implementing its functions. In actual applications, the above functions can be distributed and completed by different functional units as needed, that is, the internal structure of the apparatus can be divided into different functional units to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0464] In an exemplary embodiment, a computer device is also provided. The computer device includes a processor and a memory, wherein the memory stores at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the aforementioned image processing methods or any of the aforementioned model training methods. The computer device can be a terminal or a server. The following describes the structures of the terminal and the server, respectively.

[0465] Figure 26 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may be a smartphone, tablet computer, laptop computer, or desktop computer. The terminal may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0466] Typically, the terminal includes: a processor 2601 and a memory 2602 .

[0467] The processor 2601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 2601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 2601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 2601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 2601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0468] The memory 2602 may include one or more computer-readable storage media, which may be non-transitory. The memory 2602 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 2602 is used to store at least one instruction, which is used to be executed by the processor 2601 so that the terminal implements the image processing method or model training method provided in the method embodiment of the present application.

[0469] In some embodiments, the terminal may optionally include a peripheral device interface 2603 and at least one peripheral device. The processor 2601, memory 2602, and peripheral device interface 2603 may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 2603 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 2604, a display screen 2605, a camera assembly 2606, an audio circuit 2607, and a power supply 2609.

[0470] The peripheral device interface 2603 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 2601 and the memory 2602. The radio frequency circuit 2604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 2604 communicates with the communication network and other communication devices via electromagnetic signals. The display screen 2605 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. The camera assembly 2606 is used to capture images or videos. Optionally, the camera assembly 2606 includes a front camera and a rear camera.

[0471] Audio circuit 2607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting them into electrical signals that are then input into processor 2601 for processing or into RF circuit 2604 for voice communication. The speaker is used to convert electrical signals from processor 2601 or RF circuit 2604 into sound waves. Power supply 2609 is used to power various components in the terminal. Power supply 2609 can be AC, DC, a disposable battery, or a rechargeable battery.

[0472] In some embodiments, the terminal further includes one or more sensors 2610 , including but not limited to: an acceleration sensor 2611 , a gyroscope sensor 2612 , a pressure sensor 2613 , an optical sensor 2615 , and a proximity sensor 2616 .

[0473] The acceleration sensor 2611 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. The gyroscope sensor 2612 can detect the body direction and rotation angle of the terminal. The gyroscope sensor 2612 can cooperate with the acceleration sensor 2611 to collect the user's 3D actions on the terminal. The pressure sensor 2613 can be set on the side frame of the terminal and / or the lower layer of the display screen 2605. When the pressure sensor 2613 is set on the side frame of the terminal, it can detect the user's holding signal of the terminal, and the processor 2601 performs left and right hand recognition or quick operation based on the holding signal collected by the pressure sensor 2613. When the pressure sensor 2613 is set on the lower layer of the display screen 2605, the processor 2601 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 2605.

[0474] The optical sensor 2615 is used to collect ambient light intensity. The proximity sensor 2616, also known as a distance sensor, is typically located on the front panel of the terminal. The proximity sensor 2616 is used to collect the distance between the user and the front of the terminal.

[0475] Those skilled in the art will understand that Figure 26 The structure shown in the figure does not constitute a limitation on the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0476] Figure 27 This is a structural diagram of a server provided in an embodiment of the present application. The server may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 2701 and one or more memories 2702, wherein at least one computer program is stored in the one or more memories 2702, and the at least one computer program is loaded and executed by the one or more processors 2701 to enable the server to implement the image processing method or model training method provided in the above-mentioned various method embodiments. Of course, the server may also include other components for implementing device functions, which will not be described in detail here.

[0477] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-mentioned image processing methods or any of the above-mentioned model training methods.

[0478] In one possible implementation, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0479] In an exemplary embodiment, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the above-described image processing methods or any of the above-described model training methods.

[0480] It should be noted that the terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementations described in the above exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application as detailed in the appended claims.

[0481] It should be understood that the term "plurality" used herein refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0482] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Obtaining a first sample image and a first standard area ratio corresponding to a first object in the first sample image, wherein the first sample image corresponds to at least one color channel; Based on the first standard area proportion, obtaining a standard area segmentation result corresponding to the first sample image, where the standard area segmentation result is used to indicate the area where the first object is located in the first sample image; Determining, based on the pixel values ​​of each pixel point in each color channel in the first sample image, the pixel mean corresponding to each color channel in the first sample image; Obtaining reference values ​​corresponding to the respective color channels determined based on a target visual attribute, wherein the target visual attribute is composed of at least one sub-attribute of sharpness, focus, color richness, and brightness; Using the reference values ​​corresponding to the respective color channels, numerically transform the pixel mean values ​​corresponding to the respective color channels in the first sample image to obtain pixel value transformation benchmarks corresponding to the respective color channels in the first sample image; Based on the pixel value transformation benchmarks corresponding to the respective color channels in the first sample image, the pixel values ​​of the respective pixels in the first sample image under the respective color channels are transformed to obtain a first sample image after attribute transformation, wherein the first sample image after attribute transformation has the target visual attributes; A first initial segmentation model is trained based on the first sample image after attribute transformation and the standard region segmentation result to obtain a first target segmentation model.

2. The method according to claim 1, characterized in that The obtaining, based on the first standard area proportion, a standard area segmentation result corresponding to the first sample image includes: calling at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result, wherein the at least one second target segmentation model is trained based on the second sample image and a standard region division image corresponding to the second sample image, and the standard region division image corresponding to the second sample image is obtained based on a second standard region ratio corresponding to the first object in the second sample image; The candidate region segmentation result that satisfies a first matching condition with the first standard region proportion in the at least one candidate region segmentation result is used as the standard region segmentation result corresponding to the first sample image.

3. The method according to claim 2, characterized in that The at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models respectively; Before calling at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result, the method further includes: Based on the second standard area ratio, obtaining a standard area division image corresponding to the second sample image; Based on the second sample image and the standard region division image, the reference number of second initial segmentation models are trained respectively to obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models and obtained through different rounds of training.

4. The method according to claim 3, characterized in that The acquiring, based on the second standard area ratio, a standard area division image corresponding to the second sample image includes: Obtaining a threshold interval corresponding to a reference color channel and a sample channel image corresponding to the second sample image under the reference color channel; Performing binarization processing on the sample channel image using the threshold interval to obtain a binarized image; Performing morphological processing on the binary image to obtain at least one candidate region division image; The candidate area division image that satisfies a second matching condition with the second standard area ratio in the at least one candidate area division image is used as the standard area division image corresponding to the second sample image.

5. The method according to claim 4, characterized in that The obtaining of the threshold value interval corresponding to the reference color channel includes: Acquire a first channel image corresponding to a first reference image in the reference color channel and a second channel image corresponding to a second reference image in the reference color channel, wherein the first reference image does not include the first object, and the second reference image includes the first object; Obtaining a first pixel value distribution of the first channel image; Obtaining a second pixel value distribution of the second channel image; A threshold interval corresponding to the reference color channel is determined based on the first pixel value distribution and the second pixel value distribution.

6. The method according to claim 4 or 5, characterized in that The performing morphological processing on the binary image to obtain at least one candidate region division image includes: Determining, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; In response to the predicted area ratio being smaller than the second standard area ratio, the binary image is dilated using a reference dilation kernel to obtain the at least one candidate area division image.

7. The method according to claim 4 or 5, characterized in that The performing morphological processing on the binary image to obtain at least one candidate region division image includes: Determining, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; In response to the predicted area ratio being greater than the second standard area ratio, the binarized image is reduced using a reference reduction kernel to obtain the at least one candidate area division image.

8. An image processing method, characterized in that: The method comprises: Acquire an image to be processed, where the image to be processed includes a first object; Performing attribute transformation on the image to be processed so that the image to be processed after the attribute transformation has target visual attributes; Calling a first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation, to obtain a first region segmentation result corresponding to the image to be processed, wherein the first region segmentation result is used to indicate a region where the first object in the image to be processed is located, and the first target segmentation model is trained according to any one of claims 1 to 7; Based on the first region segmentation result, a measurement index corresponding to the image to be processed is obtained, where the measurement index is used to indicate a proportion of an area corresponding to a first object that meets a region restriction condition in the image to be processed.

9. The method according to claim 8, characterized in that The image to be processed corresponds to at least one color channel, and performing attribute transformation on the image to be processed includes: Determining the pixel mean corresponding to each color channel in the image to be processed based on the pixel value of each pixel point in each color channel in the image to be processed; Using the reference values ​​corresponding to the respective color channels, numerically transforming the pixel mean values ​​corresponding to the respective color channels in the image to be processed to obtain pixel value transformation benchmarks corresponding to the respective color channels in the image to be processed, wherein the reference values ​​corresponding to the respective color channels are determined based on the target visual attributes; Based on the pixel value transformation benchmarks corresponding to the respective color channels in the image to be processed, the pixel values ​​of the respective pixel points in the image to be processed under the respective color channels are transformed.

10. The method according to claim 8, characterized in that The performing attribute transformation on the image to be processed includes: Acquire an image transformation model, wherein the image transformation model is used to output an image having the target visual attribute; The image transformation model is called to perform attribute transformation on the image to be processed.

11. A model training device, characterized in that: The device comprises: A first acquiring unit is configured to acquire a first sample image and a first standard area ratio corresponding to a first object in the first sample image, wherein the first sample image corresponds to at least one color channel; a second acquiring unit, configured to acquire a standard region segmentation result corresponding to the first sample image based on the first standard region proportion, wherein the standard region segmentation result is used to indicate a region in the first sample image where the first object is located; A training unit is used to determine the pixel mean corresponding to each color channel in the first sample image based on the pixel value of each pixel point in each color channel in the first sample image; obtain the reference numerical value corresponding to each color channel determined based on the target visual attribute, where the target visual attribute is composed of at least one sub-attribute of sharpness, focus, color saturation, and brightness; use the reference numerical value corresponding to each color channel to perform a numerical transformation on the pixel mean corresponding to each color channel in the first sample image to obtain the pixel value transformation benchmark corresponding to each color channel in the first sample image; based on the pixel value transformation benchmark corresponding to each color channel in the first sample image, transform the pixel value of each pixel point in the first sample image under each color channel to obtain a first sample image after attribute transformation, and the first sample image after attribute transformation has the target visual attribute; train a first initial segmentation model based on the first sample image after attribute transformation and the standard area segmentation result to obtain a first target segmentation model.

12. The device according to claim 11, characterized in that The second acquisition unit is configured to call at least one second target segmentation model to perform region segmentation on the first sample image to obtain at least one candidate region segmentation result, wherein the at least one second target segmentation model is trained based on the second sample image and a standard region division image corresponding to the second sample image, and the standard region division image corresponding to the second sample image is obtained based on a second standard region ratio corresponding to the first object in the second sample image; The candidate region segmentation result that satisfies a first matching condition with the first standard region proportion in the at least one candidate region segmentation result is used as the standard region segmentation result corresponding to the first sample image.

13. The device according to claim 12, characterized in that The at least one second target segmentation model includes at least one second target segmentation model corresponding to a reference number of second initial segmentation models respectively; the device further includes: a third acquiring unit, configured to acquire a standard area division image corresponding to the second sample image based on the second standard area ratio; The training unit is also used to train the reference number of second initial segmentation models based on the second sample image and the standard area division image, and obtain at least one second target segmentation model corresponding to the reference number of second initial segmentation models through different rounds of training.

14. The device according to claim 13, characterized in that The third acquisition unit is configured to acquire a threshold value interval corresponding to a reference color channel and a sample channel image corresponding to the second sample image under the reference color channel; Performing binarization processing on the sample channel image using the threshold interval to obtain a binarized image; Performing morphological processing on the binary image to obtain at least one candidate region division image; The candidate area division image that satisfies a second matching condition with the second standard area ratio in the at least one candidate area division image is used as the standard area division image corresponding to the second sample image.

15. The device according to claim 14, characterized in that The third acquisition unit is further used to acquire a first channel image corresponding to the first reference image under the reference color channel and a second channel image corresponding to the second reference image under the reference color channel, wherein the first reference image does not include the first object, and the second reference image includes the first object; acquire a first pixel value distribution of the first channel image; acquire a second pixel value distribution of the second channel image; and determine a threshold interval corresponding to the reference color channel based on the first pixel value distribution and the second pixel value distribution.

16. The device according to claim 14 or 15, characterized in that The third acquisition unit is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being less than the second standard area ratio, the binarized image is expanded using a reference expansion kernel to obtain the at least one candidate area division image.

17. The device according to claim 14 or 15, characterized in that The third acquisition unit is further used to determine, based on the binarized image, a predicted area ratio corresponding to the first object in the second sample image; in response to the predicted area ratio being greater than the second standard area ratio, the binarized image is reduced using a reference reduction kernel to obtain the at least one candidate area division image.

18. An image processing device, characterized in that: The device comprises: A first acquiring unit, configured to acquire an image to be processed, where the image to be processed includes a first object; a transforming unit, configured to perform attribute transformation on the image to be processed so that the image to be processed after attribute transformation has target visual attributes; a segmentation unit, configured to call a first target segmentation model to perform region segmentation on the image to be processed after the attribute transformation, to obtain a first region segmentation result corresponding to the image to be processed, wherein the first region segmentation result is used to indicate a region in the image to be processed where the first object is located, and the first target segmentation model is trained according to any one of claims 1 to 7; The second acquisition unit is used to acquire a measurement index corresponding to the image to be processed based on the first region segmentation result, where the measurement index is used to indicate the area ratio corresponding to the first object that meets the area restriction condition in the image to be processed.

19. The device according to claim 18, characterized in that The image to be processed corresponds to at least one color channel, and the transformation unit is used to determine the pixel mean corresponding to each color channel in the image to be processed based on the pixel value of each pixel point in the image to be processed under each color channel; Using the reference values ​​corresponding to the respective color channels, numerically transforming the pixel mean values ​​corresponding to the respective color channels in the image to be processed to obtain pixel value transformation benchmarks corresponding to the respective color channels in the image to be processed, wherein the reference values ​​corresponding to the respective color channels are determined based on the target visual attributes; Based on the pixel value transformation benchmarks corresponding to the respective color channels in the image to be processed, the pixel values ​​of the respective pixel points in the image to be processed under the respective color channels are transformed.

20. The device according to claim 18, wherein The transformation unit is used to obtain an image transformation model, which is used to output an image with the target visual attributes; and call the image transformation model to perform attribute transformation on the image to be processed.

21. A computer device, characterized in that: The computer device includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements the model training method as described in any one of claims 1 to 7, or implements the image processing method as described in any one of claims 8 to 10.

22. A computer-readable storage medium, characterized in that At least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by the processor so that the computer implements the model training method as described in any one of claims 1 to 7, or implements the image processing method as described in any one of claims 8 to 10.

23. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the model training method as described in any one of claims 1 to 7, or implements the image processing method as described in any one of claims 8 to 10.

Images