Training method of image processing model and computer-aided diagnosis method

Abstract

This application provides a training method for an image processing model and a computer-aided diagnostic method, relating to the field of image processing. The first training sample includes a first sample image and information about a first reference object representing a first reference object. The first reference object comes from an image different from the first sample image. Thus, the same first reference object can be combined with multiple first sample images to construct unpaired training data, improving the resource utilization of scarce samples. Multiple candidate results output by the first image processing model based on the same first training sample are obtained. The first image processing model is trained according to the training weights (relative advantages within the group) of the multiple candidate results. That is, by adapting GRPO to the flow matching generation architecture, the first image processing model is trained to optimize the generation trajectory, improving the coherence between the first generated object and the background features while retaining the true features of the first reference object.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a training method for an image processing model and a computer-aided diagnostic method. Background Technology

[0002] In the field of artificial intelligence medical imaging, the extreme scarcity of high-quality labeled data has become a key bottleneck restricting the performance improvement of screening models. For computed tomography (CT) tumor screening projects, the current mainstream research and development model mainly relies on large-scale, multi-center raw data collection and manual annotation, and then develops screening models based on this. This approach not only faces high manpower and time costs, but also has inherent limitations in covering rare case characteristics and improving the model's cross-center generalization ability. Summary of the Invention

[0003] In a first aspect, embodiments of this application provide a training method for an image processing model, comprising: acquiring a first training sample, the first training sample including a first sample image, first target region information, and first reference object information, wherein the first target region information is used to identify a first target region in the first sample image, the first reference object information is used to represent a first reference object, and the first reference object comes from an image different from the first sample image; inputting the first training sample into a first image processing model to be trained, obtaining multiple candidate results output by the first image processing model, wherein each candidate result includes a first predicted image, first generated object information, and a first velocity field parameter, and the first predicted image is a first target region in the first sample image. An image is obtained by generating a first generated object in a first target region. The first generated object information is used to represent the first generated object, and the first generated object is used to reconstruct the first reference object. The first velocity field parameter is used to represent the generation trajectory evolving from a noisy state to the first generated object. For any candidate result, training weights are determined based on the first predicted image, the first generated object information, the first sample image, and the first reference object information. The training weights are used to represent the degree of deviation of the quality evaluation value of the candidate result from the quality evaluation values ​​of the plurality of candidate results. The first image processing model is trained to adjust the generation trajectory based on the training weights corresponding to each candidate result.

[0004] Secondly, embodiments of this application provide an image processing method, comprising: acquiring a target image, third target region information, and third reference object information, wherein the third target region information is used to identify a third target region in the target image, and the third reference object information is used to represent a third reference object; inputting the target image, the third target region information, and the third reference object information into a first image processing model to obtain a third predicted image and third generated object information output by the first image processing model, wherein the third predicted image is an image obtained by generating a third generated object in the third target region of the target image, the third generated object information is used to represent the third generated object, and the third generated object is used to reconstruct the third reference object; wherein the first image processing model is trained based on the method described in the first aspect.

[0005] Thirdly, embodiments of this application provide a training method for an image processing model, comprising: obtaining a third predicted image containing a third generated object output by a first image processing model, wherein the third predicted image is obtained based on the method described in the second aspect; inputting the third predicted image into a second image processing model to be trained, thereby obtaining a predicted segmentation result for the third generated object output by the second image processing model; and training the second image processing model using a loss value between the predicted segmentation result and the labeled segmentation result of the third generated object.

[0006] Fourthly, embodiments of this application provide a computer-aided diagnostic method, comprising: acquiring a medical image containing a screening object; inputting the medical image into a second image processing model to obtain a predicted segmentation result for the screening object output by the second image processing model, wherein the second image processing model is trained based on the method described in the third aspect; and detecting whether the screening object has any abnormalities based on the predicted segmentation result.

[0007] Fifthly, embodiments of this application provide an image processing apparatus, comprising: a target data acquisition module for acquiring a target image, third target region information, and third reference object information; and an output result acquisition module for inputting the target image, the third target region information, and the third reference object information into a first image processing model to obtain a third predicted image and third generated object information output by the first image processing model, wherein the third predicted image is an image obtained by generating a third generated object in the third target region, and the third generated object is used to reconstruct the third reference object, wherein the first image processing model is trained based on the method described in the first aspect.

[0008] Sixthly, embodiments of this application provide a computer-aided diagnostic system, comprising: a client, configured to send an anomaly screening request, the anomaly screening request carrying a medical image containing a screening object, the anomaly screening request being used to detect whether the screening object has an anomaly; a server, configured to receive the anomaly screening request from the client, input the medical image into a second image processing model, obtain a predicted segmentation result for the screening object output by the second image processing model, detect whether the screening object has an anomaly based on the predicted segmentation result, and send the detection result to the client, wherein the second image processing model is trained based on the method described in the third aspect, and the detection result is used to characterize whether the screening object has an anomaly; the client is further configured to output the detection result.

[0009] In a seventh aspect, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0010] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0011] Ninthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the embodiments of this application.

[0012] According to the technical solution of the embodiments of this application, the first training sample includes a first sample image, first target region information (identifying the first target region in the first sample image), and first reference object information (representing the first reference object). The first reference object comes from an image different from the first sample image. That is, the first reference object and the first sample image constitute unpaired training data. Thus, the same scarce first reference object can be combined with multiple ordinary sample images (i.e., first sample images) that do not contain the first reference object to construct more unpaired training data, thereby improving the resource utilization rate of scarce samples (i.e., first reference objects) and enhancing the generalization ability of the first image processing model under unpaired training data conditions. This improves the coherence between the first generated object (used to reconstruct the first reference object in the first target region) and the background features (i.e., the first sample image) while retaining the real features of scarce samples.

[0013] Furthermore, the first training sample is input into the first image processing model to be trained, obtaining multiple candidate results output by the first image processing model. This allows the first image processing model to output multiple different candidate results under the same input conditions. Group Relative Policy Optimization (GRPO) is employed, which determines the training weight of each candidate result based on its quality evaluation value (representing the deviation of the candidate result's quality evaluation value from the quality evaluation values ​​of the other candidate results). Based on the training weight of each candidate result, the first image processing model is trained to adjust the generation trajectory. This allows the first image processing model to encourage generation trajectories with higher quality evaluation values ​​and suppress generation trajectories with lower quality evaluation values, thereby guiding the first image processing model to gradually optimize the generation trajectory. In other words, by adapting GRPO to the flow matching generation architecture, the first image processing model is trained to optimize the generation trajectory between two competing objectives: the true features of the first reference object and the coherence between the first generated object and the background features of the first sample image.

[0014] Therefore, while preserving the true features of the first reference object, the first generated object can be naturally adapted to the background features of the first sample image. Even when the number of first reference objects is scarce, the visual features of the first reference object can be transferred to different first sample images.

[0015] Specifically in the context of medical imaging, the unique visual features of rare lesions can be used as the first reference object information input into the first image processing model. This enables the first image processing model to generate images that retain the true features of the rare lesion under different host anatomical backgrounds (as the first sample images), thereby effectively overcoming the problem of limited model performance caused by insufficient rare case samples.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more easy to understand, specific embodiments of this application are given below. Attached Figure Description

[0017] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0018] Figure 1This diagram illustrates the architecture of an image processing system 100 provided in an embodiment of this application.

[0019] Figure 2 This diagram illustrates the architecture of a computer-aided diagnostic system 200 provided in an embodiment of this application.

[0020] Figure 3 A flowchart illustrating a training method 300 for an image processing model according to an embodiment of this application is shown.

[0021] Figure 4 This illustration shows an example of training the RL stage in a medical image scene according to an embodiment of this application.

[0022] Figure 5 A flowchart illustrating a training method 500 for an image processing model according to an embodiment of this application is shown.

[0023] Figure 6 This illustration shows an example of training the SL stage in a medical image scene according to an embodiment of this application.

[0024] Figure 7 A flowchart illustrating the image processing method 700 provided in an embodiment of this application is shown.

[0025] Figure 8 This diagram illustrates a training example of the inference stage in a medical image scene according to an embodiment of this application.

[0026] Figure 9 A flowchart illustrating the training method 900 for the image processing model provided in an embodiment of this application is shown.

[0027] Figure 10 A flowchart of a computer-aided diagnostic method 1000 provided in an embodiment of this application is shown.

[0028] Figure 11 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0029] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0030] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0031] First, some of the terms or terms used in the embodiments of this application will be explained as follows.

[0032] CT: is an imaging technique that uses X-ray beams to scan layers of the human body and reconstruct cross-sectional images using a computer. Its data is usually stored in the form of a three-dimensional voxel grid, where each voxel represents a volume unit in space and contains a specific Hounsfield Unit to reflect tissue density.

[0033] Magnetic Resonance Imaging (MRI) is an imaging technique that uses signals generated by the resonance of atomic nuclei in a strong magnetic field to reconstruct images. It does not use ionizing radiation and has extremely high resolution for soft tissues (such as the brain, spinal cord, muscles, ligaments, etc.). It is often used for detailed examination of the nervous system, joints, and abdominal anatomical structures.

[0034] Lesion: refers to an abnormal area in an anatomical structure or tissue formed due to pathological changes. It is a core focus of diagnosis and treatment planning, such as tumors.

[0035] Flow matching is a type of machine learning architecture designed to learn the underlying probability distribution of data and synthesize novel, realistic samples. Flow matching directly learns a time-varying vector field to define a deterministic probability flow between a source distribution (such as Gaussian noise) and a target data distribution (such as medical images). This vector field generates a flow that transforms samples from the source distribution to the target data distribution according to ordinary differential equations.

[0036] GRPO is a policy optimization algorithm that does not require a commentator network. It calculates the relative advantage of each candidate result by normalizing multiple candidate results under the same conditions within the group, thereby providing a stable training baseline.

[0037] Segmentation Mask: Also known simply as a mask, it is a binary or multi-labeled matrix with the same spatial dimensions as the original image. It is used to accurately delineate the boundaries of objects (such as lesion structures or anatomical structures), separate the target region from the background, and provide standardized annotation information for quantitative analysis and subsequent deep model training.

[0038] Dice loss is a commonly used metric to measure the similarity between two samples (such as the degree of overlap between a predicted segmentation mask and a ground truth labeled mask).

[0039] In the field of artificial intelligence medical imaging, the performance improvement of deep learning models is often limited by the long-tail distribution of disease cases, in which the representativeness of early or rare lesions is severely insufficient, which is precisely the most critical part of clinical diagnosis. Although the collection and annotation of large-scale multi-center data is the current mainstream training mode, it not only faces high human and time costs, but also has inherent limitations in covering rare lesion features and improving cross-center generalization.

[0040] To address the aforementioned data scarcity issue, controllable lesion synthesis in 3D CT has become a crucial means of supplementing training data. One generative image processing scheme implants simulated single lesions into healthy CT images using a generative model. However, this scheme can only generate single-type tumor data, lacking coverage of multiple anatomical structures and complex pathological features. Another generative image processing scheme specifies the lesion implantation location through manually designed rules and combines text prompts and binary masks to synthesize medical images containing lesions based on spatial consistency. However, whether based on low-dimensional signals such as text prompts and binary masks, or relying on manually specified implantation locations, these sparse control signals are inherently insufficiently defined and cannot capture the fine textures and complex morphological variations of real lesions in CT images. This leads the model to tend to fill information gaps with smoothing or averaging patterns, and the full-image generation architecture often lacks explicit constraints on the topological features of anatomical structures, thus making it difficult to guarantee the rationality and realism of the generated images in terms of anatomical structures.

[0041] In addition, the above methods all rely on a large amount of paired training data (such as extracting tumor and background anatomical images separately as paired training data from images of anatomical structures containing tumors), while paired training data is extremely scarce in clinical practice.

[0042] In view of this, the embodiments of this application aim to provide a training scheme for an image processing model and a computer-aided diagnosis scheme. Using unpaired training data as the first training sample, and through the adaptation of GRPO and the streaming matching generation architecture, the first image processing model is trained to improve the coherence between the first generated object (used to reconstruct the first reference object) and the background features while preserving the true features of the first reference object. A detailed description follows.

[0043] To facilitate understanding, let's first combine Figure 1 This paper introduces the application scenarios of the technical solution presented in this application. Figure 1 This diagram illustrates the architecture of an image processing system 100 provided in an embodiment of this application. Figure 1 As shown, the image processing system 100 includes a client 101 and a server 102.

[0044] In this system, client 101 sends image processing requests to server 102. For example, a user can trigger an image processing request by inputting natural language commands or uploading an image containing a target object through the user interface provided by client 101. Server 102 receives the image processing request and inputs the image containing the target object into a trained second image processing model to obtain the predicted segmentation result of the target object output by the second image processing model.

[0045] For example, server 102 can send the predicted segmentation result to client 101, and client 101 can output the predicted segmentation result, so that users can intuitively view the predicted segmentation result of the target object, or perform subsequent data analysis or task processing based on the predicted segmentation result.

[0046] In this embodiment, the second image processing model is trained using the training method 900 of this embodiment. In this training method 900, the target image, third target region information (used to identify the third target region in the target image), and third reference object information (used to represent the third reference object) are input into the first image processing model to obtain a third predicted image (an image obtained by generating a third generated object in the third target region of the target image). Then, the third predicted image is input into the second image processing model to be trained to obtain a predicted segmentation result for the third generated object output by the second image processing model. Finally, the second image processing model is trained using the loss value between the predicted segmentation result and the corresponding labeled segmentation result.

[0047] The first image processing model is trained using the training method 300 of this application. In this training method 300, the first training sample includes a first sample image, first target region information (identifying the first target region in the first sample image), and first reference object information (representing the first reference object). The first reference object is extracted from an image different from the first sample image; that is, the first reference object and the first sample image constitute unpaired training data. Further, the first training sample is input into the image processing model to be trained, obtaining multiple candidate results output by the image processing model. This allows the image processing model to output multiple different candidate results under the same input (first training sample). The GRPO method is used to train the image processing model to optimize the generated trajectory. This enables the image processing model to encourage generated trajectories with higher quality evaluation values ​​and suppress generated trajectories with lower quality evaluation values ​​based on training weights, thereby guiding the image processing model to gradually optimize the generated trajectory (i.e., the flow matching generation architecture). Thus, by adapting GRPO to the flow matching generation architecture, the image processing model can be trained to optimize the generated trajectory between two competing objectives (the true features of scarce sample objects and the coherence between the first generated object and background features).

[0048] For example, the image processing system 100 can be applied to different scenarios, including but not limited to autonomous driving scenarios, industrial quality inspection scenarios, remote sensing monitoring scenarios, smart retail scenarios, and medical diagnostic scenarios.

[0049] For example, in autonomous driving scenarios, image processing requests could be real-time traffic analysis requests, with target objects such as pedestrians, vehicles, or traffic signs on the road. Segmentation results allow users to accurately identify obstacle outlines and locations, enabling safe path planning and obstacle avoidance decisions. Similarly, in industrial quality inspection scenarios, target objects could be scratches, cracks, or defects on product surfaces, with segmentation results allowing users to quickly locate defective areas and determine product quality levels. Furthermore, in smart retail scenarios, target objects could be goods on shelves, with segmentation results enabling users to quickly count the quantity and location of displayed goods, thus aiding inventory management.

[0050] The following is combined Figure 2 This paper introduces the application of the technical solution of this application in medical imaging scenarios. Figure 2 This diagram illustrates the architecture of a computer-aided diagnostic system 200 provided in an embodiment of this application. Figure 2 As shown, the computer-aided diagnostic system 200 includes a client 201 and a server 202.

[0051] Client 201 is used to send an anomaly screening request to server 202. The anomaly screening request carries a medical image containing the object to be screened. For example, a user can upload a medical image containing the object to be screened, such as a CT image, MRI scan, ultrasound image, or photoacoustic image, based on the user interface provided by client 201, thereby triggering the anomaly screening request.

[0052] Server 202 receives an anomaly screening request from client 201 and responds to the request by inputting the medical image carried in the request into a second image processing model to obtain a predicted segmentation result for the screening object output by the second image processing model. Server 202 further detects whether the screening object has anomalies based on the predicted segmentation result. For example, server 202 can input the segmentation result into an anomaly detection model to obtain a detection result characterizing whether the screening object has anomalies, and then send the detection result to client 201.

[0053] Client 201 is also used to output test results, allowing users (such as doctors) to intuitively see whether there are any abnormalities in the screened individuals, or to make subsequent clinical analysis or diagnostic decisions based on the test results.

[0054] In this application scenario, the unique visual features of rare lesions can be used as the first reference object information input into the first image processing model, enabling the first image processing model to generate images that retain the true features of the rare lesions under different host anatomical backgrounds (as the first sample images), thereby effectively overcoming the problem of limited model performance caused by insufficient rare case samples.

[0055] The first image processing model, trained from real tumor samples, can generate predicted images that retain the true features of the tumor against different host anatomical backgrounds, using real tumor samples as high-dimensional pathological templates. This effectively overcomes the performance limitations caused by insufficient rare case samples. Furthermore, these high-fidelity synthetic images are used as training data to train a second image processing model, enabling it to have stronger generalization ability and segmentation accuracy in real medical image scenarios.

[0056] It is understood that the image processing system, computer-aided diagnostic system, and other components of the client and server described above can employ various technical solutions now and in the future known to those skilled in the art, and will not be described in detail here. Furthermore, the deployment form or other components of the above system may differ in different application scenarios, and the embodiments of this application do not specifically limit this.

[0057] For example, a connection is established between the client and the server via a network. The network provides the medium for communication between the client and the server. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. The client can be hardware, such as mobile phones, personal computers, tablets, wearable devices, and other electronic devices; it can also be browsers, applications, etc., deployed on electronic devices. The server can include servers providing various services, which can be deployed as a distributed server cluster of multiple servers or implemented as a single server, which can be a physical server or a cloud server.

[0058] It should be noted that the application scenarios or examples provided in this application are for ease of understanding, and this application does not specifically limit the application of the technical solutions. Furthermore, the user information (including but not limited to user device information, personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0059] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0060] Figure 3 A flowchart illustrating a training method 300 for an image processing model according to an embodiment of this application is shown. This method 300 can be applied to a server for training a first image processing model during the reinforcement learning (RL) phase. Figure 3 As shown, the method 300 may include steps S301 to S304.

[0061] Step S301: Obtain a first training sample. The first training sample includes a first sample image, first target region information, and first reference object information. The first target region information is used to identify the first target region in the first sample image, and the first reference object information is used to represent the first reference object. The first reference object comes from an image different from the first sample image.

[0062] In this embodiment, the first sample image is configured as a background image that does not contain the first reference object. During the generation phase, the first reference object serves as a reference, and the first image processing model reconstructs the corresponding first generated object based on the feature distribution of the first reference object. Unlike traditional paired training, in this embodiment, the first reference object comes from an image different from the first sample image, and the two together constitute unpaired training data.

[0063] In a specific medical image scenario example, the first sample image can be a CT image of a healthy anatomical structure without lesions (such as a CT image of a liver without tumors), while the first reference object is a reference lesion structure (also called a lesion example, such as a reference tumor or tumor example). For example, the first reference object can be a real lesion structure extracted beforehand from medical images of other patients. Through this cross-image recombination method, the same scarce first reference object can be combined with multiple first sample images that do not contain that object, thereby constructing a large amount of unpaired training data. For example, combining real hepatocellular carcinoma tissue extracted from a liver CT image containing a tumor with multiple healthy liver CT images not only allows for the construction of a rich unpaired training set at extremely low cost, significantly improving the resource utilization of scarce samples, but also enhances the generalization ability of the first image processing model under unpaired data conditions, fundamentally solving the problem of insufficient model training data caused by the scarcity of rare case samples.

[0064] The first reference object information is used to represent the first reference object, including but not limited to: an image representing the first reference object, a feature vector / embedding representation of the first reference object, a mask of the first reference object, etc.

[0065] The first target region information is used to identify the first target region to be generated in the first sample image, that is, to generate the object to be generated in the first target region. The first target region information can take the form of bounding boxes, bounding box coordinates, segmentation masks or heatmaps, etc. For example, the first target region information can be encoded as a binary mask image, in which the pixel value within the first target region is 1 and the pixel value outside the region is 0.

[0066] Step S302: Input the first training sample into the first image processing model to be trained, and obtain multiple candidate results output by the first image processing model. Each candidate result includes a first predicted image, first generated object information, and a first velocity field parameter. The first predicted image is an image obtained by generating a first generated object in the first target region of the first sample image. The first generated object information is used to represent the first generated object. The first generated object is used to reconstruct the first reference object. The first velocity field parameter is used to represent the generation trajectory that evolves from the noise state to the first generated object.

[0067] The first image processing model employs a flow-matching-based generative architecture, defining a deterministic probability flow from the source distribution (e.g., a noise state) to the target data distribution (e.g., a first generated object) by learning a time-varying vector field (parameterized by a neural network). In this embodiment, the first image processing model is configured to learn the generation trajectory (represented by a first velocity field parameter) from the noise state to the first generated object, conditioned on the first training samples.

[0068] For example, under the same first training sample input conditions, the first image processing model outputs a set of different candidate results by introducing a sampling strategy with randomness (such as converting ordinary differential equations into stochastic differential equations for sampling), and each candidate result in the set represents a sampling stream for generating a first generated object in the first target region.

[0069] The first generated object is used to reconstruct the first reference object. For example, the first reference object is a reference lesion structure used as a reference, and the first generated object is the generated lesion structure. The first predicted image represents the image after the first generated object is generated in the first target region, such as a CT image containing the lesion structure. The first generated object information represents the first generated object, including but not limited to: the image representing the first generated object, the feature vector / embedding representation of the first generated object, and the mask of the first generated object (such as the mask of the generated lesion structure). The first velocity field parameter represents the generation trajectory evolving from the noisy state to the first generated object.

[0070] For example, the first image processing model generates candidate results by solving the following Ordinary Differential Equation (ODE): .

[0071] in, This represents the time step for generating the trajectory, and its value range is typically 1000. ; Indicates at time step The hidden variable state at time, when hour For the initial noise sampled from the source distribution, when hour This is the first generated object. The input conditions, in this embodiment, include a first sample image, first target region information, and first reference object information, which are used to guide the generation process to always be anchored to the specified first target region and first reference object. This represents the first image processing model (model parameters are...). The predicted first velocity field parameter defines the instantaneous rate of change of the latent variable from the noise state to the first generated object at each time step, and is the basic unit that constitutes the complete generated trajectory. The preset noise control coefficient, The two terms together constitute a random disturbance term, which is a standard Gaussian noise term, and under the same input conditions... and initial noise By sampling independently multiple times, multiple different generated trajectories can be obtained, thus outputting a set of different candidate results.

[0072] Step S303: For any candidate result, determine the training weight of the candidate result based on the first predicted image, the first generated object information, the first sample image and the first reference object information, wherein the training weight is used to represent the degree of deviation of the quality evaluation value of the candidate result from the quality evaluation values ​​of multiple candidate results.

[0073] The quality evaluation value is used to quantify the quality of the generated candidate results. For example, the quality evaluation value can be determined through a reward function. The reward function is a consistency evaluation function based on the candidate results and the first training samples. The reward value calculated through the reward function can be used as the quality evaluation value, thereby providing feedback signals for subsequent group-based strategy optimization and avoiding reliance on manual annotation, additional discriminator networks, or resource-intensive critic networks.

[0074] Training weights are used to represent the degree of deviation of the quality evaluation value of a candidate result from the quality evaluation values ​​of multiple candidate results, i.e., the relative advantage of the candidate result within the group. Therefore, the Group Relative Policy Optimization (GRPO) method is adopted to directly use the difference in the quality evaluation values ​​of candidate results within the group as the advantage estimate.

[0075] For example, for the first Each candidate result has its training weights. The calculation is as follows: .

[0076] in, Indicates the first Quality evaluation value of each candidate result; This indicates the number of candidate results sampled from the same first training sample; This group The mean of the quality evaluation values ​​of the candidate results; This group Standard deviation of the quality evaluation values ​​of each candidate result. Training weights As the first The normalized relative advantage of each candidate result can eliminate the absolute difference in reward scale between different samples, retaining only the relative difference within the group, so that the first image processing model can focus on learning which generation trajectory can produce a better image under the current input conditions.

[0077] Step S304: Train the first image processing model to adjust the generated trajectory according to the training weights corresponding to each candidate result.

[0078] For example, the GRPO objective function is used to update the model parameters of the first image processing model, thereby training the first image processing model to adjust the generated trajectory: .

[0079] in, Indicates the importance sampling ratio.

[0080] in, The first training sample (including the first sample image, the first target region information, and the first reference object information) corresponds to the first training sample. Corresponding to the The generated trajectory of each candidate result (defined by the first velocity field parameter). This reflects the probability ratio of the first image processing model with the current model parameters to that of the first image processing model with the old model parameters on the generated trajectory. The pruning threshold is used to limit the magnitude of a single policy update and prevent catastrophic deviations in the generated trajectory. This is a pruning function to ensure the conservatism of policy updates; For the current strategy Reference Strategy Kullback-Leibler Divergence (KL) between (typically initial pre-trained models); The coefficients are used to constrain the generated trajectory from deviating too far from the original flow matching prior, ensuring the anatomical rationality and basic quality of the generated image; the GRPO objective function By minimizing gradient descent, the first image processing model is driven to continuously optimize the generation trajectory in the direction of generating higher quality and more realistic images.

[0081] In medical image scenarios, this training method enables the first image processing model to optimize the generation trajectory between two competing objectives: the pathological fidelity of the reference tumor (first reference object), i.e., the generated tumor (first generated object) should retain the real features of the reference tumor; and the anatomical coherence between the first generated object and the background features of the first sample image, i.e., the generated tumor should naturally adapt to the topological features of the host anatomical structure.

[0082] For example, in order to balance training efficiency and generation quality, the training process of the first image processing model adopts a noise reduction strategy. For instance, when generating multiple candidate results based on the SDE solver, only a portion of the denoising steps are used to accelerate the generation and collection of candidate results, thereby reducing the time cost of online evaluation. However, in the inference stage of the first image model (described below through method 700), the complete number of denoising steps is retained to ensure high-quality image generation.

[0083] According to the technical solution of this application embodiment, the first training sample includes a first sample image, first target region information, and first reference object information. The first reference object comes from an image different from the first sample image; that is, the first reference object and the first sample image constitute unpaired training data. Therefore, the same scarce first reference object can be combined with multiple ordinary sample images that do not contain the first reference object to construct more unpaired training data, thereby improving the resource utilization rate of scarce samples and enhancing the generalization ability of the first image processing model under unpaired training data conditions. This improves the coherence between the first generated object and the background features of the first sample image while retaining the true features of the scarce samples. Further, the first training sample is input into the first image processing model to be trained to obtain multiple candidate results output by the first image processing model. This allows the first image processing model to output multiple different candidate results under the same input conditions. Through the adaptation of GRPO and the flow matching generation architecture (i.e., the Flow-GRPO architecture), the first image processing model is trained to optimize the generation trajectory between two competing objectives (the true features of the scarce sample object and the coherence between the background features of the first generated object and the first sample image). Therefore, while preserving the true features of the first reference object, the first generated object can be naturally adapted to the background features of the first sample image. Even when the number of first reference objects is scarce, the visual features of the first reference object can be transferred to different first sample images.

[0084] Specifically in the context of medical imaging, the unique visual features of rare lesions can be used as the first reference object information input into the first image processing model. This enables the first image processing model to generate images that retain the true features of the rare lesion under different host anatomical backgrounds (as the first sample images), thereby effectively overcoming the problem of limited model performance caused by insufficient rare case samples.

[0085] In one implementation, in step S303, determining the training weights of candidate results based on the first predicted image, the first generated object information, the first sample image, and the first reference object information may include: determining the quality evaluation values ​​corresponding to the candidate results under multiple evaluation dimensions based on the first predicted image, the first generated object information, the first sample image, and the first reference object information; for any evaluation dimension, calculating the normalized result of the quality evaluation values ​​under that evaluation dimension based on the quality evaluation values ​​corresponding to the multiple candidate results under that evaluation dimension, and determining the training weights of each candidate result under that evaluation dimension based on the normalized result of the quality evaluation values; and determining the training weights of the candidate results based on the training weights corresponding to the candidate results under each evaluation dimension.

[0086] The evaluation system employs multiple dimensions to assess the generation quality of the first generated object from different perspectives. Each dimension corresponds to an evaluation criterion used to measure the similarity or consistency between the first generated object and the first reference object in a specific feature dimension. For example, the evaluation dimensions include at least one of shape, texture, and object type dimensions. Specifically, the quality evaluation value under the shape dimension evaluates the similarity between the first generated object and the first reference object in shape features; the quality evaluation value under the texture dimension evaluates the similarity between the first generated object and the first reference object in texture distribution; and the quality evaluation value under the object type dimension evaluates the consistency between the type of the first generated object and the type of the first reference object.

[0087] For each evaluation dimension, the first image processing model calculates the quality evaluation value for each candidate result among multiple candidate results under that evaluation dimension. For example, for each evaluation dimension... , The calculation formula is as follows: .

[0088] in, Indicates the number of candidate results generated; This represents the first predicted image; Indicates the first Information about the first generated object corresponding to each candidate result; Indicates the first Each candidate result in the evaluation dimension The quality evaluation value below; express Each candidate result in the evaluation dimension The set of quality evaluation values ​​below.

[0089] For each evaluation dimension, the quality evaluation values ​​of multiple candidate results under that dimension are normalized within the group. For example, the normalization method can be: calculate the mean and standard deviation of the quality evaluation values ​​of multiple candidate results under that evaluation dimension; for each candidate result, subtract the mean from its quality evaluation value under that evaluation dimension and divide by the standard deviation to obtain the normalized result. This normalized result is the training weight (i.e., relative advantage) of each candidate result under that evaluation dimension. Specifically, for any evaluation dimension... , No. The training weights of each candidate result under this evaluation dimension The calculation formula is as follows: .

[0090] in, Indicates the first Each candidate result in the evaluation dimension The quality evaluation value below; express Each candidate result in the evaluation dimension The mean of the quality evaluation values ​​under the following conditions; express Each candidate result in the evaluation dimension The standard deviation of the quality evaluation values; Indicates the first Each candidate result in the evaluation dimension The training weights under the group relative strategy (i.e., the relative advantage within the group) are determined by the group relative strategy. By performing within-group normalization, the differences in units and numerical ranges between different evaluation dimensions can be eliminated, making the training weights under each dimension comparable.

[0091] Furthermore, based on the training weights corresponding to each candidate result under each evaluation dimension, the final training weight of that candidate result is determined comprehensively. For example, a weighted average or simple average can be performed on the training weights across each evaluation dimension to obtain the final training weight of each candidate result. Specifically, the first... The final training weights of each candidate result The calculation formula is as follows: .

[0092] in, Indicates the first Each candidate result in the evaluation dimension The training weights are set below; This indicates summing the training weights for each evaluation dimension. This represents a simple average of the three evaluation dimensions (the denominator is adjusted accordingly if the number of evaluation dimensions is different). Positive This indicates that the generation trajectory of the candidate result will be encouraged (i.e., the generation probability of the candidate result will be increased in subsequent training), while negative ones will be discouraged. This means that the generation trajectory of the candidate result will be penalized (i.e., the generation probability of the candidate result will be reduced in subsequent training).

[0093] Based on this implementation, by evaluating the generated quality from multiple evaluation dimensions and performing intra-group normalization within each evaluation dimension, the quality evaluation values ​​under different evaluation dimensions are comparable, thereby enabling the determination of training weights by comprehensively considering the evaluation results of multiple evaluation dimensions. This approach avoids generation bias caused by relying on a single evaluation metric (such as focusing only on shape while ignoring texture), allowing the first image processing model to simultaneously optimize the generation trajectory across multiple evaluation dimensions, generating high-quality images that closely approximate the real first reference object in multiple aspects.

[0094] In one implementation, determining the quality evaluation value of the candidate result under multiple evaluation dimensions based on the first predicted image, the first generated object information, the first sample image, and the first reference object information may include at least one of the following: extracting the first shape feature of the first generated object and the second shape feature of the first reference object using a pre-trained feature extraction model, and determining the quality evaluation value of the candidate result under the shape dimension based on the distance between the first shape feature and the second shape feature; determining the first pixel value distribution of the first generated object and the second pixel value distribution of the first reference object, and determining the quality evaluation value of the candidate result under the texture dimension based on the similarity between the first pixel value distribution and the second pixel value distribution; determining the first segmentation result of the object type belonging to the first reference object in the first predicted image using a pre-trained segmentation network, and determining the quality evaluation value of the candidate result under the object type dimension based on the segmentation loss value between the first segmentation result and the first reference object information, wherein the first training sample further includes the first object type information, which is used to represent the object type of the first reference object.

[0095] The shape dimension is used to evaluate the similarity in shape features between the first generated object and the first reference object. The pre-trained feature extraction model is a lightweight network. This is used to extract abstract shape features from an image. For example, a shape feature extractor... A contrastive learning strategy using triplet loss can be employed for training. For example, multiple tumors (first reference objects) can be extracted from a CT image dataset to construct a tumor template dataset (a collection of information about the first reference objects). A shape feature extractor can then be trained on the tumor template dataset using triplet loss that incorporates random data augmentation transformations (such as centering, rotation, and scaling) to capture abstract shape features. This triplet loss... The calculation formula is as follows: ; .

[0096] in, Represents the first in the tumor template dataset One sample (i.e., the reference tumor used as the anchor point). This represents random data augmentation transformations, including centering, rotation, and scaling. The feature vector obtained by the shape feature extractor for the anchor point. This is the feature vector of a positive sample (the same reference tumor after enhancement transformation). The feature vectors of negative samples (different reference tumors); This represents the feature distance between the anchor point and the positive sample. This represents the feature distance between the anchor point and the negative sample; These are predefined hyperparameters used to define the distance margin boundary that distinguishes between positive and negative samples; Indicates taking The operation ensures that the loss is non-negative. The shape feature extractor is trained using this triplet loss, enabling it to maintain feature consistency across different enhanced views of the same reference tumor while also possessing discriminative features for different reference tumors.

[0097] After training, the distance between the first shape feature of the first generated object and the second shape feature of the first reference object is used as the quality evaluation value in the shape dimension. Specifically, the shape reward function... Defined as: .

[0098] in, This indicates the information of the first generated object (such as the mask of the generated tumor). This indicates the information of the first reference object (such as the mask of the input reference tumor). The first shape feature extracted from the first generated object by the shape feature extractor. The second shape feature extracted from the first reference object using the same shape feature extractor; This represents the square of the Euclidean distance. The shape bonus value. This is the quality evaluation value in the shape dimension. The larger the shape reward value (i.e. the smaller the absolute value of the negative distance), the more similar the first generated object is to the first reference object in shape.

[0099] The texture dimension is used to evaluate the similarity in texture distribution between the first generated object and the first reference object. For example, the first pixel value distribution of the first generated object and the second pixel value distribution of the first reference object can be calculated separately, and the Bhattacharyya Distance (BD) can be used to measure the similarity between the two distributions. Texture reward function The calculation formula is as follows: ; .

[0100] in, This represents the first predicted image (such as the final generated CT image containing the tumor). This represents the pixel value of the first target region extracted from the first predicted image using a mask of the first generated object; Indicates to Calculated A normalized histogram, which is a specific representation of the distribution of the first pixel value and the distribution of the second pixel value; This represents the number of bins in the histogram. This is the Bach distance, used to quantify the difference between two normalized histograms. Texture reward value. This is the quality evaluation value in the texture dimension. Because it's negative, The larger the value (i.e., the smaller the Bach distance), the more similar the first generated object is to the first reference object in terms of texture distribution.

[0101] The object type dimension is used to evaluate the consistency between the type of the first generated object and the type of the first reference object. The first training sample also includes first object type information, which represents the object type of the first reference object. In a medical image scenario, the object type could be a tumor type, and the first object type information could be a textual description of the tumor type. (Pre-trained segmentation network) A semantic segmentation network capable of segmenting various fine-grained tumor types is proposed. This network segments a first predicted image, determining a first segmentation result belonging to the object type of a first reference object. Then, the segmentation loss value between the first segmentation result and the first reference object information is calculated as a quality evaluation value in the object type dimension. A segmentation reward function is then implemented. The calculation formula is as follows: .

[0102] Among them, "object type" represents the first object type information. Represents a segmented network The first segmentation result of the first predicted image for the specified "object type" (i.e., the segmentation mask belonging to that object type); For binarization operation, Indicates the truth value of the segmentation of the first reference object; This represents the Dice loss function, used to measure the overlap between the first segmentation result and the labeled segmentation result. Segmentation reward value. This is the quality evaluation value under the object type dimension. Because it has a negative sign, The larger the value (i.e., the smaller the Dice loss), the better the consistency of the first generated object in terms of object type, such as the generated tumor indeed belonging to the object type of the first reference object.

[0103] Based on this implementation, the generation quality is evaluated from multiple perspectives using three evaluation dimensions: shape, texture, and object type. This allows the training weights to comprehensively reflect the overall performance of the first generated object in terms of shape fidelity, texture consistency, and semantic correctness. Specifically, the shape dimension captures abstract shape features through a pre-trained feature extraction model, which can more accurately measure shape similarity compared to simple pixel-level comparison. The texture dimension measures texture consistency by the similarity of pixel value distribution, which can effectively capture subtle texture differences in CT images. The object type dimension introduces object type information as a condition to ensure that the generated object is similar to the reference object not only in low-level features (shape and texture) but also in high-level semantics (object type).

[0104] In one implementation, training a first image processing model to adjust the generation trajectory based on the training weights corresponding to each candidate result may include: for each candidate result, determining the adjustment direction of the generation trajectory corresponding to the candidate result based on the relationship between the training weights of the candidate result and the comparison benchmark, and adjusting the probability distribution of each generation step on the generation trajectory according to the adjustment direction, wherein the comparison benchmark is the average of the training weights of multiple candidate results; wherein, when the training weights are greater than the comparison benchmark, the adjustment direction is closer to the distribution mean; when the training weights are less than the comparison benchmark, the adjustment direction is farther from the distribution mean, wherein the distribution mean is the mathematical expectation of the probability distribution of each generation step on the generation trajectory.

[0105] For each candidate result, the first image processing model calculates the relationship between the training weights of that candidate result and the benchmark (the average of the training weights of multiple candidate results). If the training weights are greater than the benchmark, it indicates that the generation quality of the candidate result is better than the average level within the group. The adjustment direction is determined to be closer to the distribution mean, that is, shifting the log probability of each generation step on the generation trajectory of the candidate result towards the distribution mean, thereby increasing the probability of the generation trajectory appearing in subsequent sampling, strengthening the probability distribution of the generation steps of the candidate result, and making the first image processing model tend to generate objects along this direction. For example, when the adjustment direction is closer to the distribution mean, adjusting the probability distribution of each generation step on the generation trajectory may include: for each current generation step, calculating the difference vector between the distribution mean and the log probability of the current generation step, multiplying this difference vector by a preset positive adjustment coefficient as a positive offset, and adding it to the log probability of the current generation step to update the log probability of the current generation step. The positive adjustment coefficient is a real number greater than zero, which causes the updated log probability to shift in the log probability space along the direction of approaching the distribution mean, thereby strengthening the generation path of the candidate result.

[0106] If the training weights are less than the benchmark, it indicates that the generation quality of the candidate result is lower than the group average. The adjustment direction is determined to be away from the distribution mean, that is, shifting the log probability of each generation step on the generation trajectory of the candidate result in a direction away from the distribution mean. This reduces the probability of the generation trajectory appearing in subsequent sampling, suppresses the probability distribution of the generation steps of the candidate result, and prevents the first image processing model from generating objects along this direction. For example, when the adjustment direction is away from the distribution mean, adjusting the probability distribution of each generation step on the generation trajectory can include: for each current generation step, calculating the difference vector between the distribution mean and the log probability of the current generation step, multiplying this difference vector by a preset negative adjustment coefficient as a reverse offset, and adding it to the log probability of the current generation step to update the log probability of the current generation step. Here, the negative adjustment coefficient is a real number less than zero, causing the updated log probability to shift in the log probability space in a direction away from the distribution mean, thereby suppressing the generation path of the candidate result.

[0107] In medical image scenarios, this training method enables the first image processing model to optimize the generation trajectory between two competing objectives: on the one hand, preserving the true pathological features of the first reference object (such as a rare tumor sample); on the other hand, ensuring the anatomical coherence between the first generated object and the background features of the first sample image (such as a CT image of the host's anatomical structure). By adjusting the probability distribution of each generation step on the generation trajectory based on the relationship between the training weights and the comparison benchmark, the first image processing model can selectively strengthen high-quality generation paths and suppress low-quality generation paths, thereby gradually optimizing the generation trajectory. This achieves reinforcement learning training under unpaired data conditions, ensuring that the final generated object retains the true features of the first reference object while naturally adapting to the background features of the first sample image.

[0108] Based on this implementation, since the benchmark is the average of the training weights of multiple candidate results, there is no need to introduce an additional critic network to calculate the baseline, thus reducing model complexity and computational overhead. Simultaneously, by determining the adjustment direction based on the relationship between the training weights and the benchmark, and adjusting the probability distribution of each generation step on the generation trajectory according to the adjustment direction, the first image processing model can perform efficient policy optimization based on a group-relative strategy, improving the coherence between the generated object and the background while preserving the true features of scarce samples.

[0109] In one implementation, the first image processing model includes a generator network module and an adapter module. During the training of the first image processing model to adjust the generation trajectory, the model parameters of the generator network module are frozen, and the model parameters of the adapter module are updated.

[0110] The generative network module is the core network responsible for image generation in the first image processing model, such as a flow matching architecture based on a U-Net. The generative network module has already been trained on paired data during the supervised learning (SL) phase (described below via method 500), learning basic generative capabilities. The adapter module is a lightweight supplementary network module that takes a first reference object (such as a reference tumor) as input. The output of the adapter module is added to the intermediate features of the generative network module through zero-initialized convolutional layers to prevent excessive perturbation of the generative network module in the early stages of training.

[0111] During the RL phase (i.e., while training the first image processing model to adjust the generated trajectory), the model parameters of the generator network module are frozen, and only the model parameters of the adapter module are updated. This strategy preserves the parameters of the pre-trained generator network module, preventing the first image processing model from significantly deviating from the distribution learned in the supervised learning phase, thereby ensuring the stable growth of training weights and the stability of training. Simultaneously, training only the lightweight adapter module significantly reduces the number of trainable parameters, lowering computational overhead and GPU memory requirements.

[0112] Based on this implementation, since the model parameters of the generator network module are frozen and the model parameters of the adapter module are updated during the process of training the first image processing model to adjust the generated trajectory (RL phase), the generation capabilities already learned by the pre-trained generator network module can be preserved, preventing model performance degradation caused by RL phase training. Simultaneously, by introducing a lightweight adapter module, the first image processing model can learn to transfer features of the first reference object to different first sample images without significantly increasing computational overhead, thereby achieving effective RL training under unpaired data conditions.

[0113] In one implementation, the first reference object is the lesion structure, and the first sample image includes the anatomical structure to which the lesion structure belongs but does not include the lesion structure itself.

[0114] In the context of medical imaging, the first reference object is a real lesion structure (such as a tumor) extracted from medical images of other patients, and the first sample image is an image that includes the anatomical structure to which the lesion structure belongs but does not contain the lesion structure itself (such as a CT image of a healthy anatomical structure that does not contain a tumor). The first target region information identifies the target region in the first sample image where the lesion structure to be generated is located.

[0115] For example, the first reference object may be gastric cancer tissue extracted from CT images of gastric cancer patients, and the first sample image may be a CT image of a healthy stomach that does not contain gastric cancer tissue. Based on the first training sample, the first image processing model generates simulated gastric cancer tissue in a specified target region of the healthy stomach CT image, such that the generated gastric cancer tissue retains the shape, texture, and semantic features of real gastric cancer tissue while naturally adapting to the background features of the healthy stomach CT image.

[0116] Based on this implementation, since the first reference object is the lesion structure, and the first sample image contains the anatomical structure to which the lesion structure belongs but does not contain the lesion structure itself, the first image processing model can generate the lesion structure on a healthy anatomical background, achieving controllable generation of synthetic lesion images. This method is particularly suitable for data augmentation scenarios of rare lesions. Even if real samples of rare lesions are scarce, their features can be transferred to different healthy anatomical backgrounds to generate diverse synthetic training data.

[0117] In a specific example within a medical imaging scenario, such as Figure 4 As shown, the first target region information example is a bounding box, the first sample image example is a CT image of a healthy anatomical structure, and the first reference object information example is a mask for a reference tumor. The first reference object is extracted from a CT image containing a tumor anatomical structure. Therefore, the first reference object information and the first sample image constitute unpaired training data.

[0118] The first image processing model includes a generator network module and an adapter module. The generator network module adopts a flow matching architecture, receiving a first sample image and information about a first target region as conditional inputs, and combining this with information about a first reference object injected by the adapter module. It then performs multi-step SDE sampling to generate multiple candidate results. Based on a reward function (multi-dimensional), such as shape, texture, and object type, a quality evaluation value is calculated for each candidate result. Further, through intra-group normalization and cross-dimensional weighted fusion, a final training weight (intra-group relative advantage) is generated for each candidate result. This training weight serves as a reinforcement learning signal for the Flow-GRPO algorithm, driving the adapter module to update its model parameters. This means that only the model parameters of the adapter module are optimized, while the model parameters of the generator network module are frozen. This allows for the precise transfer of features from scarce reference objects and the synergistic optimization of background adaptation capabilities while preserving pre-trained generation capabilities. The first image processing model, obtained through iterative training, can naturally and coherently synthesize the real morphology, texture, and semantic features of rare lesions (the first reference object) into different host anatomical backgrounds (the first sample images) without pairing, significantly improving the modeling ability and generalization performance of rare pathological phenotypes in medical image generation tasks.

[0119] Figure 5A flowchart illustrating a training method 500 for an image processing model according to an embodiment of this application is shown. This method 500 can be applied to a server-side environment for training generative network modules during the SL (Simulation) phase. Figure 5 As shown, the method 500 may include steps S501 to S503.

[0120] Step S501: Obtain a second training sample. The second training sample includes a second sample image, second target region information, and second reference object information. The second target region information is used to identify the second target region in the second sample image, and the second reference object information is used to represent the second reference object. The second sample image and the second reference object come from the same image.

[0121] In the SL (Simulation and Training) phase, the second sample image and the second reference object constitute paired training data. For example, the second sample image can be a medical image containing an object (such as a CT image containing a tumor), and the second reference object is a real object (such as a real tumor) extracted from that medical image. Thus, a pairing relationship exists between the second sample image and the second reference object, which can be used as a supervisory signal to train the basic generative capabilities of the generative network module.

[0122] The second reference object information is used to represent the second reference object, including but not limited to: an image representing the second reference object, a feature vector / embedding representation of the second reference object, and a mask of the second reference object. The second target region information is used to identify the second target region to be generated in the second sample image, i.e., to generate the object to be generated in the second target region. The second target region information can take the form of a bounding box, bounding box coordinates, a segmentation mask, or a heatmap, for example, the second target region information can be encoded as a binarized mask image.

[0123] Step S502: Input the second training sample into the generator network module to be trained, and obtain the prediction result output by the generator network module. The prediction result includes a second prediction image, second generated object information and second velocity field parameters. The second prediction image is an image obtained by generating a second generated object in the second target region of the second sample image. The second generated object information is used to represent the second generated object. The second generated object is used to reconstruct the second reference object. The second velocity field parameters are used to represent the generation trajectory that evolves from the noise state to the second generated object.

[0124] The second generated object is used to reconstruct the second reference object, such as a generated lesion structure. The second predicted image represents the image after the second generated object is generated in the second target region, such as a CT image containing the lesion structure. The second generated object information represents the second generated object, including but not limited to: the image representing the second generated object, the feature vector / embedding representation of the second generated object, the mask of the second generated object, etc., where the second generated object information is, for example, a lesion structure mask. The second velocity field parameter represents the generation trajectory evolving from the noisy state to the second generated object.

[0125] Step S503: Adjust the model parameters of the generator network module using the target loss value. The target loss value includes: the segmentation loss value between the second generated object information and the second reference object information, and the velocity field loss value between the second velocity field parameter and the target velocity field parameter. The target velocity field parameter is used to represent the generated trajectory evolving from the noise state to the second reference object.

[0126] The segmentation loss value is the loss between the information of the second generated object and the information of the second reference object, used to ensure that the second generated object is consistent with the second reference object at the segmentation level. For example, the segmentation loss value can be the Dice loss, which is the Dice loss calculated between the segmentation result of the second generated object and the labeled segmentation result of the second reference object.

[0127] The velocity field loss value is the loss between the second velocity field parameters and the target velocity field parameters, used to ensure that the generated trajectory of the second generated object approximates the true trajectory (represented by the target velocity field parameters) evolving from the noisy state to the second reference object. For example, the velocity field loss value can be the mean square error, i.e., the mean square error between the velocity field predicted by the model and the true velocity field.

[0128] Based on this implementation, a two-stage training strategy is achieved by training the basic generative capabilities of the network module using paired data in a supervised learning phase, and then further enhancing the model's generalization ability using unpaired data in a reinforcement learning phase. This strategy enables the first image processing model to first learn the basic generative distribution, and then optimize the generative trajectory through reinforcement learning, achieving high-quality example-driven generation even under unpaired data conditions.

[0129] In one implementation, the second training sample further includes second object type information, which is used to indicate the type of the second reference object.

[0130] The second object type information is used to introduce semantic conditions for object types during the supervised learning phase, enabling the generative network module to learn specific features of different object types while learning basic generative capabilities. For example, the second object type information can be a textual description of tumor types (such as "gastric cancer," "liver cancer," "kidney cancer," etc.), which can be semantically encoded by a text encoder and then input into the generative network module.

[0131] Based on this implementation, by introducing second object type information into the second training samples, the generative network module learns to control the generated content according to the object type information during the supervised learning stage, laying the foundation for quality assessment based on the object type dimension in the subsequent reinforcement learning stage.

[0132] In a specific example within a medical imaging scenario, such as Figure 6 As shown, the second target region information example is a bounding box, the second sample image example is a CT image containing a tumor, and the second reference object information example is a mask for the reference tumor. The second sample image and the second reference object come from the same CT image containing a tumor. Therefore, the second reference object information and the second sample image constitute paired training data.

[0133] Specifically, in the SL stage, a large number of tumor CT scans with segmentation labels are used for flow matching training similar to repair, enabling the generative network module to learn the basic tumor distribution. For example... Figure 6 As shown, the SL stage uses a frozen VAE encoder to train the generative network module (including a frozen text encoder and a trainable U-net). The construction and processing flow of the second training sample is as follows: The initial image is a CT scan image of a tumor containing lesions. This initial image is input to the frozen VAE encoder, which outputs the corresponding latent variables after encoding. Simultaneously, the lesion structure region is extracted from it to form the extracted image of the second reference object. Given a tumor CT image as the initial image... and corresponding tumor reference examples as second reference information. Using the frozen VAE encoder Projecting visual information into the latent space, CT latent variables Second reference object hidden variable It can be obtained through the following formula: .

[0134] in, Represents the initial image via VAE encoder The latent variables obtained after encoding Indicates information about the second reference object After data augmentation transformation Then by VAE encoder The latent variables obtained through encoding. This represents a random data augmentation transformation, including centering, rotation, and scaling operations. This transformation is designed to enable the first image processing model to flexibly utilize reference tumor information to generate a plausible tumor, rather than simply performing a "copy-paste" operation.

[0135] Meanwhile, based on the initial image Based on the actual location of the lesion structure, the bounding box surrounding the lesion is derived. The bounding box After downsampling and channel expansion, information about the second target area is formed. , making its dimensions and Alignment. Second target area information. The region used to identify the area of ​​the second generated object in the second sample image is also a binary mask aligned with the latent variable space. Subsequently, based on the information of the second target region... noise CT latent variables Injecting noise into a portion of the image generates a latent representation for the second sample image. This operation is achieved through the following formula: .

[0136] in, This represents the latent representation of the second sample image after injecting noise into a portion of the region. Indicates information about the second target area. Represents the CT latent variables of the initial image. This represents a latent variable in random noise. This indicates element-wise multiplication. The formula represents the mask for the non-target region, using information from the second target region. As a mask, the original latent variable values ​​in non-target regions are preserved, while those in target regions are replaced with random noise latent variables, thereby constructing a conditional input with local destruction.

[0137] The hidden representation of the second sample image Second reference object information Second target area information The main input features of U-net are formed by associating features through a pairing mechanism and fusing them through channel-by-channel addition. The text encoder represents the object type of the second reference object as an embedding vector as additional conditional input to the U-net network. After receiving the fused features, U-net outputs the second velocity field parameters, representing the continuous generation trajectory from the noisy state to the second generated object. Based on the second velocity field parameters, the second predicted image and the second generated object information can be reconstructed by solving stochastic differential equations and combining them with a VAE decoder; wherein, the second generated object information is the mask of the generated lesion, and the second predicted image is the complete image after the second generated object is generated in the second target region of the second sample image.

[0138] In the model optimization stage, the target loss value used in step S503 includes two parts: a segmentation loss value, i.e., the Dice loss between the second generated object information and the second reference object information, used to constrain the generated object to be consistent with the true value in pixel-level segmentation results; and a velocity field loss value, i.e., the mean square error between the second velocity field parameters and the target velocity field parameters, wherein the target velocity field parameters are derived from the latent variables of the second reference object. An ideal linear interpolation path is derived between the noisy latent state and the target trajectories to supervise the physical plausibility of the generated trajectories. Finally, the weighted total loss of the segmentation loss and velocity field loss is minimized through backpropagation, updating only the U-net model parameters while keeping the VAE encoder, text encoder, and related preprocessing modules frozen.

[0139] Figure 7 A flowchart of an image processing method 700 provided in an embodiment of this application is shown. This method 700 can be applied to a server for the inference phase of a first image processing model. Figure 7 As shown, the method 700 may include steps S701 and S702.

[0140] Step S701: Obtain the target image, third target region information and third reference object information. The third target region information is used to identify the third target region in the target image, and the third reference object information is used to represent the third reference object.

[0141] The target image is the image to be processed during the inference stage. In a medical image scenario, the target image can be a 3D CT image of a healthy anatomical structure that does not contain lesions. Third target region information is used to indicate the third target region of the third generated object to be generated within the target image. Third reference object information is used to represent the third reference object, which can be a real object extracted from other medical images (such as a real tumor).

[0142] Step S702: Input the target image, the third target region information, and the third reference object information into the first image processing model to obtain the third predicted image and the third generated object information output by the first image processing model. The third predicted image is an image obtained by generating the third generated object in the third target region. The third generated object information is used to represent the third generated object. The third generated object is used to reconstruct the third reference object. The first image processing model is trained based on the method 300 of the embodiment of this application.

[0143] During the inference phase, the first image processing model can generate a third generated object with features similar to the third reference object within the third target region specified by the third target region information, using the third reference object represented by the third reference object information as a high-dimensional pathological template. This results in the output of a third predicted image (such as a 3D CT image of the lesion structure). The third generated object information represents the third generated object and includes, but is not limited to, images representing the third generated object, feature vectors / embedding representations of the third generated object, and masks of the third generated object. While preserving the true features of the third reference object, the third generated object naturally adapts to the background features of the target image in the third predicted image.

[0144] In a specific example within a medical imaging scenario, such as Figure 8 As shown, the third target region information is first downsampled and channel expanded to generate a mask aligned with the latent space dimension. This mask is combined with the CT latent variables encoded by the frozen VAE encoder of the target image. By injecting noise into some regions, a latent representation of the third target region after noise destruction is constructed. The original latent variable values ​​are retained in the non-third target regions, while those in the third target region are replaced with random noise latent variables. Simultaneously, the third reference object information is transformed by random data augmentation (including centering, rotation, and scaling) and encoded by the same frozen VAE encoder to obtain the second reference object latent variables. This transformation aims to enable the model to flexibly utilize the reference tumor (third reference object) to generate a reasonable tumor (third generated object), rather than simply performing a copy-paste operation. In addition, the text encoder encodes the input object type (such as tumor type) into an object type embedding vector, which is fed into the generator network module to perform multi-step SDE sampling and output velocity field estimation. The final latent state is obtained by integrating along the generation trajectory. Subsequently, this latent state is input into the VAE decoder to reconstruct the third predicted image, i.e., a complete CT image containing the generated tumor.

[0145] Therefore, the first image processing model can generate tumors with highly similar morphology, texture and object type in a specified third target region of the target image, using the information of the third reference object as a high-dimensional pathological template.

[0146] Experiments show that the first image processing model exhibits excellent distribution fidelity on multiple internal and external medical datasets. Its generated tumors significantly outperform existing baseline methods in the Fréchet Inception Distance (FID) metric, and the enhanced training set using this generated tumor effectively improves the cross-site generalization performance of downstream segmentation models. Furthermore, by introducing a joint training strategy combining RL and SL, the model's generalization ability is further enhanced. In the RL training phase, by configuring reward functions from morphological, texture, and object type dimensions, the first image processing model successfully overcomes the competition between shape and segmentation quality caused by a single reward function. This not only eliminates boundary artifacts and enhances texture realism but also ensures that the generated tumor maintains a high degree of morphological similarity to the reference tumor rather than being a simple copy. The generated tumor blends naturally into the anatomical background of the target image, preserving the true pathological features of the reference tumor while adapting to the host anatomical structure, thus effectively supporting the training of downstream segmentation models and alleviating the problem of scarce real-labeled data.

[0147] Figure 9 A flowchart illustrating a training method 900 for an image processing model provided in an embodiment of this application is shown. This method 900 can be applied to a server to train a downstream second image processing model. Figure 9 As shown, the method 900 may include: step S901, obtaining a third predicted image containing a third generated object output by a first image processing model, wherein the third predicted image is obtained based on method 700; step S902, inputting the third predicted image into a second image processing model to be trained, and obtaining a predicted segmentation result for the third generated object output by the second image processing model; step S903, training the second image processing model using the loss value between the predicted segmentation result and the labeled segmentation result of the third generated object.

[0148] The second image processing model is a model that performs segmentation tasks, such as the U-Net network, a deep convolutional neural network semantic segmentation model, a segmentation model based on a Transformer segmentation architecture, or other convolutional neural networks specifically designed for medical image segmentation.

[0149] The second image processing model, as a downstream task model of the first image processing model, can use the third predicted image output by the first image processing model as a training sample, thereby effectively solving the technical problems of scarce labeled data, high acquisition costs, and difficulty in covering rare samples in real-world scenarios.

[0150] Figure 10 A flowchart illustrating a computer-aided diagnostic method 1000 provided in an embodiment of this application is shown. This method 1000 can be applied to a server. Figure 10As shown, the method 1000 may include: step S1001, acquiring a medical image containing the screening object; step S1002, inputting the medical image into a second image processing model to obtain a predicted segmentation result for the screening object output by the second image processing model, wherein the second image processing model is trained based on method 900; and step S1003, detecting whether there are any abnormalities in the screening object based on the predicted segmentation result. For specific implementation methods and technical effects, please refer to the introduction in the application scenario section.

[0151] Corresponding to the application scenario and method 300 provided in the embodiments of this application, the embodiments of this application also provide a training device for an image processing model, including: a first training sample acquisition module, used to acquire a first training sample, the first training sample including a first sample image, first target region information and first reference object information, the first target region information being used to identify a first target region in the first sample image, the first reference object information being used to represent a first reference object, the first reference object being from an image different from the first sample image; a first training sample input module, used to input the first training sample into a first image processing model to be trained, to obtain multiple candidate results output by the first image processing model, wherein each candidate result includes a first predicted image, first generated object information and a first velocity field parameter, the first predicted image... The first image is an image obtained to generate a first generated object in the first target region of the first sample image. The first generated object information is used to represent the first generated object, the first generated object is used to reconstruct the first reference object, and the first velocity field parameter is used to represent the generation trajectory evolving from a noisy state to the first generated object. The training weight determination module is used to determine the training weight of any candidate result based on the first predicted image, the first generated object information, the first sample image, and the first reference object information. The training weight is used to represent the degree of deviation of the quality evaluation value of the candidate result from the quality evaluation values ​​of the plurality of candidate results. The first training module is used to train the first image processing model to adjust the generation trajectory based on the training weights corresponding to each candidate result.

[0152] In one implementation, the training weight determination module is specifically used to: determine the quality evaluation values ​​corresponding to the candidate results under multiple evaluation dimensions based on the first predicted image, the first generated object information, the first sample image, and the first reference object information; for any evaluation dimension, calculate the normalized result of the quality evaluation value under the evaluation dimension based on the quality evaluation values ​​corresponding to each candidate result under the evaluation dimension, and determine the training weight of the candidate result under the evaluation dimension based on the normalized result of the quality evaluation value; and determine the training weight of the candidate result according to the training weights corresponding to each evaluation dimension.

[0153] In one implementation, the training weight determination module is specifically used for at least one of the following: extracting a first shape feature of the first generated object and a second shape feature of the first reference object using a pre-trained feature extraction model, and determining the quality evaluation value of the candidate result in the shape dimension based on the distance between the first shape feature and the second shape feature; determining a first pixel value distribution of the first generated object and a second pixel value distribution of the first reference object, and determining the quality evaluation value of the candidate result in the texture dimension based on the similarity between the first pixel value distribution and the second pixel value distribution; determining a segmentation result of the object type belonging to the first reference object in the first predicted image, and determining the quality evaluation value of the candidate result in the object type dimension based on the segmentation loss value between the segmentation result and the first reference object information, wherein the first training sample further includes first object type information, which is used to represent the object type of the first reference object.

[0154] In one implementation, the first training module is specifically used to: for each candidate result, determine the adjustment direction of the generation trajectory corresponding to the candidate result based on the relationship between the training weight of the candidate result and the comparison benchmark, and adjust the probability distribution of each generation step on the generation trajectory according to the adjustment direction, wherein the comparison benchmark is the mean of the training weights of the multiple candidate results; wherein, when the training weight is greater than the comparison benchmark, the adjustment direction is closer to the distribution mean; when the training weight is less than the comparison benchmark, the adjustment direction is farther from the distribution mean, wherein the distribution mean is the mathematical expectation of the probability distribution of each generation step on the generation trajectory.

[0155] In one embodiment, the first image processing model includes a generator network module and an adapter module. During the training of the first image processing model to adjust the generated trajectory, the model parameters of the generator network module are frozen, and the model parameters of the adapter module are updated.

[0156] In one embodiment, the device further includes: a second training sample acquisition module, configured to acquire a second training sample, the second training sample including a second sample image, second target region information, and second reference object information, wherein the second sample image and the second reference object are from the same image; a second training sample input module, configured to input the second training sample into a generator network module to be trained, and obtain the prediction result output by the generator network module; and a second training module, configured to adjust the model parameters of the generator network module using a target loss value, wherein the target loss value includes: a segmentation loss value between the second generated object information and the second reference object information, and a velocity field loss value between the second velocity field parameter and the target velocity field parameter.

[0157] In one implementation, the second training sample further includes second object type information, which is used to represent the type of the second reference object.

[0158] In one embodiment, the first reference object is a lesion structure, and the first sample image includes the anatomical structure to which the lesion structure belongs but does not include the lesion structure itself.

[0159] Corresponding to the application scenario and method 700 provided in the embodiments of this application, the embodiments of this application also provide an image processing apparatus, including: a target data acquisition module, used to acquire a target image, third target region information and third reference object information; and an output result acquisition module, used to input the target image, the third target region information and the third reference object information into a first image processing model to obtain a third predicted image and third generated object information output by the first image processing model, wherein the third predicted image is an image obtained by generating a third generated object in the third target region, the third generated object is used to reconstruct the third reference object, and the first image processing model is trained based on the training method 300 of the embodiments of this application.

[0160] Corresponding to the application scenario and method 900 provided in the embodiments of this application, the embodiments of this application also provide a training device for an image processing model, including: a prediction image acquisition module, used to acquire a third prediction image containing a third generated object output by a first image processing model; a segmentation module, used to input the third prediction image into a second image processing model to be trained, and obtain a prediction segmentation result for the third generated object output by the second image processing model; and a training module, used to train the second image processing model using the loss value between the prediction segmentation result and the labeled segmentation result of the third generated object.

[0161] Corresponding to the application scenario and method 1000 provided in the embodiments of this application, the embodiments of this application also provide a computer-aided diagnostic device, including: a medical image acquisition module, used to acquire a medical image containing a screening object; a segmentation result acquisition module, used to input the medical image into a second image processing model to obtain a predicted segmentation result for the screening object output by the second image processing model; and a detection module, used to detect whether there is an abnormality in the screening object based on the predicted segmentation result.

[0162] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0163] Figure 11 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 11 As shown, the electronic device includes a memory 1101 and a processor 1102. The memory 1101 stores a computer program that can run on the processor 1102. When the processor 1102 executes the computer program, it implements the method described in the above embodiments. The number of memories 1101 and processors 1102 can be one or more. In a specific implementation, the electronic device may also include a communication interface 1103 for communicating with external devices and performing data exchange and transmission.

[0164] In practical implementation, if the memory 1101, processor 1102, and communication interface 1103 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0165] Optionally, in a specific implementation, if the memory 1101, processor 1102, and communication interface 1103 are integrated on a single chip, then the memory 1101, processor 1102, and communication interface 1103 can communicate with each other through an internal interface.

[0166] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0167] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0168] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0169] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this application.

[0170] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0171] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0172] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0173] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0174] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0175] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0176] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0177] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0178] Furthermore, the functional units in the various embodiments of this application can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0179] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training an image processing model, comprising: A first training sample is obtained, the first training sample includes a first sample image, first target region information and first reference object information, the first target region information is used to identify the first target region in the first sample image, the first reference object information is used to represent the first reference object, and the first reference object comes from an image different from the first sample image. The first training sample is input into the first image processing model to be trained to obtain multiple candidate results output by the first image processing model. Each candidate result includes a first predicted image, first generated object information, and a first velocity field parameter. The first predicted image is an image obtained by generating a first generated object in the first target region of the first sample image. The first generated object information is used to represent the first generated object. The first generated object is used to reconstruct the first reference object. The first velocity field parameter is used to represent the generation trajectory that evolves from a noisy state to the first generated object. For any candidate result, a training weight is determined based on the first predicted image, the first generated object information, the first sample image, and the first reference object information, wherein the training weight is used to represent the degree of deviation of the quality evaluation value of the candidate result from the quality evaluation values ​​of the plurality of candidate results; The first image processing model is trained to adjust the generated trajectory based on the training weights corresponding to each candidate result.

2. The method according to claim 1, wherein, The step of determining the training weights of the candidate results based on the first predicted image, the first generated object information, the first sample image, and the first reference object information includes: Based on the first predicted image, the first generated object information, the first sample image, and the first reference object information, the quality evaluation values ​​corresponding to the candidate results under multiple evaluation dimensions are determined respectively. For any evaluation dimension, based on the quality evaluation value corresponding to each candidate result under the evaluation dimension, calculate the normalized result of the quality evaluation value under the evaluation dimension, and determine the training weight of the candidate result under the evaluation dimension based on the normalized result of the quality evaluation value. The training weights of the candidate results are determined based on the training weights corresponding to each of the evaluation dimensions.

3. The method according to claim 2, wherein, The step of determining the quality evaluation value of the candidate result under multiple evaluation dimensions based on the first predicted image, the first generated object information, the first sample image, and the first reference object information includes at least one of the following: Using a pre-trained feature extraction model, the first shape features of the first generated object and the second shape features of the first reference object are extracted respectively. Based on the distance between the first shape features and the second shape features, the quality evaluation value of the candidate result in the shape dimension is determined. The first pixel value distribution of the first generated object and the second pixel value distribution of the first reference object are determined respectively, and the quality evaluation value of the candidate result in the texture dimension is determined based on the similarity between the first pixel value distribution and the second pixel value distribution. The segmentation result of the object type belonging to the first reference object in the first predicted image is determined, and the quality evaluation value of the candidate result in the object type dimension is determined according to the segmentation loss value between the segmentation result and the first reference object information. The first training sample further includes first object type information, which is used to represent the object type of the first reference object.

4. The method according to claim 1, wherein, The step of training the first image processing model to adjust the generated trajectory according to the training weights corresponding to each of the candidate results includes: For each candidate result, the adjustment direction of the generation trajectory corresponding to the candidate result is determined according to the relationship between the training weights of the candidate result and the comparison benchmark. Then, the probability distribution of each generation step on the generation trajectory is adjusted according to the adjustment direction. The comparison benchmark is the average of the training weights of the multiple candidate results. Wherein, when the training weight is greater than the comparison benchmark, the adjustment direction is closer to the distribution mean; when the training weight is less than the comparison benchmark, the adjustment direction is farther from the distribution mean, where the distribution mean is the mathematical expectation of the probability distribution of each generation step on the generation trajectory.

5. The method according to claim 1, wherein, The first image processing model includes a generator network module and an adapter module. During the training of the first image processing model to adjust the generated trajectory, the model parameters of the generator network module are frozen and the model parameters of the adapter module are updated.

6. The method according to claim 5, wherein, The method further includes: A second training sample is obtained, which includes a second sample image, second target region information and second reference object information. The second target region information is used to identify the second target region in the second sample image, and the second reference object information is used to represent the second reference object. The second sample image and the second reference object come from the same image. The second training sample is input into the generator network module to be trained to obtain the prediction result output by the generator network module. The prediction result includes a second prediction image, second generated object information and a second velocity field parameter. The second prediction image is an image obtained by generating a second generated object in the second target region of the second sample image. The second generated object information is used to represent the second generated object. The second generated object is used to reconstruct the second reference object. The second velocity field parameter is used to represent the generation trajectory that evolves from the noise state to the second generated object. The model parameters of the generator network module are adjusted using a target loss value, which includes: a segmentation loss value between the second generated object information and the second reference object information, and a velocity field loss value between the second velocity field parameter and the target velocity field parameter. The target velocity field parameter is used to represent the generated trajectory evolving from the noise state to the second reference object.

7. The method according to claim 6, wherein, The second training sample also includes second object type information, which is used to represent the type of the second reference object.

8. The method according to any one of claims 1 to 7, wherein, The first reference object is the lesion structure, and the first sample image includes the anatomical structure to which the lesion structure belongs but does not include the lesion structure itself.

9. An image processing method, comprising: Acquire a target image, third target region information, and third reference object information, wherein the third target region information is used to identify the third target region in the target image, and the third reference object information is used to represent the third reference object; The target image, the third target region information, and the third reference object information are input into the first image processing model to obtain the third predicted image and the third generated object information output by the first image processing model. The third predicted image is an image obtained by generating a third generated object in the third target region of the target image. The third generated object information is used to represent the third generated object. The third generated object is used to reconstruct the third reference object. The first image processing model is trained based on the method described in any one of claims 1 to 8.

10. A method for training an image processing model, comprising: Obtain a third predicted image containing a third generated object, output by a first image processing model, wherein the third predicted image is obtained based on the method described in claim 9; The third predicted image is input into the second image processing model to be trained, and the predicted segmentation result of the second image processing model for the third generated object is obtained. The second image processing model is trained using the loss value between the predicted segmentation result and the labeled segmentation result of the third generated object.

11. A computer-aided diagnostic method, comprising: Acquire medical images containing the subjects to be screened; The medical image is input into a second image processing model to obtain the predicted segmentation result of the second image processing model for the screening object. The second image processing model is trained based on the method described in claim 10. The screening object is detected to have any abnormalities based on the predicted segmentation results.

12. An image processing apparatus, comprising: The target data acquisition module is used to acquire the target image, third target region information, and third reference object information; The output result acquisition module is used to input the target image, the third target region information and the third reference object information into the first image processing model to obtain the third predicted image and the third generated object information output by the first image processing model. The third predicted image is an image obtained by generating a third generated object in the third target region. The third generated object is used to reconstruct the third reference object. The first image processing model is trained based on the method of any one of claims 1 to 8.

13. A computer-aided diagnostic system, comprising: A client is used to send an anomaly screening request, the anomaly screening request carrying a medical image containing the screening object, the anomaly screening request being used to detect whether the screening object has any anomalies; The server is configured to receive the anomaly screening request from the client, input the medical image into a second image processing model, obtain the predicted segmentation result of the second image processing model for the screening object, detect whether the screening object has an anomaly based on the predicted segmentation result, and send the detection result to the client. The second image processing model is trained based on the method described in claim 10, and the detection result is used to characterize whether the screening object has an anomaly. The client is also used to output the detection results.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 11.

15. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 11.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1 to 11.

Images